A recent study has found that half of the medical advice provided by popular artificial intelligence chatbots contains problematic information that could potentially mislead users. Published in BMJ Open, the research evaluated five leading generative AI models by posing a series of open and closed questions related to health topics, including cancer, vaccines, stem cells, nutrition, and athletic performance.

The questions were carefully designed to reflect common inquiries as well as prevalent misinformation patterns encountered online. Researchers assessed the AI responses based on whether they might prompt individuals to pursue ineffective treatments or cause harm. According to the findings, 50% of answers were categorized as problematic, with 30% deemed somewhat problematic and 20% classified as highly problematic.

The study highlights ongoing concerns about the reliability of AI-driven medical information amid growing reliance on these technologies for health guidance. Experts caution that while AI has the potential to assist users in accessing health knowledge, significant risks remain if responses are inaccurate or misleading. This underscores the importance of professional medical advice and critical evaluation of AI-supplied information.

The research does not specify which of the five AI chatbots performed better or worse but emphasizes a broad vulnerability across the current generation of models. As AI tools become more widely integrated into everyday life, the findings suggest a need for improved safeguards, more rigorous training, and clearer communication to prevent harm resulting from erroneous health recommendations.