The U.S. Department of Health and Human Services (HHS) is advancing efforts to integrate artificial intelligence (AI) into medical care, focusing on projects that use AI to diagnose and prescribe treatments. This federal push, accelerated under the Trump administration, has generated concerns among some officials about the rapid pace of adoption and the sufficiency of evidence regarding AI’s safety and effectiveness.
At the center of these developments is Vinod Khosla, a Silicon Valley venture capitalist who has been influential in discussions with top HHS officials, including Medicare and Medicaid head Dr. Mehmet Oz. Khosla, whose son leads Curai Health, an AI-driven primary care company, has long predicted that AI will replace many physician roles. He has described AI as a “holy grail” in reducing health costs and expanding access, particularly for underserved populations.
Despite this enthusiasm, medical professionals and health advocates caution that AI models designed to independently deliver care remain premature. Dr. John Whyte, CEO of the American Medical Association (AMA), said the current evidence favoring AI is largely based on simulations and not ready for widespread clinical use. He emphasized that AI deployment should be guided by the clinical community rather than technology firms, warning that the tech sector currently wields disproportionate influence over healthcare decisions.
The Department of Health and Human Services, through Medicare and Medicaid Services (CMS), is exploring payment reforms to reimburse AI-based health technologies. These reforms may include new Medicare categories for AI software that supports care delivery, with an encouragement for private insurers to reimburse for technology-enabled services that demonstrate patient benefit and cost savings.
Meanwhile, the Food and Drug Administration (FDA) has approved over 1,500 medical devices incorporating AI to perform specific tasks, such as tumor detection. However, the agency currently lacks comprehensive regulations for emerging AI models that act with greater autonomy, including chatbots and AI systems capable of prescribing medications. The FDA is actively considering how to regulate these evolving technologies amid concerns about safety, such as risks seen with large language models providing incorrect or potentially dangerous medical advice.
Health Secretary Robert F. Kennedy Jr. has acknowledged these risks but remains optimistic about AI’s transformative potential in medicine. Notably, his son Finn Kennedy has launched a venture capital fund focused on health technology investments, including AI, and has advocated for structural changes within the FDA to oversee AI innovation.
Among pilot projects receiving federal support is Limbic, a company providing AI-driven cognitive behavioral therapy to Medicare beneficiaries with depression or anxiety. The AI counselor, named “Hope,” conducts weekly sessions and has been shown in a study published in Nature Medicine to perform comparably or better than human therapists when assessed by clinicians who were blinded to the nature of the provider. Limbic CEO Ross Harper argued that AI therapy addresses critical gaps in mental health access, stressing the ethical importance of delivering care to individuals facing long waiting lists or no treatment at all.
As federal agencies weigh regulatory frameworks and payment models, debate continues over balancing innovation with patient safety and the appropriate roles of the tech industry and clinical experts in shaping the future of AI in healthcare.