Healthcare

The Algorithm Will See You Now: Healthcare AI, Patient Rights, and the Race to Regulate

On January 1, 2026, Texas became the first US state to require that patients be told — before or at the moment of interaction — when artificial intelligence is involved in their diagnosis or treatment. Across the Atlantic, Europe was building something more ambitious.

P J Laszkowicz

A patient walks into a clinic. An AI system has flagged an anomaly in their chest scan — a pattern that a human radiologist might have caught on the third or fourth review, but that the algorithm identified in seconds. The clinician reviews the AI's recommendation, agrees with the assessment, and orders a biopsy. The patient is told they need further testing. They are not told that a machine made the initial finding.

Under most jurisdictions in the world, there is no legal requirement that they be told. Under Texas law, as of January 1, 2026, there is.

The Texas Responsible AI Governance Act — TRAIGA — signed into law on June 22, 2025, requires providers of healthcare services or treatments to make "clear and conspicuous" disclosure to patients when artificial intelligence systems are used in their care.1 The disclosure must be written in plain language, free of dark patterns, and delivered no later than the date the service or treatment is first provided — or, in emergencies, as soon as reasonably possible.2 The principle is straightforward: patients have a right to know when an algorithm is involved in decisions about their health.

Texas was not the first jurisdiction to consider the question. But it was the first to answer it with a binding legal obligation. And the answer it gave — transparency — was only one of several possible responses to a problem that was rapidly outpacing the regulatory frameworks designed to address it. Across four major regulatory jurisdictions — US state law, the European Union, the United Kingdom, and the US federal government through the FDA — different approaches to healthcare AI were taking shape in late 2025 and early 2026. Each reflected a different theory of what made AI in healthcare risky, and each proposed a different combination of disclosure, classification, testing, and oversight to manage that risk.

None of them had solved the problem. But together, they mapped the territory in which the solution would eventually be found.

The Texas precedent

TRAIGA was the product of a legislative session that saw AI governance emerge as a bipartisan priority in Texas. The healthcare disclosure provision was one element of a broader framework that addressed AI governance across sectors, but it was the provision that attracted the most attention — partly because of its novelty, and partly because it touched a domain where the stakes of AI decision-making were most immediately personal.3

The law's disclosure requirement was deliberately narrow. It did not regulate which AI systems could be used in healthcare, or impose technical standards on their performance. It did not require that AI systems be tested for bias before deployment, or that their outputs be validated against clinical evidence. It required one thing: that patients be told.4

The narrowness was both a strength and a limitation. As a transparency measure, TRAIGA established a clear, enforceable obligation that was simple for healthcare providers to understand and implement. It created an informational baseline: once patients knew that AI was involved in their care, they could ask questions, seek second opinions, or exercise whatever agency the clinical context allowed. The Akerman LLP analysis noted that by January 1, 2026, healthcare providers in Texas needed to have disclosure protocols in place that covered every point of care where AI systems were used — from diagnostic imaging to treatment recommendations to administrative triage.5

But disclosure, by itself, addressed only one dimension of the problem. A patient told that an AI system was involved in their diagnosis had no way of knowing whether that system was accurate, whether it performed equally well across demographic groups, or whether its recommendations had been validated in clinical trials. The disclosure obligation created a right to know. It did not create a right to know whether the algorithm was trustworthy.

Europe's dual approach

The European Union pursued a fundamentally different strategy: combining investment in healthcare AI development with regulatory requirements designed to ensure that the systems developed were safe, unbiased, and subject to human oversight.

On the investment side, the EU committed close to 700 million euros to healthcare AI through the GenAI4EU initiative — a programme funded through Horizon Europe and the Digital Europe Programme that targeted seven strategic sectors, with healthcare among the most prominent.6 The programme supported the development of generative AI applications for clinical use, including diagnostic support, drug discovery, and personalised treatment planning. The Testing and Experimentation Facility for Health, co-funded under the Digital Europe Programme, provided infrastructure for small and medium-sized enterprises to test AI solutions in real-world clinical environments and verify compliance with both the AI Act and medical device regulations before bringing products to market.7

On the regulatory side, the EU AI Act classified most healthcare AI systems as "high-risk" — a designation that triggered a comprehensive set of obligations. High-risk AI systems were required to be trained, validated, and tested on datasets that were "relevant, representative, free of errors, and complete," with adequate representation of the intended patient population so that results could reasonably be generalised across demographic groups.8 Systems had to implement risk-management processes throughout their lifecycle, document their technical characteristics and performance, provide clear information to users about their capabilities and limitations, and be designed to enable effective human oversight.9

The obligations were among the most detailed AI governance requirements in the world. But they were not yet fully operational. The AI Act entered force in stages, with the high-risk requirements applying from August 2026 — more than two years after the legislation was adopted. The harmonised standards that would define how manufacturers demonstrated compliance were still being developed by European standardisation bodies. In the interim, healthcare AI systems continued to be regulated primarily through the Medical Device Regulation, which addressed safety and performance but had not been designed with AI-specific challenges like dataset bias and model drift in mind.10

The European approach reflected a conviction that transparency alone was insufficient — that patients were protected not by being told that AI was involved, but by ensuring that the AI systems involved met defined standards of quality, fairness, and accountability. Whether those standards, once fully implemented, would deliver on that promise remained an open question.

The FDA's evolving framework

In the United States, the Food and Drug Administration occupied a middle ground between Texas's transparency mandate and Europe's comprehensive classification system. The FDA had been reviewing AI-enabled medical devices since 2017 and had, by early 2025, authorised more than a thousand AI-enabled devices across specialties including radiology, cardiology, ophthalmology, and pathology.11 But the regulatory framework — designed for traditional medical devices with fixed functionality — was under increasing strain from AI systems that learned, adapted, and changed over time.

On January 7, 2025, the FDA published draft guidance titled "Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations."12 The guidance articulated a Total Product Lifecycle approach to AI-enabled devices — a framework that treated the regulation of an AI medical device not as a one-time approval decision but as a continuous process spanning design, development, deployment, and post-market monitoring.

The guidance was notable for several reasons. First, it explicitly addressed bias. The FDA recommended that manufacturers include bias analysis and mitigation throughout the device lifecycle — from ensuring that training and testing data reflected the intended use population to proactively identifying performance disparities across demographic groups including race, ethnicity, sex, and age.13 This was not a vague aspiration. The guidance recommended that marketing submissions include specific evidence demonstrating that the device performed equitably across relevant populations.

Second, the guidance addressed transparency — not at the patient level, as Texas had done, but at the regulatory level. Manufacturers were expected to document model descriptions, data lineage, training methodologies, and the human-AI workflow in which the device would be used.14 The aim was to give the FDA sufficient information to evaluate not just whether a device worked, but how it worked, on what data it had been trained, and how it was intended to interact with clinical decision-making.

Third, the guidance acknowledged the challenge of AI systems that changed after deployment. Traditional medical device regulation assumed a fixed product: a device was approved in a specific configuration, and modifications required new regulatory review. AI systems that learned from new data or updated their models did not fit this paradigm. The Total Product Lifecycle approach was designed to address this, but the operational details — how much change required new review, what monitoring obligations applied to deployed systems, how manufacturers should document performance drift — remained works in progress.15

The American Hospital Association, in its public comments on the draft guidance, raised concerns about the pace of AI deployment relative to the pace of regulatory development — noting that AI systems were being integrated into clinical workflows faster than the FDA could evaluate them, and that the draft guidance, while welcome, would take years to finalise and implement.16

The UK's regulatory sandbox

The United Kingdom took yet another approach: iterative experimentation through a regulatory sandbox. The Medicines and Healthcare products Regulatory Agency's AI Airlock programme, launched as a pilot in April 2024, created a structured environment in which AI medical device developers could test their products under regulatory supervision before seeking formal approval.17

The pilot phase, which ran through March 2025, partnered the MHRA with four companies working on diverse AI healthcare applications: Philips Healthcare on large language models for clinical use, AutoMedica on automated clinical decision support, OncoFlow on cancer care pathways, and Newton's Tree on continuous monitoring.18 The programme explored practical questions that formal regulation struggled to address: how to generate new datasets for testing AI in clinical settings, how to make large language models transparent enough for healthcare use, and how to monitor AI products continuously once deployed.

In March 2025, the government confirmed that the AI Airlock would run a second phase during the 2025–2026 financial year. Seven technologies were selected for Phase 2, spanning AI-powered clinical note-taking, advanced cancer diagnostics, eye disease detection, and obesity treatment support.19 The current phase was scheduled to run until March 2026.

The sandbox approach reflected a regulatory philosophy that differed from both the American and European models. Where the FDA sought to adapt existing device regulation to AI through revised guidance, and the EU sought to classify and regulate AI through comprehensive legislation, the UK sought to learn by doing — testing regulatory approaches alongside the technologies they were meant to govern, and adapting the framework based on what worked in practice.20

The limitation was scale. A sandbox that accommodated four companies in its first year and seven in its second could not, by itself, regulate an industry deploying AI across thousands of clinical applications. The AI Airlock was a laboratory for regulatory innovation, not a regulatory system. Its value depended on whether the insights it generated would be translated into formal regulatory requirements — and whether the MHRA had the institutional capacity and political support to do so.

The bias problem

Across all four regulatory approaches, one issue appeared with uncomfortable consistency: AI systems trained on unrepresentative data produced unequal outcomes, and no jurisdiction had yet developed a fully effective response.

The evidence was particularly stark in dermatology. A study published in the Journal of the European Academy of Dermatology and Venereology found that among four thousand AI-generated dermatological images, only 10.2 per cent reflected dark skin tones, and only fifteen per cent accurately depicted the intended condition.21 Research in Science Advances documented substantial performance limitations in state-of-the-art dermatology AI models when tested on diverse datasets, with diagnostic accuracy dropping dramatically for dark skin tones and uncommon diseases.22 A Northwestern University study found that when physicians used AI assistance for dermatological diagnosis, accuracy improved more for light skin tones than dark ones — meaning that AI assistance actually widened the diagnostic disparity by five percentage points rather than narrowing it.23

The pattern extended beyond dermatology. Cardiology AI models had demonstrated gender-based performance disparities. Triage systems trained predominantly on data from high-income healthcare settings performed less reliably in under-resourced environments. The consistent finding across specialties was that AI systems inherited and sometimes amplified the biases present in their training data — and that the training data available for medical AI was disproportionately drawn from populations that were white, male, and treated in well-resourced Western healthcare systems.24

Each regulatory jurisdiction acknowledged the problem. The FDA's draft guidance explicitly required bias analysis across demographic groups. The EU AI Act required representative datasets and bias testing for high-risk systems. The MHRA's AI Airlock examined how to generate diverse datasets for testing. Texas's disclosure requirement at least ensured that patients knew AI was involved, creating the possibility of informed questioning.

But none of these measures, individually or collectively, addressed the root cause: the datasets on which medical AI was trained did not represent the populations on which it would be used. Fine-tuning models on diverse data could close performance gaps, as research had demonstrated.25 But generating diverse medical datasets was expensive, logistically complex, and dependent on the participation of healthcare systems in communities that had historically been underserved and had reason to distrust both the healthcare system and the technology industry.

Four frameworks, one question

The four regulatory approaches that had crystallised by early 2026 each answered a different aspect of the same question: how should healthcare AI be governed?

Texas answered with transparency. Patients had a right to know when AI was involved in their care. The disclosure obligation was clear, enforceable, and implementation-ready. But it placed the burden of evaluation on patients who typically lacked the expertise to assess AI system quality.

The European Union answered with classification and standards. Healthcare AI systems were subject to comprehensive requirements for data quality, bias testing, risk management, and human oversight. The framework was the most thorough in the world but would not be fully operational until 2026 at the earliest, and its effectiveness depended on harmonised standards that were still being developed.

The FDA answered with lifecycle management. AI medical devices were subject to continuous regulatory oversight from development through post-market deployment, with explicit requirements for bias analysis and performance documentation. The approach was technically sophisticated but depended on guidance that remained in draft form and an approval process that struggled to keep pace with deployment.

The United Kingdom answered with experimentation. The AI Airlock tested regulatory approaches in real-world clinical environments, generating practical insights that formal regulation could draw upon. But the sandbox's small scale limited its direct impact, and the translation of experimental findings into binding requirements remained uncertain.

No single approach was sufficient. Transparency without standards told patients something was happening but not whether it was safe. Standards without transparency created accountability structures that patients could not see or engage with. Lifecycle management without capacity created regulatory ambitions that could not be fulfilled. Experimentation without scale produced insights that could not yet be applied broadly.

Trust and the algorithm

Healthcare AI will expand. The economic incentives — faster diagnosis, reduced clinician workload, broader access to specialist expertise — are too powerful, and the technology too capable, for adoption to slow. The question is not whether AI will be integrated into healthcare but whether patients will trust it, and whether they should.

Trust requires three things: transparency, accountability, and evidence. Different jurisdictions are building different combinations of these elements. Texas has chosen transparency. Europe has chosen accountability through classification. The FDA has chosen evidence through lifecycle documentation. The UK is generating all three through experimentation at small scale.

The missing element, across all four frameworks, is integration. A patient in Texas in January 2026 was told that AI was involved in their diagnosis. They were not told whether that AI system met European data quality standards, whether it had been evaluated by the FDA for demographic bias, or whether its clinical performance had been tested in the MHRA's sandbox. The information existed — or was being generated — in separate regulatory systems that did not communicate with each other and were not designed to.

The challenge for the next phase of healthcare AI governance is not to choose among these approaches but to connect them: to build regulatory frameworks in which transparency, accountability, and evidence reinforce each other, and in which the protections available to a patient do not depend on the jurisdiction in which they happen to seek care. That integration does not yet exist. But the building blocks — each imperfect, each incomplete — are being laid, on both sides of the Atlantic, one regulatory experiment at a time.

Footnotes

  1. Norton Rose Fulbright, "The Texas Responsible AI Governance Act: What Your Company Needs to Know before January 1," June 2025. ↩

  2. Holland & Knight, "Texas Enacts Comprehensive AI Governance Laws with Sector-Specific Healthcare Provisions," June 2025. ↩

  3. Perkins Coie, "Texas Enacts Responsible Artificial Intelligence Governance Act," June 2025. ↩

  4. K&L Gates, "Pared Back Version of the Texas Responsible Artificial Intelligence Governance Act Signed into Law," June 2025. ↩

  5. Akerman LLP, "HRx: New Year, New AI Rules: Healthcare AI Laws Now in Effect," January 2026. ↩

  6. European Commission, "GenAI4EU: Funding Opportunities to Boost Generative AI 'Made in Europe,'" 2025. ↩

  7. European Commission, "Artificial Intelligence in Health," 2025. ↩

  8. EU AI Act, Article 10; Greenberg Traurig LLP, "EU Commission Consultation on High-Risk AI Systems: Key Points for Life Sciences and Health Care," June 2025. ↩

  9. EU AI Act, Articles 9, 11, 13, 14. ↩

  10. Hogan Lovells, "AI Health Law & Policy: Comparing Regulatory Landscapes for AI in Medical Devices in the EU and US," 2025. ↩

  11. Bipartisan Policy Center, "FDA Oversight: Understanding the Regulation of Health AI Tools," 2025. ↩

  12. FDA, "Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations; Draft Guidance," 7 January 2025. ↩

  13. CenterWatch, "FDA Guidance on AI-Enabled Devices: Transparency, Bias, & Lifecycle Oversight," January 2025. ↩

  14. Greenberg Traurig LLP, "FDA Releases Draft Guidance on AI-Enabled Medical Devices," January 2025. ↩

  15. DLA Piper, "FDA Issues Artificial Intelligence-Enabled Device Software Functions Draft Guidance," January 2025. ↩

  16. Sterne Kessler, "FDA Issues Draft Guidance Documents on Artificial Intelligence for Medical Devices, Drugs, and Biological Products," January 2025. ↩

  17. MHRA, "AI Airlock: The Regulatory Sandbox for AIaMD," 2024. ↩

  18. MedRegs, "Exploring AI in Healthcare: Insights from the AI Airlock Pilot," 26 March 2025. ↩

  19. MHRA, "AI Tools That Could Detect Diseases Earlier Selected for Next Phase of MHRA's 'AI Airlock' Programme," 2025. ↩

  20. MedRegs, "How the AI Airlock Is Charting a Path for Regulating AI in Healthcare through Sandboxes," 31 October 2025. ↩

  21. Joerg et al., "AI-Generated Dermatologic Images Show Deficient Skin Tone Diversity and Poor Diagnostic Accuracy: An Experimental Study," Journal of the European Academy of Dermatology and Venereology, 2025. ↩

  22. Daneshjou et al., "Disparities in Dermatology AI Performance on a Diverse, Curated Clinical Image Set," Science Advances, 2022. ↩

  23. Northwestern University, "Racial Bias Exists in Photo-Based Medical Diagnosis despite AI Help," February 2024. ↩

  24. Stanford HAI, "AI Shows Dermatology Educational Materials Often Lack Darker Skin Tones," 2024. ↩

  25. Practical Dermatology, "AI and Skin of Color: Hidden Biases Raise Questions," 2024. ↩