Healthcare

Patients are told AI was used, not that it works

Since 1 January 2026, Texas has been the first US state requiring that patients be told when artificial intelligence is involved in their diagnosis or treatment. It does not require anybody to establish that the system is any good.

P J Laszkowicz

A patient walks into a clinic. An AI system has flagged an anomaly in their chest scan, a pattern a human radiologist might have caught on the third or fourth review but that the algorithm identified in seconds. The clinician reviews the recommendation, agrees with the assessment, and orders a biopsy. The patient is told they need further testing. They are not told that a machine made the initial finding.

In most jurisdictions there is no legal requirement that they be told. Under Texas law, since 1 January 2026, there is.

The Texas Responsible AI Governance Act, known as TRAIGA, was signed into law on 22 June 2025 and requires providers of healthcare services or treatment to make a "clear and conspicuous" disclosure to patients when artificial intelligence is used in their care. The disclosure must be written in plain language, must not use a dark pattern, meaning an interface designed to steer someone toward a choice they would not otherwise make, and must arrive no later than the date the service is first provided, or in an emergency as soon as reasonably possible. The principle is straightforward: patients have a right to know when an algorithm is involved in decisions about their health.

Texas was not the first jurisdiction to consider the question, but it was the first to answer it with a binding legal obligation. The answer it gave, transparency, is only one of several possible responses to a problem outpacing the frameworks designed to address it. Four major jurisdictions took shape through late 2025 and early 2026: Texas state law, the European Union, the United Kingdom, and the United States federal government through the Food and Drug Administration. Each reflects a different theory of what makes AI in healthcare risky, and each proposes a different combination of disclosure, classification, testing, and oversight.

None has solved the problem. Together they set out the ground on which a solution will have to be found.

The Texas precedent

TRAIGA emerged from a legislative session in which AI governance became a bipartisan priority in Texas. The healthcare disclosure provision was one element of a broader framework covering AI across sectors, and it attracted the most attention, partly for its novelty and partly because it touches the domain where the stakes of algorithmic decision-making are most immediately personal.1

The requirement is deliberately narrow. It does not regulate which AI systems may be used in healthcare, nor impose technical standards on their performance. It does not require that systems be tested for bias before deployment, or that outputs be validated against clinical evidence. It requires one thing: that patients be told.2

That narrowness is both a strength and a limitation. As a transparency measure it establishes a clear, enforceable obligation that providers can understand and implement. It creates an informational baseline, since a patient who knows AI was involved can ask questions, seek a second opinion, or exercise whatever agency the clinical context allows. By 1 January 2026, Texas providers needed disclosure protocols covering every point of care where AI systems operate, from diagnostic imaging through treatment recommendations to administrative triage.3

Disclosure by itself addresses only one dimension of the problem. A patient told that an AI system contributed to their diagnosis has no way of knowing whether that system is accurate, whether it performs equally well across demographic groups, or whether its recommendations have been validated in clinical trials. The obligation creates a right to know that AI was used. It does not create a right to know whether the algorithm is trustworthy.

Europe's dual approach

The European Union pursues a different strategy, combining investment in healthcare AI with regulatory requirements intended to ensure the resulting systems are safe, unbiased, and subject to human oversight.

On the investment side, the Commission's GenAI4EU initiative carries close to 700 million euros across Horizon Europe, the Digital Europe Programme, and the European Innovation Council. That budget spans seven strategic sectors, including manufacturing, robotics, energy, agrifood, mobility, and aerospace alongside healthcare, so only a portion reaches clinical applications; individual healthcare projects draw grants in the range of fifteen to seventeen million euros. The programme supports diagnostic assistance, drug discovery, and personalised treatment planning. A Testing and Experimentation Facility for Health, co-funded under the Digital Europe Programme, gives smaller enterprises somewhere to test AI in real clinical environments and verify compliance with both the AI Act and medical device rules before going to market.4

On the regulatory side, the AI Act classifies most healthcare AI as high-risk, a designation triggering a comprehensive set of obligations. High-risk systems must be trained, validated, and tested on datasets that are "relevant, representative, free of errors, and complete", with adequate representation of the intended patient population so results generalise across demographic groups.5 Systems must carry risk management throughout their lifecycle, document their technical characteristics and performance, tell users what they can and cannot do, and be designed so humans can exercise meaningful oversight.6

These are among the most detailed AI governance requirements anywhere, and they are not yet fully operational. The Act entered force in stages, with the high-risk requirements applying from August 2026, more than two years after adoption.7 The harmonised standards defining how manufacturers demonstrate compliance are still being written by European standardisation bodies. In the meantime healthcare AI is regulated mainly through the Medical Device Regulation, which addresses safety and performance but was not designed with dataset bias or model drift in mind.8

The European approach rests on a conviction that transparency alone is insufficient, and that patients are protected less by being told AI was involved than by knowing the systems involved meet defined standards of quality, fairness, and accountability. Whether those standards deliver on that promise once fully implemented remains open.

The FDA's evolving framework

In the United States the Food and Drug Administration occupies a position between the Texas transparency mandate and Europe's classification system. It has reviewed AI-enabled medical devices since 2017 and had authorised more than 1,250 of them by early 2025, across radiology, cardiology, ophthalmology, and pathology, with radiology accounting for roughly three-quarters of the total.9 The framework, built for traditional devices with fixed functionality, is under strain from systems that learn, adapt, and change.

On 7 January 2025 the agency published draft guidance titled "Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations".10 It sets out a total product lifecycle approach, treating the regulation of an AI medical device not as a single approval decision but as a continuous process spanning design, development, deployment, and post-market monitoring.

The guidance is notable for three reasons. It addresses bias explicitly, recommending that manufacturers build bias analysis and mitigation into the whole device lifecycle, from ensuring training and testing data reflect the intended population to identifying performance disparities across race, ethnicity, sex, and age.11 This is not a vague aspiration: marketing submissions are expected to carry specific evidence that the device performs equitably across relevant populations.

It also addresses transparency, though at the regulatory rather than the patient level. Manufacturers are expected to document model descriptions, data lineage, training methods, and the human-AI workflow the device sits within.12 The aim is to give the agency enough information to evaluate not merely whether a device works but how it works, what it learned from, and how it is meant to interact with clinical judgement.

Third, it acknowledges systems that change after deployment. Traditional device regulation assumes a fixed product approved in a specific configuration, with modifications requiring fresh review. Systems that learn from new data do not fit that model. The lifecycle approach is designed to accommodate them, though the operational detail remains unsettled: how much change triggers new review, what monitoring applies to deployed systems, and how manufacturers should document performance drift.13

The American Hospital Association, commenting publicly on the draft, raised the mismatch between the pace of deployment and the pace of regulation, noting that AI is entering clinical workflows faster than the agency can evaluate it and that the guidance, however welcome, will take years to finalise.14

The UK's regulatory sandbox

The United Kingdom took a third route: iterative experimentation through a regulatory sandbox. The Medicines and Healthcare products Regulatory Agency launched its AI Airlock as a pilot in April 2024, creating a structured environment in which developers of AI medical devices could test products under regulatory supervision before seeking formal approval.15

The pilot ran through March 2025 and partnered the agency with four companies: Philips Healthcare, whose AutoImpression tool drafts the summary section of radiology reports; AutoMedica, whose SmartGuideline surfaces guideline-based information for clinicians; OncoFlow, working on oncology decision support; and Newton's Tree, on continuous monitoring. The programme examined questions formal regulation struggles with, including how to validate systems using synthetic data, how to make large language models explainable enough for clinical use, and how to handle emerging risks such as models generating confident but fabricated output.

In March 2025 the government confirmed a second phase for the 2025 to 2026 financial year. Seven technologies were selected, spanning faster bowel cancer diagnostics, earlier detection of skin cancer and genetic eye disease, clinical note-taking, and blood test interpretation. That phase runs until March 2026.

The sandbox reflects a philosophy distinct from both the American and European models. Where the FDA adapts existing device regulation through revised guidance, and the EU classifies and regulates through comprehensive legislation, the UK learns by doing, testing regulatory approaches alongside the technologies they are meant to govern and adapting according to what works.16

The limitation is scale. A sandbox accommodating four companies in its first year and seven in its second cannot by itself regulate an industry deploying AI across thousands of clinical applications. The AI Airlock is a laboratory for regulatory innovation rather than a regulatory system, and its value depends on whether its findings translate into binding requirements, and on whether the agency has the capacity and political support to make that translation.

The bias problem

Across all four approaches one issue recurs: AI systems trained on unrepresentative data produce unequal outcomes, and no jurisdiction has developed a fully effective response.

The evidence is starkest in dermatology. A study in the Journal of the European Academy of Dermatology and Venereology generated four thousand dermatological images across four leading models and found that only 10.2 per cent depicted dark skin, and only 15 per cent were identifiable by raters as the intended condition. Research in Science Advances documented substantial performance limitations in dermatology models tested against diverse datasets, with accuracy falling sharply for dark skin tones and uncommon diseases.

A Northwestern study published in Nature Medicine complicates the picture in a way worth stating carefully. Physicians were already 4 percentage points less accurate on darker skin before any AI was involved. AI assistance improved accuracy for every skin tone, and yet it widened the gap between light and dark skin by five percentage points among primary care physicians. The researchers attribute this not to a biased model but to how clinicians use one: specialists contextualise a recommendation against their own knowledge, while generalists appear to lean on it more heavily for conditions they see less often. Bias in the training data is a real problem—this particular finding is about the human half of the system.

The pattern extends beyond dermatology. Cardiology models have shown gender-based performance disparities. Triage systems trained largely on data from well-resourced settings perform less reliably in under-resourced ones. The consistent finding is that AI systems inherit and sometimes amplify the biases in their training data, and that medical training data is disproportionately drawn from populations that are white, male, and treated in well-funded Western healthcare systems.17

Every jurisdiction acknowledges this. The FDA's draft guidance requires bias analysis across demographic groups. The AI Act requires representative datasets and bias testing for high-risk systems. The AI Airlock examined how to generate diverse datasets for testing. The Texas disclosure requirement at least ensures patients know AI was involved, which creates the possibility of informed questioning.

None of these measures, singly or together, reaches the underlying cause: the datasets on which medical AI is trained do not represent the populations on which it is used. Fine-tuning on diverse data can close performance gaps, as research has demonstrated.18 But assembling diverse medical datasets is expensive, logistically complex, and dependent on participation from healthcare systems serving communities that have historically been underserved and have reason to distrust both medicine and the technology industry.

Four frameworks, one question

The four approaches that had crystallised by early 2026 each answer a different part of the same question: how should healthcare AI be governed?

Texas answers with transparency. Patients have a right to know when AI is involved. The obligation is clear, enforceable, and ready to implement. It places the burden of evaluation on patients who generally lack the expertise to assess system quality.

The European Union answers with classification and standards. Healthcare AI faces comprehensive requirements for data quality, bias testing, risk management, and human oversight. The framework is the most thorough in the world, will not be fully operational before 2026 at the earliest, and depends on harmonised standards still being written.

The FDA answers with lifecycle management. AI medical devices face continuous oversight from development through post-market deployment, with explicit requirements for bias analysis and performance documentation. The approach is technically sophisticated, rests on guidance still in draft, and depends on an approval process struggling to match the pace of deployment.

The United Kingdom answers with experimentation. The AI Airlock tests regulatory approaches in real clinical environments and generates practical findings formal regulation can draw on. Its small scale limits its direct effect, and the translation of those findings into binding requirements is uncertain.

No single approach suffices. Transparency without standards tells patients something is happening but not whether it is safe. Standards without transparency create accountability structures patients cannot see or engage with. Lifecycle management without capacity creates ambitions that cannot be met. Experimentation without scale produces findings that cannot yet be applied broadly.

What trust requires

Healthcare AI will expand. The incentives of faster diagnosis, reduced clinician workload, and broader access to specialist expertise are too strong, and the technology too capable, for adoption to slow. The question is not whether AI will be integrated into healthcare but whether patients will trust it, and whether they should.

Trust requires transparency, accountability, and evidence. Each of the four jurisdictions has built one of the three well and the others only partly, which is why no patient anywhere is currently protected by all three at once.

What is missing across all four is integration. A patient in Texas in January 2026 is told AI was involved in their diagnosis. They are not told whether that system meets European data quality standards, whether the FDA evaluated it for demographic bias, or whether its clinical performance was tested in the MHRA's sandbox. The information exists—or is being generated—inside separate regulatory systems that do not communicate and were never designed to.

The challenge for the next phase is not to choose among these approaches but to connect them: to build frameworks in which transparency, accountability, and evidence reinforce one another, and in which the protection available to a patient does not depend on where they happen to seek care. That integration does not exist yet. The components of it, each imperfect and each incomplete, are being assembled on both sides of the Atlantic, one regulatory experiment at a time.

Footnotes

  1. Perkins Coie, "Texas Enacts Responsible Artificial Intelligence Governance Act," June 2025. ↩

  2. K&L Gates, "Pared Back Version of the Texas Responsible Artificial Intelligence Governance Act Signed into Law," June 2025. ↩

  3. Akerman LLP, "HRx: New Year, New AI Rules: Healthcare AI Laws Now in Effect," January 2026. ↩

  4. European Commission, "Artificial Intelligence in Health," 2025. ↩

  5. EU AI Act, Article 10; Greenberg Traurig LLP, "EU Commission Consultation on High-Risk AI Systems: Key Points for Life Sciences and Health Care," June 2025. ↩

  6. EU AI Act, Articles 9, 11, 13, 14. ↩

  7. This was the position when this piece was written. The August 2026 date for standalone high-risk systems under Annex III was subsequently deferred to 2 December 2027 by the Digital Omnibus regulation, with product-embedded systems under Annex I moving to August 2028. The transparency duties under Article 50 and the general-purpose model obligations kept their original dates. ↩

  8. Hogan Lovells, "AI Health Law & Policy: Comparing Regulatory Landscapes for AI in Medical Devices in the EU and US," 2025. ↩

  9. Bipartisan Policy Center, "FDA Oversight: Understanding the Regulation of Health AI Tools," 2025. ↩

  10. FDA, "Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations; Draft Guidance," 7 January 2025. ↩

  11. CenterWatch, "FDA Guidance on AI-Enabled Devices: Transparency, Bias, & Lifecycle Oversight," January 2025. ↩

  12. Greenberg Traurig LLP, "FDA Releases Draft Guidance on AI-Enabled Medical Devices," January 2025. ↩

  13. DLA Piper, "FDA Issues Artificial Intelligence-Enabled Device Software Functions Draft Guidance," January 2025. ↩

  14. Sterne Kessler, "FDA Issues Draft Guidance Documents on Artificial Intelligence for Medical Devices, Drugs, and Biological Products," January 2025. ↩

  15. MHRA, "AI Airlock: The Regulatory Sandbox for AIaMD," 2024. ↩

  16. MedRegs, "How the AI Airlock Is Charting a Path for Regulating AI in Healthcare through Sandboxes," 31 October 2025. ↩

  17. Stanford HAI, "AI Shows Dermatology Educational Materials Often Lack Darker Skin Tones," 2024. ↩

  18. Practical Dermatology, "AI and Skin of Color: Hidden Biases Raise Questions," 2024. ↩