Overview
- The PLOS Digital Health analysis published Wednesday found that of 1,357 FDA‑cleared or authorized AI/ML medical devices only three had been tested on patient‑centered outcomes such as death, hospitalization, or quality of life.
- Only 34 devices (2.5%) were linked to registered prospective clinical trials, just 12 had results posted to ClinicalTrials.gov, and 12 had peer‑reviewed manuscripts reporting clinical data.
- Most cleared AI tools are radiology applications, and the study shows validation is concentrated in well‑resourced health systems while key groups like pregnant people, adults over 75, and non‑English speakers were often excluded from studies.
- The authors say the FDA’s 510(k) pathway, which clears devices by showing ‘substantial equivalence’ to existing products, can propagate weak evidence across device generations and leave clinical benefit unproven.
- To close the gap researchers propose a staged roadmap with retrospective diverse‑dataset checks followed by workflow‑embedded prospective studies and larger post‑market multicenter outcome trials, while noting their analysis used only public records and may miss proprietary or unregistered validations.