AI & Psychotherapy

Published in the BMJ in March 2025, PROBAST+AI is the first quality assessment tool for clinical prediction models that holds classical statistical and artificial intelligence approaches to the same standards of rigour. Its starting finding is damning: most published models are of poor quality, their performance is overestimated and their biases go unnoticed. Sixth instalment of our series on AI evaluation frameworks in healthcare.

In April 2025, CONSORT and SPIRIT published their first coordinated update since 2010 and 2013. Major advance: Open Science is now integrated into the base standards. But artificial intelligence? Absent. Despite CONSORT-AI existing since 2020, the new guidelines mention neither AI, nor existing extensions, nor algorithmic transparency issues.

Major AI health studies — published in Nature, JAMA, The Lancet — rely on Prolific workers, text-based vignettes, and inadequate comparators. Five layers of distance separate these protocols from clinical reality. A detailed analysis with data and a practical reading framework for the clinician.

Out of 137 studies published in the year following ChatGPT's launch, fewer than 40% report key elements of their prompting strategy. An international consortium of 531 experts proposes 12 criteria to fix this — and change how we read these studies.