Systematic Review of AI Clinical Decision Support Predictive Performance

William J. Waldock's systematic review and meta-analysis evaluated 50 studies on AI-based clinical decision support systems across 17 medical specialties. It analyzed performance metrics including pooled AUC, specificity, accuracy, and sensitivity from a screening of 3,296 records. The study, following PRISMA guidelines, identified substantial heterogeneity and gaps in real-world clinical utility reporting.

Use Cases

• Analyze the distribution of pooled AUC (0.652) and specificity (0.819) values across the 17 medical specialties covered in the review.
• Compare the proportion of studies involving prospective deployment (24%) versus those reporting only technical metrics (64%) to assess research design bias.
• Investigate the relationship between reported sensitivity (0.660, 95% CI: 0.535–0.785) and accuracy (0.765, 95% CI: 0.734–0.796) metrics across the included studies.
• Apply the proposed ROADMAP framework domains to categorize the limitations and recommendations found in the 50 reviewed studies.

Strengths

• Analysis is based on a systematic review of 50 studies identified from 3,296 screened records, providing a broad evidence base.
• Performance metrics are quantitatively synthesized, reporting pooled AUC (0.652), specificity (0.819), accuracy (0.765), and sensitivity (0.660) with confidence intervals.
• The review spans 17 medical specialties, offering a multi-domain perspective on AI-CDSS performance.
• Study quality was assessed using the standardized QUADAS-2 tool, and the review adhered to PRISMA guidelines.

Limitations

• The dataset is a summary of study results (21.5 KB XLS file), not the underlying clinical data, limiting direct analytical applications.
• Substantial heterogeneity (I² ≥ 98.9%) was reported across all performance measures, indicating high variability between the included studies.
• The review's conclusions are constrained by the predominance of retrospective study designs and limited implementation reporting in the source literature.

Provenance

Source

William J. Waldock via figshare.

Collection Method

Systematic review and meta-analysis following PRISMA guidelines, with searches in PubMed and Cochrane Library.

Time Range

Literature search covered studies through December 2024.

Freshness

Literature search was conducted through December 2024, and the record was last updated in March 2026.