LLM Performance Scores on Sleep Medicine Diagnostic Tasks

April 2026 results from a study evaluating ChatGPT-5 and Grok-4 on sleep medicine tasks. The dataset likely contains performance metrics from 79 clinical vignettes and 897 multiple-choice questions, authored by Anshum Patel and shared under a CC-BY-4.0 license.

Use Cases

• Benchmarking LLM diagnostic accuracy based on final diagnosis scores from clinical vignettes
• Comparing differential diagnosis generation quality based on F1-score metrics
• Analyzing LLM foundational knowledge performance based on MCQ accuracy percentages

Strengths

• Performance metrics are derived from 79 clinical vignettes and 897 multiple-choice questions, providing a substantive test base
• Results include specific accuracy percentages (e.g., 92.4% for final diagnosis) and F1-scores (e.g., 0.55 ± 0.20) for detailed comparison
• Statistical comparison between models is provided using the Mann–Whitney U test

Limitations

• Column-level documentation is absent; field semantics must be inferred after download
• Row count is unknown, which may limit suitability assessment
• The dataset is 231.9 KB, indicating a limited scope focused solely on evaluation scores

Provenance

Source

figshare

Collection Method

Evaluation scores compiled from a research study comparing LLMs.

Freshness

Last updated 2026-04-29 05:57:06; freshness should be verified