Performance of 5 LLMs vs. Junior Physicians in Emergency Internal Medicine Diagnosis

A retrospective study comparing the diagnostic performance of five large language models against emergency department junior physicians. The evaluation used 154 anonymized patient cases from a single hospital center between January and May 2025, assessing main diagnostic accuracy, differential diagnosis comprehensiveness, and response time. The dataset was authored by Jintao Wei and published on figshare in May 2026.

Use Cases

• Benchmarking LLM diagnostic accuracy against human clinicians based on real-world emergency cases.
• Analyzing the comprehensiveness of differential diagnoses generated by AI models.
• Comparing response time efficiency between AI systems and human physicians in a diagnostic workflow.

Strengths

• Includes performance metrics for 5 specific LLMs (ChatGPT-4o, Gemini-2.0, Grok3, DeepSeek-V3, Doubao) and a human control group.
• Based on 154 real-world, anonymized patient cases from a defined time period (January-May 2025).
• Reports specific statistical results, including main diagnostic accuracy percentages and response times in seconds.

Limitations

• Row count and column-level documentation are absent; field semantics must be inferred after download.
• Data is from a single-center study, which may limit generalizability.
• The dataset is very small (17.0 KB), indicating limited scope, likely containing summary results rather than raw case data.

Provenance

Source

figshare, authored by Jintao Wei.

Collection Method

Single-center retrospective analysis of anonymized emergency internal medicine patient cases.

Time Range

January 2025 to May 2025

Freshness

Last updated 2026-05-08 05:56:42; freshness should be verified.

Geography

Likely China, based on the hospital name (Second Affiliated Hospital of Zhejiang University School of Medicine).