Performance of 5 LLMs vs. Junior Physicians in Emergency Internal Medicine Diagnosis, 2025

A single-center retrospective study from the Second Affiliated Hospital of Zhejiang University School of Medicine. It contains evaluation results for 5 large language models and 15 emergency department junior physicians diagnosing 154 anonymized emergency internal medicine cases from January to May 2025. The dataset was authored by Jintao Wei and last updated in May 2026.

Use Cases

• Benchmarking diagnostic accuracy of medical LLMs based on real-world emergency cases mentioned in the description
• Analyzing the comprehensiveness of differential diagnoses generated by AI versus human physicians
• Comparing response time efficiency between AI models and clinicians in a simulated diagnostic workflow

Strengths

• Includes performance metrics for 5 specific LLMs (ChatGPT-4o, Gemini-2.0, Grok3, DeepSeek-V3, Doubao) and 15 junior physicians
• Based on 154 real-world, anonymized patient cases from a defined period (January to May 2025)
• Reports specific statistical results, including main diagnostic accuracy percentages and response times in seconds

Limitations

• Column-level documentation is absent; field semantics must be inferred after download
• Row count is unknown, which may limit suitability assessment
• Data is from a single center, which may reflect geographic or institutional bias

Provenance

Source

figshare, authored by Jintao Wei

Collection Method

Retrospective analysis of anonymized emergency internal medicine patient cases

Time Range

January to May 2025

Freshness

Last updated 2026-05-08 05:56:41; freshness should be verified

Geography

Second Affiliated Hospital of Zhejiang University School of Medicine, China