Diabetes Diagnosis Performance of Open-Source Large Language Models

11,329 adult diabetes patient records from a Chinese tertiary center (2010–2020) were used to evaluate open-source LLMs. The dataset supports analysis of model performance metrics like F1-scores for diabetes subtyping and comorbidity classification. It captures results from testing various LLMs using different prompting strategies in English and Chinese.

Use Cases

• Analyze F1-score performance for multi-class diabetes subtyping across different prompting strategies.
• Compare binary classification results for diabetic kidney disease (DKD) and metabolic syndrome (MetS) diagnosis between English and Chinese prompts.
• Investigate the impact of model characteristics, such as the optimal 32B parameter size, on diagnostic accuracy.
• Evaluate the effect of Chain-of-Thought prompting on MetS classification versus DKD performance degradation.

Strengths

• Contains performance metrics from a clinical evaluation of 11,329 patient records.
• Focuses on a specific, clinically relevant application: diagnosing diabetes subtypes and comorbidities.
• Compares results from multiple prompting strategies and two languages (English and Chinese).

Limitations

• The dataset is very small at 15.3 KB, indicating it likely contains summary results, not the underlying patient-level data.
• Data is sourced from a single Chinese tertiary center, limiting geographic generalizability.
• Performance metrics are based on a retrospective analysis, not prospective validation.

Provenance

Source

Shuting Yang via figshare.

Collection Method

Retrospective analysis of patient records to test various open-source LLMs with different prompting strategies.

Time Range

Patient data covers 2010 to 2020.

Freshness

Last updated March 25, 2026.

Geography

Data originates from a large Chinese tertiary center.