Benchmarking Five LLMs for High-Myopia Multiple-Choice Question Generation

300 multiple-choice questions generated by five large language models for ophthalmic education on high myopia. The dataset was created by Ligang Jiang and last updated on May 5, 2026. It includes objective metrics and expert ratings evaluating the models' performance on 60 predefined generation tasks.

Use Cases

• Benchmarking LLM performance on medical question generation based on the 60 predefined tasks.
• Analyzing the relationship between text features and expert ratings based on the Spearman correlation results.
• Comparing structural completeness and format compliance across different LLMs based on the 100% compliance metric.
• Evaluating the trade-off between explanation length and distractor quality based on the negative correlation reported.

Strengths

• All 300 generated questions achieved 100% initial structural acceptability, completeness, and format compliance.
• Expert inter-rater agreement was good, with an ICC range of 0.835–0.885.
• The dataset includes objective evaluation metrics across five domains and subjective ratings from two ophthalmology experts.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• The dataset is very small, at 147.3 KB, indicating limited scope.

Provenance

Source

Ligang Jiang via figshare.

Collection Method

Generated by five LLMs (ChatGPT-5.4, Gemini 3, DeepSeek, Kimi K2.5, and Doubao) completing 60 predefined tasks each.

Freshness

Last updated 2026-05-05 05:28:40; freshness should be verified.