ChatGPT-4o Diabetes Education Responses Evaluated by Endocrinologists

Ten diabetes FAQ responses generated by ChatGPT-4o in March 2025 were evaluated by five board-certified endocrinologists. The responses scored a mean of 4.0 for accuracy and overall quality but had a mean Flesch Reading Ease Score of 38.19, indicating college-level complexity. This 41.3 KB document, authored by Mohammad Abuzar and licensed CC-BY-4.0, contains the study's methodology and results.

Use Cases

• Benchmarking AI-generated medical content quality based on expert ratings for accuracy and relevance.
• Analyzing the readability of AI health advice based on Flesch-Kincaid Grade Level scores.
• Studying potential ceiling effects in expert evaluations based on the narrow score range reported.
• Investigating the relationship between response length and perceived quality based on the 300-word average.

Strengths

• Expert evaluations from five board-certified endocrinologists with ≥10 years mean experience.
• High mean expert scores: 4.0 for content accuracy and overall quality, 3.98 for relevance.
• No clinically inaccurate statements were identified by the expert raters.

Limitations

• Evaluation scope is limited to ten prompts, one AI model (ChatGPT-4o), and one session in March 2025.
• Readability analysis and content are English-only, limiting generalizability.
• Row count and column-level documentation are absent; data structure must be inferred from the DOCX file.

Provenance

Source

figshare

Collection Method

Cross-sectional evaluation using ten diabetes FAQs from the Diabetic Association of India and International Diabetes Federation.

Time Range

Data generated and evaluated in March 2025.

Freshness

Last updated 2026-04-13 05:39:23; freshness should be verified.

Geography

Global (sources include international organizations, but content is English-only).