ChatGPT-5 Vasectomy Education Assessment by Expert Panel

A 2026 expert panel assessment of ChatGPT-5's responses to ten high-frequency public questions about vasectomy. Eight experts rated responses on six criteria, including medical accuracy and clarity, using 5-point Likert scales. The dataset, authored by Ali C. Albaz, contains the resulting evaluation scores and statistical analysis.

Use Cases

• Benchmarking LLM performance on sensitive medical topics based on expert ratings for accuracy and completeness.
• Analyzing inter-rater reliability in medical AI assessments based on the reported intraclass correlation coefficient.
• Studying the suitability of AI-generated health information for the public based on ratings for clarity, tone, and usefulness.

Strengths

• Expert evaluations from eight specialists, including urologists and public health experts.
• Responses assessed on six distinct 5-point Likert-scale criteria, providing multi-dimensional ratings.
• Questions derived from Google Trends data, reflecting real-world public interest.

Limitations

• Dataset is very small (27.0 KB), indicating limited scope.
• Row count and column-level documentation are unknown; field semantics must be inferred after download.
• Inter-rater reliability was reported as very low (ICC = -0.01), indicating substantial variability in expert evaluations.

Provenance

Source

Ali C. Albaz via figshare

Collection Method

Expert panel assessment of ChatGPT-5 responses to Google Trends-derived questions.

Freshness

Last updated 2026-03-18 11:17:31