AI Chatbot Response Evaluations for Intermittent Fasting in Kidney Disease

A prospective structured evaluation of ChatGPT-5 and Claude 4.5 Sonnet responses to 25 patient-centered questions on intermittent fasting in chronic kidney disease. Responses were scored for accuracy, consistency, readability, and quality using the Flesch-Kincaid Grade Level and DISCERN instrument. The dataset, authored by Joel Shah and shared under CC-BY-4.0, contains the evaluation results in a 14.6 KB DOCX file.

Use Cases

• Benchmarking AI model accuracy and consistency in clinical nutrition advice based on the structured evaluation of 25 questions.
• Analyzing the readability of AI-generated health information based on Flesch-Kincaid Grade Level scores.
• Evaluating the quality of AI chatbot responses for patient education using DISCERN instrument scores.
• Comparing the performance of different AI models (ChatGPT-5 vs. Claude 4.5 Sonnet) on a specific clinical topic.

Strengths

• Evaluation is based on 25 patient-centered questions developed by board-certified nephrologists.
• Responses were scored using established instruments: Flesch-Kincaid Grade Level and DISCERN.
• Data is shared under a permissive CC-BY-4.0 license.

Limitations

• The dataset is very small at 14.6 KB, indicating limited scope.
• Row count and column-level documentation are unknown; field semantics must be inferred after download.
• Data may reflect the specific prompt and evaluation methodology bias inherent to this single study.

Provenance

Source

figshare, authored by Joel Shah.

Collection Method

Prospective structured evaluation; questions were submitted to AI chatbots under standard user conditions.

Freshness

Last updated 2026-05-29 04:54:10.