AI Dialogue Evaluations for Cartilage Repair Questions

A study comparing ChatGPT, DeepSeek, and Google Search on answering cartilage repair questions. The dataset includes results from a matched three-way comparison across two medical domains, with quality scored by three blinded raters. The 23.6 KB document was authored by Sen Yang Xiao and last updated in May 2026.

Use Cases

• Benchmarking AI model accuracy and safety in medical domains based on the described Accuracy-Safety-Hallucination framework.
• Analyzing the readability of AI-generated medical answers compared to search engines based on Flesch-Kincaid scores.
• Studying question classification in medical contexts based on the modified Rothwell taxonomy mentioned in the methods.

Strengths

• Includes a three-way comparison of two LLMs and Google Search across two distinct medical domains.
• Answer quality was independently evaluated by three blinded raters using a structured framework.
• Released under a permissive CC-BY-4.0 license.

Limitations

• The dataset is a 23.6 KB document; the underlying evaluation data (e.g., raw questions, scores) is not directly accessible in a structured format.
• Row count and column-level documentation are unknown, limiting suitability assessment for direct analysis.
• Freshness should be verified as the last update was 2026-05-29 05:53:27.

Provenance

Source

figshare, authored by Sen Yang Xiao.

Collection Method

Questions were sourced from top Google FAQs in cartilage tissue engineering (2023) and cartilage repair surgery (2024), then submitted to three platforms for comparison.

Time Range

Questions sourced from 2023 and 2024.

Freshness

Last updated 2026-05-29 05:53:27.