AI Dialogue Evaluations for Cartilage Repair Questions

A study by Sen Yang Xiao, uploaded on 2026-05-29, compares ChatGPT, DeepSeek, and Google Search in answering cartilage repair questions. The dataset contains results from a matched three-way comparison across cartilage tissue engineering (2023) and cartilage repair surgery (2024) domains. It includes blinded quality scores for accuracy, safety, and hallucination, as well as readability analysis.

Use Cases

• Compare AI model accuracy for medical questions based on the Accuracy-Safety-Hallucination (ASH) framework scores
• Assess the safety profile of large language models in surgical contexts based on the safety analysis mentioned
• Analyze the readability of AI-generated medical answers based on Flesch-Kincaid Grade Level scores
• Benchmark information retrieval platforms for clinical decision support based on the domain-specific performance results

Strengths

• Includes a three-way comparison of ChatGPT (GPT-4), DeepSeek (V3), and Google Search, enabling direct benchmarking
• Provides blinded quality scoring by three independent raters using the ASH framework, suggesting methodological rigor
• Covers two distinct medical domains: cartilage tissue engineering (2023) and cartilage repair surgery (2024)
• Dataset is openly licensed under CC-BY-4.0, facilitating reuse and sharing

Limitations

• Dataset is very small at 20.1 KB, indicating limited scope and likely a summary of results rather than raw data
• Column-level documentation is absent; field semantics must be inferred after download
• Row count is unknown, which may limit suitability assessment for statistical analysis

Provenance

Source

figshare, author Sen Yang Xiao

Collection Method

Study queried Google Search for top FAQs, then submitted identical questions to ChatGPT, DeepSeek, and Google for comparison. Answers were classified, scored for quality, and analyzed for readability.

Time Range

Questions sourced from cartilage tissue engineering (2023) and cartilage repair surgery (2024) domains.

Freshness

Last updated 2026-05-29 05:53:27; freshness should be verified.