AI Dialogue Comparisons for Cartilage Repair: Accuracy and Safety Scores

A 2026 study by Sen Yang Xiao compares ChatGPT, DeepSeek, and Google Search on cartilage repair questions. The dataset contains results from a matched three-way comparison across cartilage tissue engineering (2023) and cartilage repair surgery (2024) domains. It includes blinded quality scores for accuracy, safety, and hallucination, as well as readability assessments.

Use Cases

• Benchmarking AI model performance in medical domains based on the Accuracy-Safety-Hallucination (ASH) framework scores.
• Analyzing the readability of AI-generated medical answers compared to search engines based on Flesch-Kincaid Grade Level scores.
• Comparing the functional roles of different AI platforms for clinical decision support and patient education as described in the conclusion.

Strengths

• Results are based on independent evaluation by three blinded raters using a structured framework.
• Includes specific median accuracy scores (e.g., 5.00 for DeepSeek in CTE) and statistical significance values (e.g., p=0.036).
• Compares three distinct platforms (Google Search, ChatGPT, DeepSeek) on identical questions across two medical domains.

Limitations

• Row count is unknown, which may limit suitability assessment.
• Column-level documentation is absent; field semantics must be inferred after download.
• The dataset is very small at 11.0 KB, indicating limited scope.

Provenance

Source

figshare, author Sen Yang Xiao

Collection Method

Top-20 Google-derived FAQs per domain were submitted to three platforms; answers were classified, scored, and analyzed for readability.

Time Range

Cartilage tissue engineering questions from 2023, cartilage repair surgery questions from 2024.

Freshness

Last updated 2026-05-29 05:53:30; freshness should be verified.