AI Dialogue Evaluations for Cartilage Repair Questions, 2023-2024

A 2023-2024 comparative study evaluating ChatGPT, DeepSeek, and Google Search responses to cartilage repair questions. The dataset contains the results of a blinded quality and readability analysis of answers across cartilage tissue engineering and cartilage repair surgery domains. Authored by Sen Yang Xiao and shared under a CC-BY-4.0 license on figshare.

Use Cases

• Benchmarking AI model accuracy and safety in medical domains based on the described Accuracy-Safety-Hallucination (ASH) framework.
• Analyzing readability differences between AI-generated and search engine content based on Flesch-Kincaid Grade Level scores.
• Comparing information source classification patterns based on the modified Rothwell taxonomy mentioned in the description.

Strengths

• Data is derived from a structured, blinded evaluation by three raters using the ASH framework.
• Includes a matched three-way comparison of three platforms (ChatGPT, DeepSeek, Google) on identical questions.
• Results include specific median accuracy scores (e.g., 5.00 for DeepSeek in CTE) and statistical significance (p-values).

Limitations

• Row count is unknown, which may limit suitability assessment.
• Column-level documentation is absent; field semantics must be inferred after download.
• The dataset is very small (21.7 KB), indicating limited scope, likely containing summary results rather than raw dialogue data.

Provenance

Source

figshare

Collection Method

Top-10 Google-derived FAQs per domain were submitted to ChatGPT (GPT-4 API), DeepSeek (V3 API), and Google Search for answer generation and evaluation.

Time Range

Cartilage tissue engineering questions from 2023, cartilage repair surgery questions from 2024.

Freshness

Last updated 2026-05-29 05:53:29; freshness should be verified.

Geography

null