Participant Characteristics for LLM-Scored Cookie Theft Cognitive Assessments

25 participants were selected from the DementiaBank corpus for a study evaluating five large language models in scoring the Cookie Theft picture description task. The dataset, created by Michael J. Kleiman and last updated in April 2026, contains participant characteristics in a 5.5 KB Excel file. It supports research on automating neuropsychological assessments.

Use Cases

• Benchmarking LLM performance against human raters based on scoring accuracy metrics mentioned in the description
• Analyzing the impact of prompt strategies like few-shot learning on scoring reliability
• Studying participant characteristics for cognitive impairment detection based on picture description tasks

Strengths

• Directly compares five LLMs (GPT-4 Turbo, GPT-4o, Claude 3 Opus, Claude 3 Sonnet, Llama 3 70b) to human consensus scores.
• Data is derived from the established DementiaBank corpus.
• Released under a permissive CC-BY-4.0 license.

Limitations

• Row count is unknown, which may limit suitability assessment.
• Column-level documentation is absent; field semantics must be inferred after download.
• The dataset is very small at 5.5 KB, indicating limited scope.

Provenance

Source

Subset from the DementiaBank corpus.

Collection Method

Participant selection for an evaluation study of LLM scoring.

Time Range

null

Freshness

Last updated 2026-04-21 17:39:54; freshness should be verified.

Geography

null