MetaTruth: Five Mechanisms of Metacognitive Failure in 15 Frontier LLMs

MetaTruth is a benchmark dataset for evaluating metacognitive failures in large language models. It contains 105 tasks designed to test five specific mechanisms of failure across 15 different models. The dataset was sourced from Kaggle, but the original author, organization, and last update date are unknown.

Use Cases

• Benchmarking model self-awareness based on the 105 metacognitive tasks.
• Analyzing failure patterns across different LLM architectures based on the 15 evaluated models.
• Researching specific mechanisms of metacognitive error as described in the dataset's framework.

Strengths

• Includes 105 distinct tasks for evaluating model performance.
• Benchmarks 15 different large language models, allowing for comparative analysis.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Last update date is unknown; freshness unverified.

Provenance

Source

Kaggle

Collection Method

Likely a curated benchmark for model evaluation.

Time Range

Temporal coverage is unknown.

Freshness

Last update date is unknown; freshness unverified.

Geography

Spatial coverage is unknown.