Halbench: A Behavioral Benchmark for LLM Sycophancy and Hallucination

A corpus of prompts built on false premises to evaluate model behavior. The benchmark, created by Specific-Labs, tests for sycophancy and hallucination in LLMs when they produce content for fabricated references, overstated scopes, misapplied authorities, or unanswerable questions. The dataset was last updated on May 21, 2026.

Use Cases

• Measuring model sycophancy based on prompts with a false premise
• Evaluating hallucination rates based on fabricated references in prompts
• Benchmarking model robustness against misapplied authority statements
• Testing model refusal capabilities for unanswerable questions

Strengths

• Focuses on a specific, defined evaluation task for LLMs
• Created by a named organization, Specific-Labs
• Last update timestamp is provided (2026-05-21 16:55:26)

Limitations

• Column-level documentation is absent; field semantics must be inferred after download
• Row count is unknown, which may limit suitability assessment

Provenance

Source

Specific-Labs

Freshness

Last updated 2026-05-21 16:55:26