Fitz-Gov: A 2,980-Case Benchmark for Epistemic Honesty in RAG Systems

Fitz-Gov is a benchmark dataset containing 2,980 cases for measuring epistemic honesty in retrieval-augmented generation systems. Each case is a (query, retrieved contexts) pair labeled with the governance mode a well-calibrated RAG system should adopt. The dataset was created by yafitzdev and last updated on Hugging Face in May 2026.

Use Cases

• Benchmarking RAG system performance on epistemic honesty based on labeled query-context pairs.
• Training models to recognize when to abstain from answering based on insufficient retrieved contexts.
• Evaluating how systems handle conflicting information based on the 'disputed' governance mode label.

Strengths

• Contains 2,980 specifically designed benchmark cases.
• Provides explicit labels for three distinct governance modes (Abstain, Disputed, etc.) to guide system behavior.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is known, but the specific content of queries and contexts is not detailed in the provided metadata.

Provenance

Source

huggingface

Freshness

Last updated 2026-05-14 17:55:52; freshness should be verified.