HalLoc Token-Level Hallucination Dataset for Vision-Language Models

Over 155,000 annotated samples comprise this dataset for localizing hallucinations in Vision-Language Models. Created by author uunicee, it spans three tasks and four hallucination types. The dataset was last updated in July 2025.

Use Cases

• Train classifiers to detect Object hallucination types in VQA model outputs.
• Benchmark model performance on Attribute hallucination localization within Image Captioning tasks.
• Analyze the frequency of Relation hallucination errors across different Instruction Following prompts.
• Develop fine-grained evaluation metrics using Scene hallucination annotations for model comparison.

Strengths

• Over 155,000 annotated samples provide a substantial basis for analysis.
• Covers three distinct tasks: VQA, Image Captioning, and Instruction Following.
• Provides fine-grained annotations across four specific hallucination types.

Limitations

• Specific sample counts per task or hallucination type are not detailed, limiting task-specific analysis.
• The underlying source data for the annotations and its potential biases are not described.

Provenance

Source

uunicee on Hugging Face Hub.

Collection Method

Annotations for hallucination localization in Vision-Language Model outputs; specific collection method not detailed.

Freshness

Last updated in July 2025.