OmniClean: A Leakage-Aware Multimodal Evaluation Set

8,551 evaluation examples form this leakage-aware omni-modal benchmark. It is built from retained examples across 9 source benchmarks to reduce visual-shortcut effects. The dataset was created by author che111 and last updated on 2026-04-23.

Use Cases

• Evaluating model robustness against data leakage based on retained examples from source benchmarks.
• Probing for visual-shortcut effects in multimodal models using visual-only probing techniques.
• Benchmarking multimodal AI systems on a consolidated set derived from 9 different evaluation sources.

Strengths

• Contains 8,551 evaluation examples, providing a substantial testbed.
• Explicitly designed to reduce visual-shortcut effects, addressing a known evaluation pitfall.
• Aggregates data from 9 source benchmarks, offering a consolidated view.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• The specific composition and balance of the 9 source benchmarks are not detailed.

Provenance

Source

huggingface, author che111

Collection Method

Built from retained examples across 9 source benchmarks.

Freshness

Last updated 2026-04-23 12:25:37; freshness should be verified.