synCOCO: Synthetic Image Pairs for Concept-Based Interpretability Evaluation

A synthetic, paired-image benchmark for evaluating concept-based interpretability. Each item is an (original, synthetic) image pair where exactly one object class is removed, generated with FLUX.2 [dev] conditioned on COCO reference images. It accompanies the paper 'Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations'.

Use Cases

• Benchmarking concept-based interpretability methods based on paired images with a single object removed.
• Evaluating the sensitivity of interpretability tools to specific object classes based on the 'removed_class' feature.
• Training or validating models that detect or explain visual concepts based on synthetic image transformations.

Strengths

• Dataset is explicitly designed as a benchmark for a specific research task: evaluating concept-based interpretability.
• Image pairs are generated with a specific, named model (FLUX.2 [dev]) conditioned on a well-known reference dataset (COCO).

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

Author: jokl. Platform: huggingface.

Collection Method

Images are synthetically generated with FLUX.2 [dev] conditioned on COCO reference images.

Freshness

Last updated 2026-06-18 21:13:51; freshness should be verified.