HFLB: Heterogeneous Federated Learning Benchmark with 100K-1M VQA Records

HFLB is a benchmark for heterogeneous federated learning containing between 100,000 and 1,000,000 records, developed by SNUMPR for the FedMosaic (ICLR 2026) study. It modifies constituent datasets like GQA and Abstract VQA into distinct subtasks to support task incremental learning research.

Use Cases

• Evaluating task incremental learning using the subtask splits from GQA
• Benchmarking federated learning algorithms in heterogeneous environments
• Training visual reasoning models on abstract-scene question answering pairs

Strengths

• Contains 100K to 1M records
• Aggregates multiple established benchmarks including GQA and Abstract VQA
• Pre-split into subtasks for task incremental learning evaluation

Limitations

• Specific column names and schema are not documented in the metadata
• Derivative nature means it inherits biases from the original GQA and Abstract VQA sources

Provenance

Source

SNUMPR, originally proposed in FedDAT and modified for FedMosaic (ICLR 2026)

Collection Method

Aggregated and modified from existing datasets (GQA, Abstract VQA) for specific benchmark configurations

Freshness

Last updated March 2026

Geography

United States