DRAKE: Multi-Modal Federated Learning Benchmark with 40 Tasks

DRAKE is a multi-modal federated continual learning benchmark featuring 40 distinct tasks and between 100,000 and 1,000,000 records. Developed by SNUMPR for ICLR 2026, it evaluates agent knowledge through vision-language question answering under realistic distribution shifts over time.

Use Cases

• Benchmarking federated learning models using the 40 distinct task categories to simulate heterogeneous client environments
• Evaluating continual learning stability by applying the temporal distribution shifts provided in the benchmark
• Training vision-language agents to perform question answering tasks across multi-modal inputs

Strengths

• Includes 40 distinct task categories
• Scale of 100,000 to 1,000,000 records
• Peer-reviewed benchmark for ICLR 2026

Limitations

• Metadata does not specify individual column names
• Requires specialized federated learning infrastructure to utilize fully

Provenance

Source

SNUMPR, ICLR 2026

Freshness

Last updated March 2026.

Geography

United States