Light-R1 DPO Data: 1,000-10,000 Preference Pairs for Math Reasoning

Qihoo360 released this dataset in March 2025 containing between 1,000 and 10,000 preference pairs used to train the Light-R1-32B reasoning model. It focuses on curriculum-based Direct Preference Optimization (DPO) for mathematical reasoning and chain-of-thought tasks. The data was specifically designed to demonstrate high-performance distillation from scratch within a limited budget.

Use Cases

• Fine-tuning LLMs using Direct Preference Optimization (DPO) to improve mathematical reasoning accuracy
• Benchmarking model performance against AIME24 and AIME25 reasoning standards
• Researching curriculum-based SFT and DPO strategies for reasoning-heavy models

Strengths

• Contains 1,000 to 10,000 curated preference pairs
• Released under the Apache 2.0 license
• Directly linked to Arxiv paper 2503.10460 for technical reproducibility

Limitations

• Small sample size of under 10,000 records compared to larger preference datasets
• Highly specialized for mathematical reasoning which may limit utility for general-purpose conversational agents

Provenance

Source

qihoo360 (Arxiv 2503.10460)

Collection Method

Curated preference pairs for curriculum learning

Freshness

Last updated March 17, 2025.