AceCode-87K: 87,000 Coding Problems with Synthetic Test Cases

AceCode-87K consists of 87,000 coding problems paired with synthetic test cases generated and filtered via an automated pipeline. Created by TIGER-Lab and released in February 2025, the dataset supports reward model training and reinforcement learning for code generation tasks.

Use Cases

• Training reward models using the synthetic test cases as ground truth for code execution
• Reinforcement learning (RL) for code LLMs using test pass rates as reward signals
• Benchmarking code generation models against synthesized test constraints

Strengths

• 87,000 records providing scale for code-related reinforcement learning
• Automated noise filtering pipeline used to improve test case reliability
• MIT license allows for broad research and commercial application

Limitations

• Synthetic test cases may contain LLM-generated hallucinations or logic errors
• Lack of human-expert verification for the full 87,000 samples

Provenance

Source

TIGER-Lab (Arxiv 2502.01718)

Collection Method

Synthetic generation using LLMs from seed code datasets followed by automated filtering

Freshness

Last updated February 2025