Scale-SWE-Verified: 17,202 Gold-Patch-Validated Python Issue-Resolving Tasks

A 2026-07-08 validated fork of the Scale-SWE dataset, containing 17,202 Python issue-resolving tasks that produce a clean reward signal end-to-end. The dataset was created by PrimeIntellect by removing 2,979 rows (14.8%) from the upstream version through validation against exclusion lists and failure categorization.

Use Cases

• Training reward models for code generation based on the described clean end-to-end reward signal.
• Evaluating AI agents on Python software issue resolution tasks.
• Fine-tuning large language models for code-related problem-solving using validated task examples.
• Studying the characteristics of successful versus unsuccessful code patches in automated software engineering.

Strengths

• Contains 17,202 validated tasks, providing a substantial corpus for training and evaluation.
• 14.8% of the original data was removed through a documented validation process targeting specific failure categories and image URLs.
• Explicitly designed to produce a clean reward signal, which is a key feature for RLHF applications.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is known for the validated subset, but the total original file size and formats are unknown.
• Data may reflect bias inherent to the original source and validation methodology on Hugging Face.

Provenance

Source

PrimeIntellect, as a validated fork of AweAI-Team/Scale-SWE.

Collection Method

Gold-patch validation process removing rows based on image_url exclusion and failure categorization.

Time Range

null

Freshness

Last updated 2026-07-08 04:04:14; freshness should be verified.

Geography

null