GitHub Pull Requests with Code Patches and Human Reviews

A collection of merged GitHub Pull Requests, code patches, and human review comments for benchmarking AI systems. It is derived from publicly available open-source repositories under permissive licenses. The specific row count, column count, and size are unknown.

Use Cases

• Benchmark AI Code Review Systems using human review comments and code patches.
• Evaluate Test Case Generation Models based on code patches from merged pull requests.
• Train Software Engineering LLMs on sequences of code patches and associated review comments.
• Analyze patterns in human review comments across different open-source projects.

Strengths

• Data is derived from publicly available open-source repositories under permissive licenses.
• Designed specifically for benchmarking AI code review and test generation systems.
• Includes human review comments, providing a high-signal evaluation target.

Limitations

• The dataset size, row count, and column structure are unspecified.
• The scope of included repositories and programming languages is not detailed.
• Potential for class imbalance or bias towards certain types of projects is unknown.

Provenance

Source

Publicly available open-source repositories on GitHub.

Collection Method

Curated from merged GitHub Pull Requests, code patches, and human review comments.

Freshness

Last updated on 2026-03-20.