Gitbugactions: Executable Program-Repair Benchmarks from GitHub Actions

Gitbugactions is a framework and collection of executable code benchmarks for program repair, maintained by the gitbugactions organization and updated as of March 2026. It facilitates the automated collection of datasets by leveraging GitHub Actions to identify and capture reproducible bug-fix instances in Python repositories.

Use Cases

• Automated program repair using captured bug-fix pairs
• Benchmarking CI/CD pipeline reliability using GitHub Action YAML files
• Analyzing software regression patterns through build log comparisons

Strengths

• MIT licensed for permissive redistribution
• Focuses on executable code rather than static snippets
• Includes CI/CD metadata from GitHub Actions

Limitations

• Primary focus on Python code limits language diversity
• Requires specific execution environments to utilize the executable nature of the data

Provenance

Source

GitHub repositories and GitHub Actions logs

Collection Method

Automated collection via GitHub Actions monitoring

Freshness

Updated as of March 2026.

Geography

Global (GitHub-hosted projects)