CommitPack: 4TB of Permissively Licensed GitHub Code Commits

CommitPack contains 4TB of GitHub commit data scraped from permissively licensed repositories by the BigCode project. Released in conjunction with the OctoPack research (arXiv:2308.07124), it provides a massive-scale collection of code changes and their corresponding natural language descriptions.

Use Cases

• Training Large Language Models (LLMs) to generate code based on commit message instructions
• Fine-tuning models for automated commit message generation using code diffs
• Analyzing code evolution and software engineering patterns across different programming languages

Strengths

• 4TB total data volume
• Restricted to permissively licensed repositories (MIT)
• Linked to peer-reviewed research (arXiv:2308.07124)

Limitations

• Extreme storage and compute requirements due to 4TB size
• Potential for noise from automated or non-descriptive commit messages

Provenance

Source

BigCode project and GitHub

Collection Method

Scraped

Freshness

Last updated February 2025; reflects repository states at the time of scraping for the OctoPack project.

Geography

Global