Diff-XYZ: A Benchmark of 1,000 Real-World Code Edits

Diff-XYZ is a dataset for the paper 'Diff-XYZ: A Benchmark for Evaluating Diff Understanding'. It contains 1,000 real-world code edits sampled and filtered from the CommitPackFT dataset. The dataset was created by JetBrains-Research and was last updated on November 14, 2025.

Use Cases

• Benchmarking LLMs on applying code edits based on diff representations
• Evaluating LLMs on summarizing code changes based on old and new code contents
• Testing LLMs on generating code diffs based on original and modified file pairs

Strengths

• Contains 1,000 examples of real-world code edits
• Provides multiple diff representations (udiff, udiff-h, udiff-l, search-replace) for each edit
• Includes both original (old_code) and modified (new_code) file contents for each example

Limitations

• Column-level documentation is absent; field semantics must be inferred after download
• Row count is unknown, which may limit suitability assessment

Provenance

Source

JetBrains-Research

Collection Method

Sampled and filtered from the CommitPackFT dataset

Time Range

null

Freshness

Last updated 2025-11-14 11:49:16; freshness should be verified

Geography

null