Multiple programming language datasets for line-level code completion tasks within the CodeXGLUE benchmark. It provides unfinished code lines and their preceding context to evaluate model performance using exact match and edit similarity metrics.
Use Cases
- Train a sequence-to-sequence model to predict the remainder of a code line using the context and unfinished line features.
- Benchmark the performance of code generation models using the exact match evaluation metric.
- Measure the structural accuracy of autocompleted code using the edit similarity score.
- Analyze model failure patterns in line-level completion compared to token-level prediction using the provided context.
Strengths
- Focuses on line-level completion tasks to test model ability beyond token-level prediction.
- Includes evaluation scripts for exact match and edit similarity metrics.
- Sourced from the Microsoft CodeXGLUE benchmark for code-to-code intelligence.
- Provides unfinished code lines paired with preceding context for sequence generation.