CodeSearchNet-ccr: A Benchmark for Code Retrieval Evaluation

CodeSearchNet-ccr is a dataset for evaluating code retrieval models, hosted by CoIR-Retrieval. It employs the MTEB evaluation framework's dataset version for assessment. The dataset was last updated on September 12, 2024.

Use Cases

• Benchmarking code retrieval models based on the MTEB framework
• Evaluating semantic search performance for code snippets
• Training or fine-tuning embedding models for programming languages

Strengths

• Designed for use with the established MTEB evaluation framework
• Last updated on September 12, 2024, indicating recent maintenance

Limitations

• Column-level documentation is absent; field semantics must be inferred after download
• Row count is unknown, which may limit suitability assessment

Provenance

Source

CoIR-Retrieval

Freshness

Last updated 2024-09-12 03:29:31