Tokenlearn Cornstack Queries Coderankembed V2: Code Embeddings for 6 Languages

600,000 code documents across six programming languages provide training targets for static embedding distillation. The dataset was created by minishlab using Tokenlearn for training Model2Vec models on code retrieval. Mean token embeddings were produced by the nomic-ai/CodeRankEmbed model.

Use Cases

• Training code retrieval models based on the described mean token embeddings.
• Distilling static embedding models using the provided training targets.
• Benchmarking code representation models across six programming languages.
• Fine-tuning embedding models for specific programming languages mentioned in the description.

Strengths

• Contains 600,000 total rows of code documents.
• Includes data from six distinct programming languages.
• Provides 100,000 rows per language, suggesting balanced representation.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Last updated 2026-05-06 03:41:56; freshness should be verified.

Provenance

Source

CornStack

Collection Method

Created with Tokenlearn; embeddings produced by nomic-ai/CodeRankEmbed.

Freshness

Last updated 2026-05-06 03:41:56.