Local Code Arena MBPP: Qwen 2.5 Coder 3B Evaluation Telemetry

ShahzebKhoso's repository hosts raw evaluation metrics and execution telemetry logs from running the Mostly Basic Python Problems (MBPP) benchmark against the Qwen 2.5 Coder 3B parameter model. The data captures a specific evaluation designed to chart the transition between hyper-lightweight edge models and larger desktop-class variants. It was last updated on 2026-05-23.

Use Cases

• Analyzing model performance on basic Python problems based on the MBPP benchmark.
• Comparing efficiency metrics between different model size classes based on the described transition point analysis.
• Studying execution telemetry and structural syntax outputs from code generation models.

Strengths

• Evaluation focuses on a specific model size (3B parameters) to isolate efficiency inflection points.
• Data includes raw evaluation metrics, execution telemetry logs, and structural syntax outputs.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count and dataset scale are unknown, which may limit suitability assessment.

Provenance

Source

huggingface

Collection Method

Telemetry logs from running the MBPP benchmark on the Qwen 2.5 Coder 3B model.

Freshness

Last updated 2026-05-23 12:01:56.