Local Code Arena Eval Results: MBPP Benchmark on DeepSeek R1 8B

ShahzebKhoso's repository hosts raw evaluation metrics, execution telemetry logs, and structural syntax outputs from running the Mostly Basic Python Problems (MBPP) benchmark against the DeepSeek R1 8B model. This specific run establishes an operational baseline and performance characteristics for a mid-tier reasoning model under automated evaluation. The dataset was last updated on 2026-05-29.

Use Cases

• Benchmarking code generation models based on MBPP evaluation metrics.
• Analyzing model execution telemetry and performance characteristics under time limits.
• Studying structural syntax outputs from a distilled reasoning architecture.
• Establishing operational baselines for mid-tier reasoning models on Python problems.

Strengths

• Contains raw evaluation metrics, execution telemetry logs, and structural syntax outputs, providing multiple data facets.
• Focuses on a specific, established benchmark (Mostly Basic Python Problems) for model evaluation.
• Targets a specific model architecture (DeepSeek R1 8B distilled reasoning) for clear comparison.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• Row count and file formats are unknown, which may limit suitability assessment.

Provenance

Source

huggingface

Collection Method

Results from running the MBPP benchmark against the DeepSeek R1 8B model.

Freshness

Last updated 2026-05-29 10:28:27; freshness should be verified.