64 samples were used to evaluate the 'cogito-405b-thinking' model on the AIME24 benchmark, resulting in a score of 0.177. This dataset, authored by NousResearch and last updated on 2025-08-28, contains evaluation results across multiple benchmarks including math, reasoning, and creative writing. Scores and metrics are provided for benchmarks such as arenahard, bbh_generative, and eqbench3.
Use Cases
- Compare model performance across diverse benchmarks based on the reported scores for arenahard, aime24, and creative-writing-v3.
- Analyze the relationship between sample size and reported metrics based on the 'Samples' and 'Overlong rate' fields.
- Track the evolution of a specific model's capabilities based on the multi-benchmark evaluation results provided.
Strengths
- Provides specific, quantitative scores for 8 distinct benchmarks, including aime24 (0.177) and arenahard (0.829).
- Includes metadata for each benchmark such as sample count (e.g., 64 samples for aime24) and overlong rate (e.g., 100.0% for aime24).
- Results are attributed to a specific model version ('cogito-405b-thinking') and author (NousResearch).
Limitations
- Column-level documentation is absent; field semantics must be inferred after download.
- Row count is unknown, which may limit suitability assessment.
- Description metadata is limited; actual data quality requires manual inspection after download.
Provenance
- Source
- NousResearch via Hugging Face.
- Collection Method
- Likely contains aggregated results from running the specified model on standardized evaluation benchmarks.
- Freshness
- Last updated 2025-08-28 08:36:20; freshness should be verified.