Scalebench is a benchmark dataset for evaluating budget-efficient sequential experimental design in scaling-law fitting. The dataset was created by author 'sijieli' and was last updated on April 30, 2026. It provides finite pools of candidate experiments, held-out target regions, task-specific covariates, and observed outcomes.
Use Cases
- Benchmarking active experiment selection algorithms based on the described finite candidate pools and target regions.
- Developing budget-aware scaling law fitting methods based on the provided covariates and observed outcomes.
- Comparing sequential experimental design strategies for machine learning based on the task-specific structure of the benchmark.
Strengths
- Designed specifically for the task of budget-aware sequential experimental design, providing a focused benchmark.
- Includes held-out high-cost target regions for evaluation, as mentioned in the description.
- Provides task-specific covariates and observed outcomes for each configuration.
Limitations
- Column-level documentation is absent; field semantics must be inferred after download.
- Row count is unknown, which may limit suitability assessment.
- Description metadata is limited; actual data quality requires manual inspection after download.
Provenance
- Source
- huggingface
- Collection Method
- Created for the research 'Spend Less, Fit Better: Budget-Efficient Scaling Law Fitting via Active Experiment Selection'.
- Time Range
- null
- Freshness
- Last updated 2026-04-30 22:39:27; freshness should be verified.
- Geography
- null