FineWeb Text Corpus for Parameter Golf Experiments

Parameter Golf FineWeb Export contains 15,318,808 training and 50,000 validation documents of English text. The dataset is a Parquet-converted fork of a ~10-billion-token subset derived from the 100-billion-token FineWeb corpus, intended for parameter-golf experiments. It was created by mishig and last updated in March 2026.

Use Cases

• Pretrain language models on the ~10-billion-token English text documents for parameter-golf experiments.
• Fine-tune models for text generation using the training and validation document splits.
• Analyze the linguistic characteristics of the English text documents from the FineWeb corpus subset.

Strengths

• Large scale with 15,318,808 training documents.
• Includes a dedicated validation split of 50,000 documents.
• Derived from the well-known 100-billion-token FineWeb corpus.

Limitations

• Specific column names and data structure are not documented.
• The dataset is a processed subset, not the original full FineWeb corpus.
• No information on data cleaning, deduplication, or quality filtering processes.

Provenance

Source

HuggingFaceFW/fineweb

Collection Method

Parquet-converted fork of willdepueoai/parameter-golf, derived from a ~10B-token subset of the 100B FineWeb cut.

Freshness

Last updated March 2026.