CleanPile-64.5: A 64.5 GB English Text Corpus Filtered for Quality

64.5 gigabytes of uncompressed English text, processed from The Cleaned Pile to exclude older writing, code, and non-text sources. AlphaMindLabs created this dataset, which underwent general human review to improve content quality, and it was last updated on Hugging Face in August 2025. The dataset is smaller than its 85 GB counterpart but is positioned as a higher-quality subset for language modeling.

Use Cases

• Train large language models based on a large-scale, quality-filtered English text corpus.
• Fine-tune text generation models using content that has been reviewed to exclude non-text sources.
• Benchmark NLP model performance on datasets that have undergone human review for quality control.
• Pre-train foundational models on English text that has been filtered to remove old writing and code.

Strengths

• Approximately 64.5 GB of uncompressed text data provides substantial volume for model training.
• The dataset has undergone general human review to ensure most content is free from non-text sources.
• Content is selected to exclude old writing and code, which may improve training data quality.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment for specific tasks.
• The description lacks specifics on the exact filtering criteria and human review process.

Provenance

Source

The Cleaned Pile (uncopyrighted), processed by AlphaMindLabs.

Collection Method

Processed, uncompressed, and filtered to exclude old writing, code, and non-text sources with general human review.

Time Range

null

Freshness

Last updated 2025-08-09 05:40:16.

Geography

null