FineWeb-Edu is a large-scale dataset containing 1.3 trillion tokens of high-quality educational web pages filtered from the broader FineWeb collection. It was developed to provide a massive, curated corpus for training language models with a focus on educational and instructional content.
Use Cases
- Large language model pre-training
- Educational content generation
- Knowledge-based AI research
Strengths
- Massive scale featuring 1.3 trillion tokens
- High-quality filtering using a classifier based on Llama3-70B-Instruct annotations
- Derived from the curated FineWeb dataset
Limitations
- Quality depends on the accuracy of the automated educational classifier
Provenance
- Source
- FineWeb dataset by HuggingFaceFW
- Collection Method
- Filtered from the FineWeb dataset using an educational quality classifier developed with annotations generated by LLama3-70B-Instruct.
- Freshness
- Last updated July 2025.