70,000 free eBooks sourced from the Project Gutenberg library, covering a wide range of public domain literature. Each entry represents a full book and includes standard metadata headers and footers delimited by specific start and end tags.
Use Cases
- Train large language models on long-form literary text by processing the full book content between the start and end tags
- Develop text cleaning pipelines to automatically remove standard Project Gutenberg headers and footers using the provided delimiters
- Perform cross-lingual analysis or machine translation tasks using specific language splits like the 'fr' subset
Strengths
- Contains over 70,000 individual eBook entries
- Includes structural delimiters '*** Start of ***' and '*** End of ***' to separate book content from metadata
- Supports multi-language splits, such as the 'fr' (French) split demonstrated in the usage examples