Sign in to view source links and access this dataset
Description
Tm Data is a text corpus used to train the TinyMistral-248m-v2 language model, containing approximately 8 million examples. The dataset was created by Locutusque and last updated on October 15, 2024. It consists of 4 million Wikipedia pages, 1 million arXiv papers, 1.5 million web pages from RefinedWeb and SlimPajama, 200,000 college textbooks, and 1 million Stack Exchange forum posts.
Use Cases
Train language models on a mix of encyclopedic, academic, and conversational text.
Fine-tune models for specific domains using subsets like academic papers or technical forum posts.
Benchmark model performance across different text genres and sources.
Study the characteristics of web-crawled text versus curated sources like textbooks.
Strengths
Approximately 8 million total text examples provides substantial training material.
Diverse source mix includes 4 million Wikipedia pages, 1 million arXiv papers, and 1 million Stack Exchange posts.
Includes 1.5 million web pages from established filtered web corpora (RefinedWeb and SlimPajama).
Limitations
Column-level documentation is absent; field semantics must be inferred after download.
Data may contain NSFW examples as noted by the author, requiring content filtering.
Row count per source is approximate; exact composition is not specified.
Provenance
Source
Aggregated from Wikipedia, arXiv, RefinedWeb, SlimPajama, college textbooks, and Stack Exchange.
Collection Method
Likely compiled from publicly available web crawls and text dumps for model training.
Freshness
Last updated 2024-10-15 06:06:47; freshness should be verified.
The dataset can contain NSFW examples, use at your own risk.