HRM-Text: A Cleaned Pretraining Dataset for Efficient Language Model Scaling

HRM-Text is a pre-built dataset for language model pretraining, created by applying data_io cleaning scripts to raw text data. The dataset was uploaded by the organization sapientinc and was last updated on May 21, 2026. It is associated with a research paper titled 'HRM-Text: Efficient Pretraining Beyond Scaling'.

Use Cases

• Pretraining language models based on the cleaned text corpus.
• Benchmarking efficient pretraining scaling laws based on the HRM-Text methodology.
• Studying the impact of data cleaning scripts on pretraining performance.

Strengths

• Dataset has been cleaned using specific data_io scripts, suggesting a structured preprocessing pipeline.
• Associated with a specific research paper (Wang et al., 2026), providing academic context.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count and total data size are unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

sapientinc

Collection Method

Created by applying data_io cleaning scripts to raw text data.

Freshness

Last updated 2026-05-21 03:34:11; freshness should be verified.