PubMed: Uncopyrighted Text Corpus for Language Model Training

Pile Uncopyrighted is a version of The Pile dataset with copyrighted content removed to respect author rights. Created by yanjx21, it was last updated on April 1, 2025. The dataset is intended for training future large language models while adhering to copyright law.

Use Cases

• Train language models based on the uncopyrighted text corpus.
• Benchmark model performance on copyright-free academic and web text.
• Conduct research on the impact of training data copyright status on model outputs.
• Fine-tune models for biomedical or scientific domains based on the PubMed component.

Strengths

• Explicitly designed for copyright compliance by removing copyrighted books and subtitles.
• Based on The Pile, a well-known composite dataset for language modeling.
• Last updated on 2025-04-01, indicating recent maintenance.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Row count, file formats, and exact license are unknown, which may limit suitability assessment.
• Column-level documentation is absent; field semantics must be inferred after download.

Provenance

Source

huggingface

Collection Method

Created by removing copyrighted content (Books3, BookCorpus2, OpenSubtitles, YTSubtitles, OWT2) from The Pile dataset.

Time Range

null

Freshness

Last updated 2025-04-01 11:50:01; freshness should be verified.

Geography

null