German Wikipedia Preprocessed for Language Model Training

Wikipedia German — Preprocessed is a cleaned version of the German Wikipedia dump, totaling 13.5 GB. The dataset was processed by raj2708 from a dump dated May 2026, sourced from de.wikipedia.org. It is intended for use in large language model pretraining.

Use Cases

• Pretrain German-language large language models based on the cleaned text corpus.
• Benchmark text generation or understanding models using a standard German knowledge source.
• Analyze linguistic patterns in formal German text based on the Wikipedia content.

Strengths

• Total size of 13.5 GB provides a substantial text corpus.
• Processing steps include parsing wikitext to plain text and filtering redirects and short articles.
• Explicitly licensed under CC-BY-SA 4.0 for clear usage rights.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• The description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

de.wikipedia.org

Collection Method

Downloaded full German Wikipedia XML BZ2 dump, parsed articles using mwparserfromhell, and applied cleaning filters.

Time Range

Dump date is May 2026.

Freshness

Last updated 2026-05-29 14:14:00; freshness should be verified.