Fandom23K: 15.6 Million Articles from Fandom.com Wikis

15,616,749 articles were scraped from approximately 23,665 Fandom.com wikis between March 14 and March 18, 2023. This dataset is a subset of the upcoming BigKnow2022 dataset and was created by RyokoAI. It is primarily intended for unsupervised training of text.

Use Cases

• Unsupervised language model pre-training based on the large-scale text corpus.
• Analyzing writing styles and topics across different fan communities based on the wiki source.
• Studying the structure and evolution of collaborative online knowledge bases based on the Fandom.com source.

Strengths

• Large scale with 15,616,749 articles.
• Broad coverage from approximately 23,665 distinct Fandom wikis.
• Clear temporal snapshot from a specific 5-day collection period in March 2023.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• The dataset card notes the BigKnow2022 project is not yet complete, and not all information may be accurate or accessible.

Provenance

Source

Articles scraped from Fandom.com wikis.

Collection Method

Web scraping.

Time Range

Data collected between March 14 and March 18, 2023.

Freshness

Last updated 2023-03-20 19:58:46; freshness should be verified.