Megawika 2: Multilingual Wikipedia Articles and Citations in 20 Languages

Created by JHU-CLSP and released in 2024, this multilingual dataset provides a structured view of Wikipedia articles across 20 languages. It integrates article text with external web citations, source quality estimates, and cross-lingual translations based on May 2024 Wikipedia dumps.

Use Cases

• Training fact-checking models using the relationship between article text and web sources
• Developing multilingual NLP systems using article text translations
• Filtering training corpora using source text quality estimates

Strengths

• Covers 20 diverse languages including Irish, Gujarati, and Finnish
• Includes quality estimates for cited external sources
• Derived from recent May 2024 Wikipedia snapshots

Limitations

• Requires external script execution to access the actual text of cited web sources
• Language coverage is restricted to 20 specific ISO codes

Provenance

Source

Johns Hopkins University Center for Language and Speech Processing (JHU-CLSP)

Collection Method

Processed from Wikipedia dumps and external web citation metadata

Time Range

2024

Freshness

Based on Wikipedia dumps from May 1, 2024.

Geography

Global