mOSCAR: Multilingual Web Text Corpus with Content Filtering and Face Detection

mOSCAR is a multilingual web-crawled text corpus developed by the oscar-corpus organization. The dataset includes additional filtering steps to remove toxic content, a complete Spanish split, and face detection in images to blur them. The dataset page was last updated on 2024-11-23.

Use Cases

• Training multilingual language models based on the web-crawled text corpus.
• Developing content moderation tools based on the described toxic content filtering.
• Researching face privacy in multimodal datasets based on the face detection and blurring feature.
• Analyzing Spanish language web content based on the complete Spanish split.

Strengths

• Includes a complete Spanish language split.
• Implements face detection and blurring for images to address privacy.
• Applies additional filtering steps to remove toxic content.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count and total dataset size are unknown, which may limit suitability assessment.
• Freshness should be verified as the last update was 2024-11-23.

Provenance

Source

oscar-corpus organization via web crawling.

Collection Method

Web crawling with subsequent content filtering and processing.

Freshness

Last updated 2024-11-23 10:09:43.