Toxicity Balanced Text Documents from The Pile

Featuring 200,000 text documents from The Pile, balanced for toxicity. It was created by selecting the 100,000 most toxic and 100,000 least toxic documents from a 7-million-document subset scored using the Perspective API. The dataset was authored by tomekkorbak and last updated in April 2022.

Use Cases

• Train a text classifier to predict Perspective API toxicity scores on documents from The Pile.
• Analyze linguistic features distinguishing the 100,000 most toxic documents from the 100,000 least toxic ones.
• Fine-tune a language model on toxicity-balanced text to mitigate bias in generative outputs.
• Study the distribution of toxicity scores within a large-scale text corpus like The Pile.

Strengths

• Contains 200,000 text documents, providing a substantial sample for analysis.
• Explicitly balanced for toxicity, with 100,000 high-toxicity and 100,000 low-toxicity examples.
• Derived from The Pile, a known large-scale, diverse text corpus.
• Documents were scored using the Perspective API, a standard tool for toxicity detection.

Limitations

• Limited to a 3% subset (7 million documents) of The Pile, not the full corpus.
• Toxicity labels are based on an external API (Perspective), which may have its own biases and limitations.
• The dataset's temporal coverage and geographic origin of the source texts are not specified.

Provenance

Source

The Pile, a large-scale text corpus.

Collection Method

Constructed by scoring a 3% chunk (7 million documents) of The Pile using the Perspective API, then selecting the 100k most toxic and 100k least toxic documents.

Freshness

Last updated on 2022-04-06.

Geography

Region tag suggests 'us', but specific spatial coverage is unknown.