Toxicity-Balanced Text Documents from The Pile

Encompassing 200,000 text documents from The Pile, scored for toxicity using the Perspective API in May 2022. It is balanced with 100,000 of the most toxic documents and 100,000 randomly sampled documents.

Use Cases

• Train a toxicity classifier using the Perspective API toxicity scores as labels on 200,000 text documents.
• Analyze linguistic patterns and features distinguishing the 100,000 most toxic documents from the randomly sampled subset.
• Benchmark text generation models on their ability to produce or avoid content similar to the high-toxicity documents.

Strengths

• Contains 200,000 text documents, providing a substantial sample for analysis.
• Explicitly balanced construction with 100,000 high-toxicity and 100,000 random documents.
• Documents are sourced from The Pile, a known large-scale text corpus.

Limitations

• Toxicity scores are derived from the Perspective API, which may have its own biases and limitations.
• The dataset is a subset of a larger corpus (2.2 million documents), not a comprehensive sample.
• Data is from a single scoring period in May 2022, limiting temporal analysis.

Provenance

Source

The Pile, via tomekkorbak/pile-chunk-toxicity-scored-3 on Hugging Face.

Collection Method

Documents were scored using the Perspective API; the dataset combines the 100k most toxic and 100k randomly sampled documents.

Time Range

Documents from The Pile, scored on 2022-05-18 to 2022-05-20.

Freshness

Last updated 2022-05-20.

Geography

null