RealToxicityPrompts: 100,000 Web Snippets for Model Safety Testing

RealToxicityPrompts contains 100,000 English sentence snippets extracted from the web by the Allen Institute for AI in 2020. It was developed to provide a standardized benchmark for researchers to quantify and mitigate the risk of neural toxic degeneration in large language models.

Use Cases

• Measuring toxic degeneration by feeding 'prompt' strings into LLMs and scoring the resulting completions
• Benchmarking toxicity classifiers against the 'challenging' subset of prompts to identify false negatives
• Analyzing the provenance of toxic content using 'filename' and offset metadata to understand source distributions

Strengths

• 100,000 unique records
• Includes 'challenging' labels to identify prompts that bypass standard safety filters
• Apache 2.0 permissive license

Limitations

• Contains highly offensive and toxic language by design
• Geographic and cultural bias inherent in English-language web scraping
• Temporal staleness as web-based slang and toxic tropes evolve over time

Provenance

Source

Allen Institute for AI (AI2)

Collection Method

scraped from the web

Time Range

2020

Freshness

Last updated in September 2022; the underlying data was collected for a 2020 research publication.