Real Toxicity Continuations for Language Model Evaluation

Real Toxicity Continuations is a text dataset for evaluating the toxicity of language model outputs. It was created by user 'sasha' and last updated on Hugging Face in July 2022. The dataset contains prompts and continuations, likely sourced from models like GPT-2, to measure the propensity of language models to generate harmful text.

Use Cases

• Benchmarking toxicity levels in text continuations from different language models.
• Training or fine-tuning toxicity classifiers using the provided prompt and continuation text fields.
• Analyzing the relationship between specific prompt words and the toxicity of model-generated continuations.
• Studying the distribution of toxic vs. non-toxic continuations across different model architectures.

Strengths

• Dataset is hosted on a major platform (Hugging Face) with established access tools.
• Platform tags indicate the dataset size is between 10,000 and 100,000 rows (Size Categories10 Kn100 K).
• Available in Parquet format for efficient data loading and processing.

Limitations

• Specific row count, column names, and exact data schema are not provided in the input.
• The dataset's creation methodology and original source are not detailed.
• Data is from 2022 and may not reflect the latest language model behaviors or toxicity definitions.

Provenance

Source

Hugging Face user 'sasha'.

Collection Method

Likely involves generating text continuations from language models (e.g., GPT-2) and annotating them for toxicity.

Freshness

Last updated on 2022-07-06.

Geography

Platform tags suggest a US region (Regionus), but data content geography is unknown.