Toxic Preference Optimization Dataset for Model De-alignment

2023-12-26 dataset from unalignment illustrates using direct preference optimization (DPO) to de-censor language models. It contains toxic and harmful text examples, many with attached warnings or disclaimers.

Use Cases

• Train a DPO model on toxic text examples to study model de-alignment and censorship removal.
• Analyze the frequency and structure of warnings and disclaimers attached to harmful content examples.
• Benchmark safety filters against a dataset of profanity and harmful text for robustness testing.

Strengths

• Created by unalignment, a known entity in AI alignment research.
• Dataset is tagged as 'Not For All Audiences', indicating clear content warnings.
• Last updated on 2023-12-26, providing a recent snapshot.

Limitations

• Dataset size, row count, and column structure are unknown.
• Content is described as 'somewhat editorialized' with warnings, potentially altering raw examples.
• Limited to US region data, which may not reflect global linguistic patterns of toxicity.

Provenance

Source

huggingface

Collection Method

null

Time Range

null

Freshness

Last updated 2023-12-26.

Geography

US