PKU-SafeRLHF-30K: 30,000+ Expert Comparisons for AI Safety

PKU-SafeRLHF-30K is a preference dataset containing over 30,000 expert comparison entries for AI safety research, created by the PKU-Alignment Team. Each entry includes a question and two responses with safety annotations. The dataset was last updated on November 20, 2023.

Use Cases

• Training safety-aligned reward models based on expert preference comparisons.
• Benchmarking AI model responses for safety and harmfulness.
• Researching methods to reduce harmful outputs in language models based on annotated safety data.

Strengths

• Contains over 30,000 expert comparison entries, providing a substantial base for analysis.
• Data is specifically intended for research to make AI models less harmful, offering a clear purpose.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• The dataset contains data that may be offensive or harmful, requiring careful handling.

Provenance

Source

PKU-Alignment Team

Collection Method

Likely consists of expert annotations comparing responses for safety.

Freshness

Last updated 2023-11-20 10:23:37; freshness should be verified.