Binarized preference pairs across multiple rating dimensions curated from the Ultrafeedback and Zephyr train_prefs datasets. The data identifies 'chosen' and 'rejected' responses by calculating the mean of preference ratings to resolve mismatches between original scores and actual response quality.
Use Cases
- Train reward models by comparing the 'chosen' and 'rejected' response columns
- Fine-tune models using Direct Preference Optimization (DPO) on the binarized pairs
- Audit the 'overall_score' column from the original Ultrafeedback dataset for quality mismatches
Strengths
- Binarized format featuring 'chosen' and 'rejected' response pairs
- Labels derived from the mean of preference ratings rather than a single 'overall_score'
- Curated through visual inspection and filtering within the Argilla platform
- Derived from the original Ultrafeedback and Zephyr train_prefs datasets