English-Chinese Preference Pairs for DPO Model Training

20,000 preference pairs for DPO training combine four curated sources: 4,000 from distilabel-capybara, 3,000 from distilabel-intel-orca, 3,000 from ultrafeedback, and 10,000 Chinese pairs. The dataset, created by shibing624, was last updated in April 2024. It contains fields like 'question', 'response_rejected', and 'response_chosen' formatted for ORPO and DPO model training.

Use Cases

• Train DPO models using the 'response_chosen' and 'response_rejected' fields to learn human preferences.
• Fine-tune ORPO models on the 'question' and paired response fields for alignment tasks.
• Analyze preference score distributions across the four source datasets using the original 'chosen score' thresholds.

Strengths

• 20,000 total preference pairs provide a substantial corpus for alignment training.
• Combines four distinct, high-quality sources, each filtered by specific 'chosen score' thresholds (>=4 or >=8).
• Specifically formatted with 'question', 'response_rejected', and 'response_chosen' fields for direct use in DPO/ORPO pipelines.

Limitations

• Sample data and column definitions are unavailable, hindering preliminary inspection of data structure.
• The dataset's composition from multiple sources may introduce inconsistencies in formatting or annotation style.
• No information on row-level metadata, licensing, or the original time range of the collected data.

Provenance

Source

Aggregated from four Hugging Face datasets: argilla/distilabel-capybara-dpo-7k-binarized, argilla/distilabel-intel-orca-dpo-pairs, argilla/ultrafeedback-binarized-preferences-cleaned, and wenbopan/Chinese-dpo-pairs.

Collection Method

Curated selection based on 'chosen score' thresholds from source datasets, with fields renamed for DPO/ORPO training.

Freshness

Last updated on 2024-04-27.