ChatML DPO Pairs: 12,000 Preference Examples for LLM Alignment

12,000 examples of preference pairs formatted in ChatML, derived from the OpenOrca dataset. This collection was created by mlabonne and last updated on April 11, 2024. It is a preprocessed version of the Intel/orca_dpo_pairs dataset, specifically structured for direct preference optimization training workflows.

Use Cases

• Fine-tuning LLMs for instruction following based on preference-ranked examples.
• Training reward models for reinforcement learning from human feedback (RLHF) pipelines.
• Benchmarking DPO algorithms using a standardized ChatML-formatted dataset.
• Studying the impact of system prompt formatting on model response quality.

Strengths

• Contains 12,000 examples, providing a substantial base for training.
• Uses the structured ChatML format, which is a common standard for conversational AI.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown beyond the stated 12k examples, which may limit suitability assessment.
• Last updated 2024-04-11 09:09:03; freshness should be verified.

Provenance

Source

Derived from Intel/orca_dpo_pairs and Open-Orca/OpenOrca.

Collection Method

Preprocessed using a custom script to convert examples into ChatML format.

Freshness

Last updated 2024-04-11 09:09:03.