Orca Preference Pairs for DPO Training

Intel released 12,000 examples of preference data pairs formatted in the Orca style for direct preference optimization training. The dataset is derived from the OpenOrca collection and was uploaded to Hugging Face in November 2023.

Use Cases

• Fine-tune a language model's output preference using DPO on 12k chosen vs. rejected response pairs.
• Train a reward model to rank text completions based on human-like preferences inherent in the Orca-style data.
• Benchmark DPO algorithm performance against other alignment methods using a standardized preference dataset.

Strengths

• Contains 12,000 distinct examples for training.
• Follows the structured Orca format for instruction and response pairs.

Limitations

• Unknown row count for individual chosen/rejected pairs within the 12k examples.
• Potential bias inherited from the underlying OpenOrca source data.

Provenance

Source

Open-Orca/OpenOrca dataset on Hugging Face.

Collection Method

Curated and formatted by Intel from the OpenOrca collection.

Freshness

Last updated November 2023.