Tulu 3 Pref Personas: 19,890 Synthetic Preference Pairs for Instruction Following

AllenAI created this dataset of 19,890 synthetically generated preference examples to enhance models' precise instruction-following capabilities. It contains chosen and rejected response pairs, intended for preference tuning methods like PPO and DPO. The dataset was last updated on November 21, 2024.

Use Cases

• Training models for precise instruction following based on the described preference pairs.
• Applying preference tuning methods like DPO based on the chosen and rejected response structure.
• Fine-tuning language models to satisfy multiple constraints as mentioned in the dataset's purpose.

Strengths

• Contains 19,890 preference examples, providing a substantial volume for training.
• Specifically designed for enhancing instruction-following capabilities under constraints.

Limitations

• Dataset is synthetically generated, which may introduce artifacts not present in human-generated data.
• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.

Provenance

Source

AllenAI

Collection Method

Synthetically created from a subset of a supervised-tuning version.

Freshness

Last updated 2024-11-21 15:40:43.