UltraFeedback Binarized Preference Data With Source Annotations

UltraFeedback binarized dataset has been cleaned by removing prompts identified as faulty and adding source annotations for filtering. The dataset was created by researchers at AllenAI and updated in December 2023.

Use Cases

• Train a reward model to score LLM outputs by comparing preferred and dispreferred responses.
• Fine-tune an instruction-following model using the source annotations to filter and balance training data from different origins.
• Benchmark alignment techniques by using the binarized preference pairs as ground truth for evaluating model preferences.
• Analyze the characteristics of faulty prompts removed during the cleaning process to improve dataset construction methods.

Strengths

• Dataset includes source annotations allowing for controlled filtering of samples.
• Cleaned version removes examples identified as faulty, potentially improving label quality.

Limitations

• Specific row count, column details, and sample size are unknown, making scale assessment difficult.
• The removal of TruthfulQA prompts reduces coverage of a specific evaluation domain.
• Potential for unknown biases or imbalances within the remaining source distributions.

Provenance

Source

AllenAI, derived from the original UltraFeedback dataset.

Collection Method

Binarized and cleaned version of a crowd-sourced preference dataset for LLM outputs.

Freshness

Last updated in December 2023.