Ultrafeedback Binarized Preferences Cleaned for DPO Training

A Direct Preference Optimization (DPO) dataset derived from the Argilla UltraFeedback Binarized Preferences dataset. It has been cleaned by filtering out samples with delta scores below 1, removing 530 from the training set and 66 from the test set. The dataset is sorted by the 'delta_score' column for curriculum learning and is formatted for direct use in Axolotl.

Use Cases

• Direct Preference Optimization (DPO) training based on the dataset's stated purpose
• Curriculum learning for LLM fine-tuning based on the 'delta_score' sorting
• Training instruction-following models using the Intel ChatML format mentioned in the description

Strengths

• Explicitly filtered for quality by removing 530 low-scoring training samples and 66 test samples
• Sorted for curriculum learning using the 'delta_score' column
• Pre-formatted for direct use in the Axolotl training framework

Limitations

• Column-level documentation is absent; field semantics must be inferred after download
• Row count is unknown, which may limit suitability assessment
• Description metadata is limited; actual data quality requires manual inspection after download

Provenance

Source

Derived from argilla/ultrafeedback-binarized-preferences-cleaned

Collection Method

Filtered and sorted from an existing preference dataset

Freshness

Last updated 2024-03-27 10:31:23