Chatml Openhermes2.5 Dpo Binarized Alpha: Preference-Tuned LLM Training Data

A Direct Preference Optimization dataset derived from argilla/OpenHermes2.5-dpo-binarized-alpha, filtered and sorted for curriculum learning. It implements a ChatML format compatible with Axolotl training and removes 530 low-scoring samples from the training set and 66 from the test set. The dataset was created by mlabonne and last updated on March 21, 2024.

Use Cases

• Fine-tuning language models with Direct Preference Optimization based on the described DPO structure.
• Implementing curriculum learning strategies for LLM training based on the dataset's sorting by 'delta_score'.
• Training models using the Axolotl framework based on the integrated ChatML format.
• Studying the effect of filtering low-quality preference pairs based on the described score threshold.

Strengths

• Implements a specific ChatML format for direct use with the Axolotl training framework.
• Filters low-quality samples by removing 530 from the training set and 66 from the test set where the delta score was less than 1.
• Sorted by 'delta_score' to facilitate curriculum learning during model training.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.

Provenance

Source

Derived from argilla/OpenHermes2.5-dpo-binarized-alpha.

Collection Method

Filtered and sorted from an existing DPO dataset.

Freshness

Last updated 2024-03-21 12:46:44; freshness should be verified.