Instruct Ppo Mix 20K: A 20,000-Row Subset for RLHF Training

A 20,000-row dataset compiled on 2024-11-06 by allura-org for training reinforcement learning from human feedback (RLHF) models. It is a mixture of 10,000 rows from the Gryphe/Sonnet3.5-SlimOrcaDedupCleaned-20k dataset and 10,000 rows from the arcee-ai/EvolKit-20k dataset, converted to the OpenRLHF prompt dataset format.

Use Cases

• Training Proximal Policy Optimization (PPO) models based on the described RLHF prompt format.
• Fine-tuning language models for instruction-following based on the mixture of instruction-tuning datasets.
• Benchmarking RLHF training pipelines based on the structured, combined subset from two established sources.

Strengths

• Combines 10,000 rows from two distinct, established instruction-tuning datasets.
• Specifically formatted for OpenRLHF, a known framework for RLHF training.

Limitations

• Row count is unknown beyond the initial 20,000 subset, which may limit suitability assessment.
• Column-level documentation is absent; field semantics must be inferred after download.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

A random subset from Gryphe/Sonnet3.5-SlimOrcaDedupCleaned-20k and arcee-ai/EvolKit-20k.

Collection Method

Random subset selection and format conversion.

Freshness

Last updated 2024-11-06 12:06:06.