Cvalues RLHF: English Preference Data for Direct Preference Optimization

A translated dataset for Direct Preference Optimization (DPO) derived from the Skepsun/cvalues_rlhf source. The prompt and rejected response fields contain outputs from the huihui-ai/Huihui-gpt-oss-20b-mxfp4-abliterated-v2 model, while the chosen response field uses outputs from openai/gpt-oss-20b. The dataset was created by author puwaer and last updated on November 15, 2025.

Use Cases

• Training preference models based on chosen and rejected text pairs.
• Fine-tuning language models for alignment using Direct Preference Optimization.
• Benchmarking model outputs against human or model-based preferences.
• Studying the characteristics of text generated by different LLM architectures (e.g., huihui-ai vs. openai models).

Strengths

• Designed specifically for Direct Preference Optimization (DPO), a key alignment technique.
• Provides a structured comparison of outputs from two distinct LLM sources (huihui-ai and openai).
• Last updated on 2025-11-15, indicating recent maintenance.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Data may reflect model bias inherent to the specific LLMs used for generation.

Provenance

Source

Based on Skepsun/cvalues_rlhf, translated into English by puwaer.

Collection Method

Text outputs generated by specified LLMs (huihui-ai/Huihui-gpt-oss-20b-mxfp4-abliterated-v2 and openai/gpt-oss-20b) to create preference pairs.

Time Range

null

Freshness

Last updated 2025-11-15 08:06:22; freshness should be verified.

Geography

null