UltraFeedback Chinese Preference Dataset for Reward Models

UltraFeedback-Chinese is a Chinese-language dataset constructed using the UltraFeedback methodology for training reward and critic models. The dataset supports PPO and DPO training methods and was created by the OpenCSG community, with its latest update in January 2025.

Use Cases

• Train a reward model to score Chinese text completions based on quality annotations.
• Fine-tune a critic model using preference pairs to distinguish between high and low-quality Chinese responses.
• Perform Direct Preference Optimization (DPO) on Chinese LLMs using the provided preference data.
• Benchmark the performance of different alignment algorithms (PPO vs. DPO) on Chinese language tasks.

Strengths

• Dataset is based on the established UltraFeedback construction methodology.
• Specifically designed for two core reinforcement learning training methods: PPO and DPO.

Limitations

• Key quantitative details like row count, size, and sample data are unavailable.
• The specific text features, annotations, and column structure are not described.
• Potential geographic or demographic bias is unknown as the data's origin is unspecified.

Provenance

Source

OpenCSG community.

Collection Method

Constructed based on the UltraFeedback dataset methodology.

Freshness

Last updated on 2025-01-14.