HelpSteer3: NVIDIA Dataset for Training High-Performance Reward Models

NVIDIA's HelpSteer3 contains between 100,000 and 1,000,000 multilingual text records designed for training Reward Models in LLM alignment. Released in 2025, it provides preference and feedback data across 14 languages to improve model helpfulness. The dataset supports high-performance alignment, reaching 85.5% on RM-Bench.

Use Cases

• Training Bradley-Terry Reward Models using the preference labels in the HelpSteer3-Preference subset
• Developing Generative Reward Models for Llama 3.3 Nemotron Super 49B v1 using the feedback labels
• Aligning multilingual LLMs for helpfulness across 14 languages including Korean and Polish

Strengths

• Multilingual support for 14 languages including Indonesian, Dutch, and Vietnamese
• High benchmark performance with 85.5% accuracy on RM-Bench
• Large scale containing between 100,000 and 1,000,000 records
• Permissive CC-BY-4.0 license for open-source use

Limitations

• Specific column names and schema are not explicitly defined in the provided metadata
• Potential for model-specific bias as it is optimized for Llama 3.3 architectures

Provenance

Source

NVIDIA

Collection Method

Annotated

Freshness

Last updated November 2025.

Geography

Global