DiscoverLLM: Multi-turn Dialogue Preferences for LLM Training

DiscoverLLM-multiturn-preferences is a dataset of multi-turn dialogue data with scored candidate completions. It was produced by best-of-N synthesis over the DiscoverLLM user simulator and is authored by kixlab. The dataset was last updated on 2026-05-13.

Use Cases

• Training offline Direct Preference Optimization (DPO) models based on scored dialogue completions.
• Training offline Group Relative Policy Optimization (GRPO) models based on scored dialogue completions.
• Training reward models for dialogue agents based on the provided reward scores.
• Fine-tuning language models for creative writing tasks, such as poem generation, based on the 'creative_writing' task subset.

Strengths

• Contains 3,052 examples specifically for the 'creative_writing' task.
• Data is structured for offline preference optimization, a core method in LLM alignment.

Limitations

• The total row count across all tasks is unknown.
• Column-level documentation is absent; field semantics must be inferred after download.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

kixlab via Hugging Face.

Collection Method

Produced by best-of-N synthesis over the DiscoverLLM user simulator.

Freshness

Last updated 2026-05-13 13:09:25.