Commonsense Inference Questions for Preference Tuning

HellaSwag is a dataset of commonsense inference questions designed to be challenging for large language models. It was adapted into a paired preference-ranked dataset by AbacusAI in February 2024, creating three preference pairs for each correct response in the training split. Each instance includes a prompt with one correct and three incorrect completions.

Use Cases

• Training reward models by learning to rank correct completions over incorrect ones using the prompt and paired completion data.
• Fine-tuning language models via direct preference optimization (DPO) on the generated preference pairs.
• Benchmarking language model performance on commonsense reasoning tasks using the original prompt and completion structure.
• Analyzing model failure modes on specific commonsense inference questions present in the prompt and completion fields.

Strengths

• Derived from the established HellaSwag benchmark for commonsense reasoning.
• Provides structured preference pairs specifically formatted for DPO training.

Limitations

• Original dataset size and specific row count for this DPO version are unknown.
• Potential for label noise or bias inherent in the original HellaSwag construction method.

Provenance

Source

Derived from the original HellaSwag dataset.

Collection Method

Created by generating three preference pairs for each correct response in the HellaSwag training split.

Time Range

null

Freshness

Last updated on 2024-02-26.

Geography

null