Multifaceted Preference Collection For LLM Alignment

Multifaceted Collection is a dataset for aligning large language models to diverse human preferences, using system messages to represent individual preferences. The dataset was created by KAIST AI and released in June 2024. Instructions are sourced from five existing datasets.

Use Cases

• Fine-tuning LLMs to follow specific user preferences encoded in system_message fields.
• Training reward models to score responses based on preference annotations from diverse sources.
• Benchmarking the robustness of LLMs across varied instruction sets from the five source datasets.
• Studying the effect of system prompt conditioning on model output for alignment tasks.

Strengths

• Contains over 100,000 data points as indicated by the '100K<n<1M' size category.
• Built upon five established instruction datasets, providing a foundation of varied prompts.
• Associated with a peer-reviewed paper (arXiv:2405.17977) detailing its construction.

Limitations

• Specific row count, column names, and data structure details are not provided in the input.
• Potential bias inherited from the five source datasets used for instruction acquisition.
• Limited information on the distribution and balance of preferences represented.

Provenance

Source

KAIST AI Lab (lklab.kaist.ac.kr/Janus/).

Collection Method

Instructions acquired from five existing datasets; system messages used to encode preferences.

Freshness

Last updated on June 7, 2024.