Chatbot Arena JA Calm2: Japanese Instruction Dataset for RLHF

Chatbot Arena Conversations JA (calm2) is a Japanese instruction dataset constructed for RLHF, as described in its associated paper. The dataset was created to test whether English datasets can be adapted for Japanese using only open-source tools and models. Prompts are Japanese translations of user inputs from the lmsys/chatbot_arena_conversations dataset, which are human-written and licensed under CC-BY 4.0.

Use Cases

• Fine-tuning Japanese LLMs for instruction-following based on translated human prompts.
• Training reward models for RLHF using single-turn conversational data.
• Benchmarking Japanese chatbot performance on human-generated queries.
• Studying the transferability of instruction-tuning methodologies from English to Japanese.

Strengths

• Prompts are derived from human-written inputs collected via the Chatbot Arena platform.
• Source user inputs are explicitly licensed under CC-BY 4.0.
• Dataset creation methodology is documented in an associated research paper.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• All conversations are limited to a single turn, as only the initial user input is used.

Provenance

Source

User inputs from lmsys/chatbot_arena_conversations, translated to Japanese.

Collection Method

Human-written English prompts were translated using open-source tools and models.

Freshness

Last updated 2024-08-15 04:51:49; freshness should be verified.