Compar:IA: French-Language Chatbot Conversations and Human Preferences

Compar:IA is a public chatbot arena run by the French Ministry of Culture. The dataset contains side-by-side conversations where users chat with two anonymous models and indicate their preferred answer. Each row represents one turn of a conversation, including the two model answers and the user's preference for that turn.

Use Cases

• Train reward models for reinforcement learning from human feedback (RLHF) based on the explicit user preference labels.
• Benchmark French-language conversational AI models based on side-by-side human evaluations.
• Analyze patterns in human preference for different model responses within multi-turn conversations.
• Study the characteristics of French-language prompts and model interactions from a public platform.

Strengths

• Data originates from a public platform run by a government ministry (French Ministry of Culture), suggesting a structured collection process.
• Includes explicit human preference judgments (win/loss/tie) for model responses, which is a key feature for alignment research.
• Captures full conversation context for each preference judgment, allowing for turn-level and dialogue-level analysis.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count, file formats, and data size are unknown, which may limit suitability assessment.
• Data may reflect geographic and linguistic bias inherent to its source, being exclusively French-language from a specific platform.

Provenance

Source

French Ministry of Culture via the Compar:IA public chatbot arena.

Collection Method

Collected from users interacting side-by-side with two anonymous AI models and voting on their preferred response.

Freshness

Last updated 2026-07-06 04:39:14; freshness should be verified.

Geography

Likely focused on French-speaking users, though specific geography is not stated.