CompaRAG Tool Votes: Human Preference Data for MCP Tool Comparison

Human preference votes collected on the CompaRAG blind comparison platform for Model Context Protocol (MCP) tools. Users submitted a task and goal, voted for the best anonymous tool response, and the data was compiled by ArthurSrz. The dataset was last updated on May 16, 2026.

Use Cases

• Benchmarking MCP tool performance based on direct human preference votes.
• Training or fine-tuning reward models for LLM alignment using human feedback.
• Analyzing task characteristics that influence user preference for different tool outputs.
• Studying the reliability of blind, pairwise comparisons for evaluating AI systems.

Strengths

• Data is derived from a structured, blind comparison platform designed to reduce bias.
• The collection method involves a specific user workflow of submitting a task, goal, and voting.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Freshness should be verified as the last update timestamp is from the future (2026).

Provenance

Source

The Borges Graph, as a spinoff of the French government initiative comparIA (Ministère de la Culture / Beta.gouv.fr).

Collection Method

Collected via user votes on the CompaRAG platform where users compare anonymous MCP tool outputs.

Freshness

Last updated 2026-05-16 15:00:19