Human Preference Data for Ranking Large Language Models on Medical Anatomy

A 5.6 MB dataset of human preference outcomes used to evaluate large language models. The data supports a novel statistical framework for online decision-making and inference in Reinforcement Learning from Human Feedback, proposed by author Nan Lu. It was last updated on 2026-05-18 and applied to analyze model performance on the Massive Multitask Language Understanding dataset.

Use Cases

• Benchmarking large language model performance based on human preference data mentioned in the description
• Training or evaluating Reinforcement Learning from Human Feedback algorithms using dynamic contextual information
• Conducting statistical inference on optimal model parameters from dependent online human feedback

Strengths

• Dataset size is 5.6 MB, indicating a focused and manageable scale for analysis.
• The underlying method is described as achieving optimal regret bound and asymptotic normality of estimators.
• Application to the Massive Multitask Language Understanding dataset provides a concrete evaluation context.

Limitations

• Row count is unknown, which may limit suitability assessment for large-scale training.
• Column-level documentation is absent; field semantics must be inferred after download.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

figshare, author Nan Lu

Collection Method

Likely contains human preference data generated for ranking large language models, as described in the associated research paper.

Freshness

Last updated 2026-05-18 13:46:03; freshness should be verified.