MLLM Pairwise Visual Preference Judgments for Highway Landscape Scenes (Malaysia NSE)

Malaysia's North–South Expressway is the subject of this dataset containing 18,960 forced-choice judgments from three multimodal large language models (MLLMs). The judgments result from pairwise comparisons of 80 highway landscape images, supporting Bradley–Terry analysis and position bias diagnosis. Hangyu Gao authored the dataset, which was last updated on May 20, 2026.

Use Cases

• Benchmarking MLLM consistency and agreement based on the 18,960 judgments from three independent models.
• Analyzing position bias in AI judgments based on the AB and BA presentation orders used in the study.
• Training or validating ranking models using the Bradley–Terry analysis framework supported by the dataset.
• Comparing AI and human visual preferences for landscape scenes based on the associated manuscript's reported analysis.

Strengths

• Contains 18,960 forced-choice judgments, providing a substantial basis for statistical analysis.
• Includes comparisons from three independently developed MLLMs (Claude Sonnet 4.6, ChatGPT 5.4, Gemini 3 Flash Preview).
• Covers all 3,160 unique image pairs in both AB and BA presentation orders, enabling bias diagnosis.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• The dataset's 703.3 KB size indicates a limited scope focused on a specific set of 80 images.

Provenance

Source

Hangyu Gao

Collection Method

Output from querying three MLLMs on pairwise comparisons of 80 highway landscape images.

Time Range

null

Freshness

Last updated 2026-05-20 05:11:42; freshness should be verified.

Geography

Malaysia's North–South Expressway