GenAI-Bench: A Human Preference Benchmark for Multimodal Reward Models

GenAI-Bench is a benchmark for evaluating multimodal large language models' ability to judge the quality of AI-generated content. The dataset is based on human preference data collected via the GenAI Arena platform and is maintained by TIGER-Lab. It was last updated on 2024-09-08.

Use Cases

• Benchmarking MLLMs as multimodal reward models based on the described evaluation framework.
• Training or fine-tuning models to predict human preferences for AI-generated content using the collected votes.
• Studying the alignment between model judgments and human preferences for generative AI outputs.

Strengths

• Dataset is explicitly designed for benchmarking multimodal reward models.
• Data is sourced from human preferences collected via the GenAI Arena platform.
• Votes were filtered using an NSFW filter, as mentioned in the description.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

TIGER-Lab via Hugging Face.

Collection Method

Human preference votes collected via the GenAI Arena platform, filtered with an NSFW filter.

Freshness

Last updated 2024-09-08 08:33:52; freshness should be verified.