Image-to-Video Human Preference Annotations for Two Models

A dataset of 2,000 human preference annotations for evaluating image-to-video generation. Each row contains a reference image, two generated videos from Pika and CogVideoX models, and 10 human annotations aggregated via majority vote. Created by datapointai and last updated in March 2026.

Use Cases

• Training or fine-tuning reward models for RLHF using the aggregated human preference annotations between Pika and CogVideoX outputs.
• Benchmarking the performance of image-to-video models like Pika against CogVideoX using the majority vote preference labels.
• Analyzing the correlation between video features and human preference scores across 20,000 individual annotations.
• Studying the consistency of human judgment in video quality assessment via the 10 annotations collected per sample pair.

Strengths

• Contains 2,000 unique comparison samples for model evaluation.
• Aggregates 10 human annotations per sample, totaling 20,000 individual preference judgments.

Limitations

• Limited to 2,000 rows in this medium subset, which may constrain statistical power for some analyses.
• Geographic bias likely present as the data region is indicated as 'us' (United States).
• Direct comparison is limited to outputs from only two specific models, Pika and CogVideoX.

Provenance

Source

datapointai via Hugging Face.

Collection Method

Human annotations collected and aggregated via majority vote for pairs of AI-generated videos.

Freshness

Last updated March 13, 2026.

Geography

United States (based on platform tag 'Regionus').