Multimodal Feedback Data for Reinforcement Learning from Human Feedback

RLHF-V-Dataset is a large-scale multimodal feedback dataset constructed using open-source models for reinforcement learning. It was released by the openbmb organization in May 2024 and has been utilized in models like MiniCPM-V 2.0. The dataset is designed for diverse tasks involving computer vision and large language models.

Use Cases

• Train multimodal large language models using feedback data for tasks like image captioning or visual question answering.
• Fine-tune reinforcement learning algorithms on human or AI-generated preference data for alignment tasks.
• Benchmark the performance of open-source vision-language models against the feedback annotations provided.

Strengths

• Dataset is described as 'large-scale' and designed for 'diverse-task' multimodal applications.
• Associated with published research, including a paper accessible on arXiv as of May 2024.
• Has been used in production models, specifically cited in the development of MiniCPM-V 2.0.

Limitations

• Specific structural details like row count, column names, and file formats are unknown.
• The dataset's construction method using open-source models may introduce biases or noise inherent to those models.
• Limited information is available on the temporal or geographic coverage of the underlying data.

Provenance

Source

openbmb organization on Hugging Face.

Collection Method

Constructed using open-source models, likely for reinforcement learning from human or AI feedback (RLHF/RLAIF).

Freshness

Last updated on 2024-05-28, with a new related dataset (RLAIF-V-Dataset) released in May 2024.