Loading...
Loading...
Image-text pairs, instruction tuning, visual QA, cross-modal data, foundation model training data
1,956 datasets
Mobile3M consists of approximately 1,000 image-based records captured from Android Cuttlefish Emulators for pre-training Mobile Vision Language Models (MobileVLM). Released by Xiaomi Corporation in late 2024, the data supports research into mobile-specific vision-language tasks and UI interaction.
MME-RealWorld is a multimodal benchmark dataset for evaluating large language models, launched on August 20, 2024. The dataset includes a lite version with 50 samples per task for inference acceleration, as noted in the November 14, 2024 update. It is authored by yifanzhang114 and is supported by evaluation frameworks like VLMEvalKit and Lmms-eval.
Art-Free-SAM contains filtered image segment IDs from the original SA-1B dataset. The dataset pairs these segments with captions sourced from SAM-LLaVA-Captions10M, organized in a hierarchical folder structure. The dataset was authored by rhfeiyang and last updated on Hugging Face in December 2024.
16,130 images and 1,353 human-annotated multiple-choice questions across 9 distinct scenarios form this benchmark for evaluating vision-centric multimodal retrieval-augmented generation (RAG) abilities in Large Vision Language Models (LVLMs). The dataset, named MRAG-Bench, was created by uclanlp and last updated on November 5, 2024. It provides a systematic evaluation framework for both open-source and proprietary models.
A multimodal dataset from HuggingFace, created by 5CD-AI and last updated on 2024-11-27. The description suggests it contains examples of visual reasoning tasks where models are instructed to explain their reasoning step-by-step before providing a final answer, as shown in a provided example about counting straws and cups.
AllenAI created this dataset of 19,890 synthetically generated preference examples to enhance models' precise instruction-following capabilities. It contains chosen and rejected response pairs, intended for preference tuning methods like PPO and DPO. The dataset was last updated on November 21, 2024.
CVQA is a culturally diverse multilingual visual question answering benchmark consisting of over 10,000 questions from 39 country-language pairs. The dataset was constructed through a collaborative effort led by researchers from MBZUAI and is designed for use as a test set. It was last updated on November 27, 2024.
SEAGULL-100w is a large-scale synthetic dataset for no-reference image quality assessment focused on regions of interest. It was created by Zevin2023 and includes images with six distortion types: blur, sharpness, exposure, contrast, colorfulness, and compression. The dataset was last updated on the Hugging Face platform in November 2024.
Safety Multimodal Jailbreaking is a dataset hosted on HuggingFace by author leeeliu. The dataset was last updated on 2026-01-28, suggesting ongoing maintenance. Its title indicates it likely contains examples for testing or bypassing safety measures in multimodal AI systems.
MMC is a multimodal instruction-tuning dataset for chart understanding published by Fuxiao Liu for NAACL 2024. It provides visual chart data paired with natural language instructions to improve the reasoning capabilities of large language models across various chart types, including stock market visualizations.
PD3M is a subset of the PD12M dataset, containing 3.3 million image-caption pairs filtered for the highest aesthetic scores. PD12M is the largest public domain image-text dataset to date, designed for training foundation models while minimizing copyright concerns. The dataset was created by Spawning and introduces community-driven governance mechanisms via the Source.Plus platform.
NUSTM developed this multimodal dataset for explainable depression recognition in clinical interviews, with the most recent update occurring in January 2025. It provides data for affective computing research, specifically focusing on the intersection of mental health and machine learning interpretability.
ReINTEL is a multimodal data challenge for identifying responsible information on social network sites. The dataset is associated with a competition hosted on AIHub, with top solutions invited to submit technical reports. It was created by ReliableAI and last updated in November 2024.
PVIT-3M is a dataset of 3 million image-text pairs designed for tuning Multimodal Large Language Models (MLLMs) on personalized visual instruction tasks. It was created by Sterzhang and introduced in the paper "Personalized Visual Instruction Tuning". The dataset was last updated on November 2, 2024.
5,000 feature explanations generated for a 131k sparse autoencoder trained on the llava-next-llama3-8B vision-language model. The dataset includes two versions: a 'revised' set using an updated prompt and cached data, and a 'legacy' set using an older prompt and a subset of the LLaVA-NeXT-Data. It was created by lmms-lab and last updated on November 22, 2024.
Released in 2024, TemporalBench is a video understanding benchmark designed to evaluate fine-grained temporal reasoning for multimodal video models. It consists of approximately 10,000 video question-answer pairs sourced from around 2,000 high-quality human-annotated video captions. The dataset was created by Microsoft.
2,000 rows of preference data for Direct Preference Optimization (DPO) fine-tuning, structured with prompt, chosen, and rejected fields. The chosen responses and prompts are sourced from the iamtarun/python_code_instructions_18k_alpaca dataset, while rejected responses are generated by a base LLAMA 3.1 model. The dataset was uploaded by quangduc1112001 to Hugging Face and last updated on November 4, 2024.
OpenSpaces is a synthetic dataset for spatial visual question answering created using VQASynth. It synthesizes data from the first 30,000 rows of the localized narratives split of the cauldron, emphasizing greater diversity in image distribution compared to related datasets. The dataset was authored by remyxai and last updated on October 25, 2024.
Open-LLaVA-NeXT 1M is a 1 million sample dataset for supervised fine-tuning, created to reproduce the LLaVA-NeXT model series. The author augmented the sharegpt4v_mix665k dataset and attempted to align with LLaVA-NeXT's training data, substituting inaccessible user interaction data with 200K samples from ALLaVA-Instruct-VFLAN-4V. This dataset was uploaded to Hugging Face by Lin-Chen on October 25, 2024.
491 images from the CountBench benchmark evaluate object counting in vision-language models. The dataset was automatically curated and manually verified from the LAION-400M dataset, introduced by author vikhyatk for the PaliGemma model.