Loading...
Loading...
Image-text pairs, instruction tuning, visual QA, cross-modal data, foundation model training data
1,956 datasets
Atsunori converted the NVIDIA HelpSteer2 dataset into preference pairs for training Direct Preference Optimization models. The conversion is based on the helpfulness score of responses, with the higher-scoring response designated as the chosen one. The dataset was last updated on July 11, 2024.
A collection of 21,930,344 synthetic English captions for 10,965,172 images from the conceptual_12m dataset. The captions were generated using the llama3-llava-next-8b model, followed by cleanup and shortening with Meta-Llama-3-8B. The dataset was created by CaptionEmporium and last updated on Hugging Face in June 2024.
A multimodal mathematics dataset collected from real middle school exams in China, featuring open-ended problems. It was created by THU-KEG and last updated on June 30, 2024. The dataset is annotated with fine-grained three-dimensional labels for difficulty, grade, and knowledge points.
A curated collection of five established code instruction datasets formatted for LLM training. The datasets, including Magicoder-OSS-Instruct-75K and glaive-code-assistant-v3, have been processed into the LLAMA chat format with markdown for code snippets. It was created by MaLA-LM and last updated in July 2024.
A subset of the VQAv2 dataset, which is a benchmark for visual question answering tasks. The dataset was published on the Hugging Face platform by user 'merve' and was last updated on August 8, 2024. The specific scale, content, and structure of this 'Small' version require verification after download.
PMC-VQA contains 227,000 visual question-answering pairs associated with 149,000 medical images sourced from PubMed Central. Released by RadGenome and updated in July 2024, the collection includes a specialized version focused on noncompound images to facilitate cleaner model training. The dataset is organized into training and testing splits with a dedicated clean test set for benchmarking.
Approximately 38,000 image-text pairs, with 10,000 sourced from LAION and 28,000 from nsfw_detect. Captions were generated by the LLaVA-NeXT model using a prompt to describe attributes of people. The dataset was created by author zxbsmk and last updated on HuggingFace in July 2024.
Conceptual Captions contains approximately 3.3 million images paired with captions. The captions are raw descriptions harvested from the Alt-text HTML attribute of web images, representing a wider variety of styles than curated annotations. The dataset was created by google-research-datasets.
BenchLMM evaluates the cross-style visual reasoning capabilities of Large Multimodal Models (LMMs) across diverse image styles. Developed by AIFEG and presented at ECCV 2024, this benchmark assesses how models generalize beyond standard natural images to various artistic and synthetic domains.
A subset of 12 million image-text pairs from the DataComp-1B-BestPool collection, released by mlfoundations in 2024. The dataset is designed for training image-text models and is licensed under Creative Commons CC-BY-4.0, though individual images retain their original copyrights. It was introduced in the MobileCLIP paper and is reported to yield better model performance than several established benchmarks.
700,000 Vietnamese vision-language samples were generated using Gemini Pro and prompt engineering techniques like few-shot learning and caption-based prompting. The dataset was created by Vi-VLM and was last updated in June 2024.
Multimodal-Mind2Web aligns HTML documents from the Mind2Web dataset with their corresponding webpage screenshot images. The dataset was created by osunlp to address the inconvenience of loading images from the original ~300GB raw dump and was last updated on June 5, 2024.
A question-answer dataset formatted for fine-tuning large language models. The data is sourced from PDF and markdown files extracted from various project repositories within the Cloud Native Computing Foundation landscape. It was created by Kubermatic and last updated on June 27, 2024.
A benchmark for evaluating embodied spatial understanding in Large Vision-Language Models, created by Phineas476 and last updated on June 23, 2024. It comprises 3,640 question-answer pairs automatically derived from embodied scenes, covering 294 object categories and 6 spatial relationships from an egocentric perspective. The associated EmbSpatial-SFT dataset provides instruction-tuning data for spatial tasks.
Vript is a fine-grained video-text dataset constructed by Mutonix, containing 12,000 annotated high-resolution videos split into approximately 400,000 clips. The annotation is inspired by video scripts, detailing scene content, shot types, and camera movements. The dataset was last updated on June 11, 2024.
An extension of the CommonCatalog CC-BY dataset with Japanese-language image captions. The author alfredplpl added one simple and three detailed captions per image, generated by a modified LLaVA-JP model. The dataset was last updated on June 23, 2024.
Approximately 10,000 synthetic Japanese conversation records generated for instruction tuning of large language models. Author Aratako created the dataset by applying the Magpie method to the NVIDIA Nemotron-4-340B-Instruct model via DeepInfra and published the generation code on July 5, 2024.
A small dataset of synthetic text captions describing food and non-food images. The text captions were generated using the Mistral Chat and Mixtral language models. It was created by author mrdbourke and last updated on June 7, 2024.
MathV360K is a multimodal dataset containing 360,000 question-answer pairs and 40,000 images sourced from 24 datasets. It was created by Zhiqiang007 and uploaded to Hugging Face on 2024-06-27 to enhance the mathematical reasoning capabilities of multimodal large language models.
Dense English captions for the CommonCatalog CC-BY image collection generated via the Phi-3 Vision model. The data is structured in a CSV format where each entry is linked to the original image repository through a unique photoid primary key.