141,564 natural language referring expressions mapped to 49,856 objects across 19,992 MS COCO images. The dataset provides bounding box coordinates and segmentation masks for each referred object, paired with multiple human-annotated descriptions that avoid spatial language.
Use Cases
- Train referring expression comprehension models to output `bbox` coordinates from text input.
- Develop referring expression generation models that produce natural language descriptions for specific `segmentation` masks.
- Benchmark vision-language models on their ability to distinguish objects using visual attributes without relying on spatial cues.
Strengths
- Contains 141,564 referring expressions for 49,856 distinct objects.
- Includes precise `bbox` coordinates and `segmentation` masks for every referred instance.
- Features a split structure including `testA` for human subjects and `testB` for non-human objects to facilitate granular performance analysis.