TVR: Large-Scale Video-Subtitle Moment Retrieval with Multimodal Queries

TVR provides video-subtitle pairs and natural language queries for temporal moment retrieval, introduced by Jie Lei at ECCV 2020. The collection focuses on the TV show domain, requiring models to utilize both visual and textual dialogue features to locate specific events.

Use Cases

• Temporal moment localization using natural language queries and subtitle timestamps
• Cross-modal retrieval by matching text descriptions to visual features and dialogue
• Multimodal representation learning using video-subtitle pairs

Strengths

• ECCV 2020 peer-reviewed source
• Includes both visual frames and synchronized subtitle text
• MIT licensed for open research

Limitations

• Restricted to TV show content which may not generalize to other video domains
• Requires significant storage and GPU resources for video feature extraction

Provenance

Source

ECCV 2020 paper 'TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval'

Collection Method

Human annotation of TV show clips and subtitles

Freshness

Last updated May 2024