MovieChat: Long Video Understanding via Sparse Memory for CVPR 2024

MovieChat is a multimodal dataset and framework for long video understanding, introduced by researcher rese1f for the CVPR 2024 conference. It focuses on bridging the gap between dense video tokens and large language models using a specialized sparse memory mechanism. The data supports research into temporal reasoning and long-form video processing.

Use Cases

• Training multimodal LLMs for long-form video question answering using sparse memory tokens
• Evaluating temporal reasoning capabilities across extended video durations
• Benchmarking dense-to-sparse token compression for video-language alignment

Strengths

• Peer-reviewed methodology from CVPR 2024
• Focuses on long-form video understanding rather than short clips
• Compatible with Llama-based multimodal large language models

Limitations

• Specific row count and dataset scale are not explicitly defined in the metadata
• Highly specialized for the MovieChat sparse memory architecture
• Lack of detailed column-level documentation in the source

Provenance

Source

rese1f via GitHub and CVPR 2024

Collection Method

Annotated video-text pairs for research

Freshness

Last updated in January 2025; reflects state-of-the-art research from CVPR 2024.