MemEye is a multimodal memory benchmark for evaluating agents that need to remember and reason over long-running image-grounded dialogues. The dataset contains user-centric multi-session conversations, associated images, and human-annotated questions. It was created by MemEyeBench and last updated on 2026-05-13.
Use Cases
- Benchmarking agent memory performance based on multi-session conversational data
- Evaluating multimodal reasoning capabilities based on image-grounded dialogue tasks
- Comparing model performance under constrained-choice and generative settings based on the provided multiple-choice and open-answer question formats
Strengths
- Provides tasks in both multiple-choice and open-answer formats for flexible model evaluation
- Includes human-annotated questions for ground-truth assessment
- Features multi-session, user-centric conversations for long-term memory testing
Limitations
- Column-level documentation is absent; field semantics must be inferred after download
- Row count is unknown, which may limit suitability assessment
- Description metadata is limited; actual data quality requires manual inspection after download
Provenance
- Source
- MemEyeBench
- Collection Method
- Likely collected and annotated for research purposes, as indicated by the associated GitHub repository.
- Freshness
- Last updated 2026-05-13 22:08:23; freshness should be verified