PhoStream: 5,572 QA Pairs for Mobile Omnimodal Streaming Benchmarking

PhoStream contains 5,572 open-ended QA pairs derived from 578 videos across 4 scenarios and 10 capabilities, released by lucky-lance in 2026. This benchmark evaluates omnimodal assistants in mobile-centric streaming environments, focusing on both on-screen and off-screen phone usage. It specifically tests a model's ability to determine both the timing and the content of responses while processing continuous audio-visual streams.

Use Cases

• Evaluating omnimodal assistants on their ability to determine 'when to speak' based on continuous audio-visual streams
• Benchmarking reasoning capabilities using the 5,572 open-ended QA pairs
• Testing mobile-centric assistant performance across the 10 defined capability categories

Strengths

• 5,572 open-ended QA pairs
• 578 videos across 4 mobile-centric scenarios
• 10 distinct capability categories for omnimodal evaluation
• Unifies on-screen and off-screen mobile scenarios

Limitations

• Specific to mobile-centric scenarios which may not generalize to other robotics or desktop environments
• Relatively small video count of 578 compared to large-scale pre-training datasets

Provenance

Source

lucky-lance via Hugging Face

Collection Method

Annotated benchmark creation from mobile-centric video scenarios

Freshness

Last updated January 2026.