MobileDev-Bench: 407 Real-World Mobile App Issue-Resolution Tasks

407 verified real-world issue-resolution tasks collected from 19 production mobile app repositories. MobileDev-Bench evaluates large language models on their ability to resolve authentic developer-reported issues in Android Native, React Native, and Flutter applications. The benchmark was created by the MobileDev-Bench organization and was last updated on 2026-05-07.

Use Cases

• Benchmarking code generation models based on real-world mobile app issues.
• Evaluating model performance across different mobile frameworks (Android Native, React Native, Flutter).
• Training models for automated bug fixing in mobile application code.
• Studying the nature of common issues in production mobile app repositories.

Strengths

• Contains 407 verified tasks, providing a substantial test set.
• Sourced from 19 production mobile app repositories, indicating real-world relevance.
• Covers three major mobile development frameworks: Android Native (Java/Kotlin), React Native (TypeScript), and Flutter (Dart).

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Freshness should be verified as the last update was 2026-05-07.

Provenance

Source

MobileDev-Bench

Collection Method

Collected from 19 production mobile app repositories.

Freshness

Last updated 2026-05-07 05:01:36.