MMC: Multimodal Chart Instruction Tuning for NAACL 2024

MMC is a multimodal instruction-tuning dataset for chart understanding published by Fuxiao Liu for NAACL 2024. It provides visual chart data paired with natural language instructions to improve the reasoning capabilities of large language models across various chart types, including stock market visualizations.

Use Cases

• Instruction tuning of MLLMs using chart-based question-answer pairs to improve visual reasoning
• Benchmarking model accuracy on stock market chart analysis using the stock-tagged data subsets
• Evaluating zero-shot performance of models like LLaVA or MiniGPT-4 on diverse chart formats

Strengths

• Peer-reviewed research foundation from NAACL 2024
• Includes specialized stock market chart categories
• Supports multiple MLLM architectures including LLaVA and MiniGPT-4

Limitations

• Total record count is not specified in the repository metadata
• Potential domain bias toward financial and stock-related charts based on available tags

Provenance

Source

Fuxiao Liu, NAACL 2024

Collection Method

Annotated for instruction tuning

Freshness

Last updated January 2025; based on 2024 NAACL publication.