Multimodal Chemistry Reaction Question Answering Benchmark

RxnBench-Doc is a document-level question answering benchmark containing 540 multiple-select questions. It assesses PhD-level understanding of organic chemistry reactions using textual and multimodal contexts, with all questions reviewed by experts for scientific rigor.

Use Cases

• Benchmark multimodal models on 540 chemistry DocQA questions requiring understanding of text and image contexts.
• Evaluate PhD-level organic chemistry knowledge using expert-reviewed multiple-select questions.
• Train models for visual question answering on scientific documents combining text and image modalities.

Strengths

• Contains 540 expert-reviewed multiple-select questions for robust evaluation.
• Designed for PhD-level assessment of organic chemistry reaction understanding.
• Features multimodal contexts combining text and image data.

Limitations

• The dataset size of 540 questions may be limited for training large models from scratch.
• Focus is exclusively on organic chemistry, limiting generalizability to other scientific domains.
• Specific details on data sources, class balance, or annotation quality are not provided.

Provenance

Source

UniParser

Collection Method

Benchmark questions underwent multiple rounds of expert review for clarity and scientific rigor.

Freshness

Last updated on March 5, 2026.