BioMysteryBench: 99 Bioinformatics Research Problems for AI Evaluation

BioMysteryBench (full set) is a benchmark of 99 bioinformatics research problems created by Anthropic. The dataset contains one row per problem, including a question prompt, an answer rubric, and metadata on human solvability. It was last updated on 2026-04-29.

Use Cases

• Benchmarking AI model performance on bioinformatics reasoning based on the 99 described problems.
• Evaluating model adherence to grading criteria based on the provided answer rubric for each problem.
• Assessing the difficulty of problems for AI versus humans based on the human_solvable flag.
• Training or fine-tuning language models on structured scientific question-answer pairs.

Strengths

• Contains 99 distinct bioinformatics problems, providing a substantial test suite.
• Includes an answer rubric for each problem, which likely provides clear grading criteria.
• Flags problems as human_solvable or not, offering a difficulty reference.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

Anthropic

Collection Method

Created as a research benchmark; specific gathering method is not detailed.

Freshness

Last updated 2026-04-29 04:13:36; freshness should be verified.