ProteinGym is the standard benchmark suite for evaluating protein fitness and mutation effect predictors, developed by the OATML / Marks lab and published at NeurIPS 2023 Datasets and Benchmarks. It provides a common, standardized task surface for comparing protein language models, inverse folding models, and supervised fitness predictors. The benchmark covers both zero-shot and supervised settings.
Use Cases
- Benchmarking protein language models based on standardized fitness prediction tasks.
- Evaluating inverse folding models on a common task surface.
- Comparing supervised fitness predictors in both zero-shot and supervised settings.
- Assessing model performance on mutation effect prediction.
Strengths
- Serves as the standard benchmark suite for the field, published at NeurIPS 2023.
- Designed for direct comparison of diverse model types on a common task surface.
- Covers both zero-shot and supervised evaluation settings.
Limitations
- Column-level documentation is absent; field semantics must be inferred after download.
- Row count is unknown, which may limit suitability assessment.
- Description metadata is limited; actual data quality requires manual inspection after download.
Provenance
- Source
- OATML / Marks lab, published at NeurIPS 2023 Datasets and Benchmarks.
- Freshness
- Last updated 2026-05-27 12:38:46; freshness should be verified.