Performance Metrics for SpliceAI Benchmark Across Six Datasets

Nathan Fortier provides performance metrics comparing the original SpliceAI tool with two open-source implementations and a legacy ensemble baseline. The dataset includes results from six evaluation sets, including a curated set of 1,316 validated variants and ClinVar-derived datasets comprising over 111,000 variants. It was last updated on May 13, 2026.

Use Cases

• Benchmarking splice prediction algorithms based on performance across canonical and deep intronic variants.
• Calibrating score thresholds for clinical variant interpretation based on results from deep intronic benchmarks.
• Comparing concordance between open-source and original implementations based on correlation analysis across event types.

Strengths

• Evaluates performance across six distinct datasets, including a set of 1,316 validated variants and ClinVar-derived sets with over 111,000 variants.
• Reports specific balanced accuracy scores (e.g., 0.940 for SpliceAI on deep intronic variants) and splice-site match rates exceeding 90%.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• The dataset is very small (5.5 KB), indicating limited scope, likely containing summary metrics rather than raw prediction data.

Provenance

Source

Nathan Fortier

Collection Method

Likely contains aggregated performance metrics from comparative benchmarking studies.

Freshness

Last updated 2026-05-13 17:43:04; freshness should be verified.