Prerequisite Link Validity Benchmark with Expert Ratings

A benchmark of 300 title pairs for validating prerequisite knowledge links, with each pair receiving two independent expert ratings. It accompanies a research paper on crowdsourcing prerequisite knowledge graphs at scale.

Use Cases

• Validate prerequisite link predictions from models using the 300 expert-rated title pairs.
• Analyze inter-rater agreement between the two independent expert ratings for each title pair.
• Benchmark automated knowledge graph construction methods against the 300 validated prerequisite links.

Strengths

• Contains 300 title pairs, providing a defined benchmark size.
• Each title pair has two independent expert ratings, enabling reliability assessment.
• Associated with a peer-reviewed research paper from MIT.

Limitations

• Limited to 300 data points, which is a small sample for some machine learning tasks.
• No column or feature details are provided, limiting understanding of data structure.
• The scope is restricted to prerequisite link validation, limiting broader applicability.

Provenance

Source

MIT Center for Collective Intelligence, Massachusetts Institute of Technology.

Collection Method

Domain-stratified benchmark creation with expert ratings.

Freshness

Last updated on 2026-02-17.