HiT: Datasets for Training and Evaluating Hierarchy Transformer Encoders

A collection of datasets for training and evaluating Hierarchy Transformer encoders (HiTs) from the University of Oxford. The datasets support multi-hop inference and mixed-hop prediction tasks, including transfer evaluation across schema.org, FoodOn, and Disease Ontology (DOID) hierarchies. The data is available via Huggingface Datasets and a Zenodo release, with the work published in Advances in Neural Information Processing Systems in 2024.

Use Cases

• Training hierarchy-aware language models based on entity triplets or labelled entity pairs.
• Evaluating multi-hop inference capabilities based on the 'multi' suffix datasets.
• Assessing mixed-hop prediction and transfer learning performance based on the 'mixed' suffix datasets.
• Studying knowledge transfer across ontologies based on the schema.org, FoodOn, and DOID hierarchies.

Strengths

• Datasets are directly linked to a peer-reviewed NeurIPS 2024 publication.
• Offers a convenient Huggingface Datasets entry for direct loading.
• Includes specific datasets for transfer evaluation across three distinct ontologies.

Limitations

• Row count and dataset size are unknown, which may limit suitability assessment.
• Column-level documentation is absent; field semantics must be inferred after download.
• Original entity IDs are not retained in the Huggingface format, requiring a separate mapping file.

Provenance

Source

University of Oxford research project.

Collection Method

Created for the paper "Language Models as Hierarchy Encoders."