Softcite Dataset: 4,971 Annotated Scientific Articles for Software Mention Recognition

4,971 full-text scientific articles in English, half in Life Sciences and half in Economics, containing annotations for software mentions. The dataset is a gold-standard corpus created through multi-stage annotation and reconciliation by a team, resulting in over 5,000 annotated software names. It was developed by James Howison at The University of Texas at Austin, with versions released in 2020 and 2023.

Use Cases

• Train named entity recognition models based on annotated software names, versions, and publishers.
• Benchmark information extraction systems using the gold-standard holdout set mentioned in the description.
• Study software citation practices in scientific literature based on the annotated mentions across Life Sciences and Economics.

Strengths

• 4,971 full-text articles provide a substantial corpus of around 46 million tokens.
• Gold-standard annotations resulted from a multi-stage process with reconciliation to resolve disagreements.
• Includes a dedicated holdout set of 20% of the full texts for evaluation purposes.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Last update date is unknown; freshness unverified.

Provenance

Source

The University of Texas at Austin

Collection Method

Manual and automatic screening of Open Access scientific articles, followed by a double annotation process with reconciliation.