Broad Twitter Corpus with Named Entity Annotations

The Broad Twitter Corpus is a dataset of tweets collected over stratified times, places, and social uses, annotated for named entities. It was created by strombergnlp to represent a broad range of activities and language on the platform.

Use Cases

• Train a named entity recognition model using the provided entity annotations on tweets.
• Analyze linguistic patterns across different social uses and places represented in the stratified collection.
• Study the variation in named entity frequency and types across stratified times within the corpus.

Strengths

• Dataset is annotated for named entities, providing a supervision signal for NLP tasks.
• Collection strategy is stratified across times, places, and social uses to increase representativeness.
• Based on tags, the corpus size is indicated to be between 100K and 1M instances.

Limitations

• Specific row count, column details, and file formats are unknown, limiting precise technical assessment.
• Annotations are crowdsourced, which may introduce label noise or inconsistency.
• Geographic focus is indicated as US region, limiting generalizability to other linguistic or cultural contexts.

Provenance

Source

huggingface, originally from strombergnlp.

Collection Method

Tweets collected over stratified times, places, and social uses.

Freshness

Last updated in July 2022.

Geography

Primarily US region based on tags.