234,000 declassified U.S. documents have been processed with optical character recognition (OCR). The collection includes 31 million extracted named entities and 1536-dimensional embeddings for each document. The dataset is hosted on Kaggle.
Use Cases
- Train or evaluate named entity recognition models based on the 31 million extracted entities.
- Perform semantic search or document clustering based on the 1536-dimensional embeddings.
- Analyze historical trends and topics within declassified government documents based on the OCR text.
Strengths
- 234,000 documents provide a substantial text corpus.
- Includes 31 million pre-extracted named entities.
- Documents are processed with OCR and have 1536-dimensional embeddings.
Limitations
- Description metadata is limited; actual data quality requires manual inspection after download.
- Column-level documentation is absent; field semantics must be inferred after download.
- Last update date is unknown; freshness unverified.
Provenance
- Source
- Declassified U.S. documents.
- Collection Method
- Documents processed with optical character recognition (OCR), entity extraction, and embedding generation.
- Time Range
- null
- Freshness
- Last updated date is unknown.
- Geography
- United States