USPTO Patent Documents from The Pile

Subset of the ROOTS corpus, containing English-language patent documents from the USPTO as part of The Pile collection. It comprises 0.5358% of the total ROOTS corpus and 2.9032% of its English-language content. The data has been processed by BigScience with filters for deduplication and document size.

Use Cases

• Train language models on specialized technical text from patent documents.
• Analyze the linguistic and structural patterns of USPTO patent filings.
• Benchmark text deduplication or document filtering algorithms on a processed corpus.

Strengths

• Contains a defined subset of The Pile, representing 0.5358% of the total ROOTS corpus.
• Comprises 2.9032% of the English-language ('en') portion of the ROOTS corpus.
• Processed with specific filters: dedup_document, filter_remove_empty_docs, and filter_small_docs_bytes_1024.

Limitations

• The exact number of documents, rows, or file size is unknown.
• Specific column structure and sample data are unavailable for inspection.
• Data is from a single region (US) and language (English), limiting geographic and linguistic diversity.

Provenance

Source

The Pile (USPTO subset), processed by BigScience.

Collection Method

Part of the ROOTS corpus; processed with deduplication and size filtering.

Freshness

Last updated on 2022-12-12.

Geography

United States (US), based on tags.