Gutenberg Text Corpus: Five Random Subsets for Algorithm Benchmarking

Five subsets of random text files from the Project Gutenberg database, with total sizes ranging from 184MB to 1.7GB. The dataset was curated to benchmark the performance of Symmetric Searchable Encryption schemes. Its unstructured text content also supports performance measurement for algorithms involving document parsing, keyword extraction, and dictionary creation.

Use Cases

• Benchmarking Symmetric Searchable Encryption schemes on unstructured text data.
• Measuring parsing algorithm performance across documents of varying sizes.
• Evaluating keyword extraction techniques on a corpus of e-book text.
• Testing dictionary creation algorithms on natural language text.

Strengths

• Provides five distinct subsets with defined total sizes (184MB, 357MB, 670MB, 1GB, 1.7GB), enabling scalability testing.
• Sourced from the well-known Project Gutenberg database, ensuring a foundation of public domain literary text.
• Cross-platform presence on PapersWithCode suggests recognized utility for computer science research.

Limitations

• Metadata is sparse: specific row counts, column names, and last update dates are unknown.
• The description is repetitive across sources, indicating a potential lack of detailed documentation.
• The 'random' selection method is not detailed, which may affect reproducibility.

Provenance

Source

Project Gutenberg database.

Collection Method

Random selection of text files (e-books in .txt format) from the source database.