Reddit Dirty Writing Prompts Cleaned and Tokenized for LLM Training

SicariusSicariiStuff provides a cleaned and reformatted version of the Reddit Dirty Writing Prompts dataset. The dataset has been processed into a ShareGPT format and includes token counts for both LLAMA1 and LLAMA3 tokenizers, as well as character counts. The dataset was last updated on July 25, 2024.

Use Cases

• Fine-tuning language models for creative writing based on Reddit-sourced prompts.
• Benchmarking tokenizer performance using the included LLAMA1 and LLAMA3 token counts.
• Analyzing the length and structure of user-generated writing prompts based on character and token metrics.

Strengths

• Dataset has undergone additional meticulous cleaning beyond the original source.
• Each entry includes token counts for two specific LLM tokenizers (LLAMA1 and LLAMA3) and character counts, which aids in data preparation.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• The original dataset's license and specific content boundaries are not detailed in this description.

Provenance

Source

Original dataset from huggingface.co/datasets/nothingiisreal/Reddit-Dirty-And-WritingPrompts.

Collection Method

Further cleaned, sorted, and reformatted into ShareGPT format by SicariusSicariiStuff.

Freshness

Last updated 2024-07-25 14:30:24; freshness should be verified.