News Articles for Machine Unlearning Benchmark Evaluation

MUSE-News is a benchmark dataset for evaluating machine unlearning models, focusing on news articles. It assesses six key properties of unlearned models, including memorization, privacy leakage, and utility preservation. The dataset was created by muse-bench and last updated in July 2024.

Use Cases

• Evaluate a model's verbatim memorization property using news article text sequences.
• Assess privacy leakage risks by testing unlearning on specific news article content.
• Benchmark a model's utility preservation on retained data after removing targeted news articles.
• Test the scalability of an unlearning algorithm with respect to the size of removal requests on news data.
• Evaluate the sustainability of an unlearning method over sequential removal requests using time-series news data.

Strengths

• Benchmark designed to assess six specific properties of machine unlearning models.
• Focuses on textual news articles, a common data type requiring unlearning.
• Dataset was updated in July 2024, indicating recent maintenance.

Limitations

• Specific dataset size, row count, and column structure are unknown.
• The scope is limited to English news articles, which may not generalize to other languages or data types.
• As a specialized benchmark, its utility is primarily for evaluating unlearning rather than general NLP tasks.

Provenance

Source

huggingface.co/datasets/muse-bench/MUSE-News

Freshness

Last updated on 2024-07-10.