AI Reviewer Diagnostic Data for Aspect-Guided Perturbation Analysis

Companion data for the CIKM 2025 paper 'Where Do LLMs Go Wrong? Diagnosing Automated Peer Review via Aspect-Guided Multi-Level Perturbation'. The dataset was created by researchers from PKU-ONELab and is hosted on HuggingFace. It was last updated on May 12, 2026.

Use Cases

• Diagnosing LLM failures in automated peer review based on the aspect-guided perturbation framework described.
• Benchmarking LLM performance on review generation tasks using the provided diagnostic perturbations.
• Training or fine-tuning models for more robust automated scientific feedback based on the identified error patterns.

Strengths

• Directly supports a peer-reviewed CIKM 2025 publication, indicating a research-grade standard.
• Associated with publicly available code and documentation for reproduction.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.

Provenance

Source

PKU-ONELab

Collection Method

Created as a companion resource for academic research on LLM evaluation.

Freshness

Last updated 2026-05-12 05:24:22; freshness should be verified.