Fine-Grained Role-Playing and Instruction-Following Benchmark

RoleMRC is a composite benchmark for evaluating large language models in role-playing and instruction-following scenarios. It focuses on maintaining role identity and ability limits while following diverse instructions.

Use Cases

• Evaluate model performance on fine-grained role-playing tasks using the benchmark's defined scenarios.
• Assess instruction-following capabilities of LLMs within specific role constraints.
• Analyze model adherence to role identity and pre-defined ability limits across diverse prompts.

Strengths

• Designed specifically for fine-grained evaluation of role-playing and instruction-following.
• Focuses on maintaining role identity and pre-defined ability limits, a noted gap in existing datasets.

Limitations

• Dataset size, column structure, and sample data are unavailable for assessment.
• Specific task formats, evaluation metrics, and data composition are not detailed in the provided input.

Provenance

Source

Hugging Face dataset authored by Junrulu.

Freshness

Last updated on 2025-03-20.