XQ-MEval: A Cross-Lingual Benchmark for Translation Metric Bias

XQ-MEval is a quality-parallel benchmark dataset for evaluating automatic metrics on cross-lingual scoring bias. The dataset was created by the naist-nlp organization and released under a CC BY-SA 4.0 license. It was last updated on the Hugging Face platform in May 2026.

Use Cases

• Benchmarking automatic translation metrics for scoring bias based on the described quality-parallel structure.
• Analyzing how different types of MQM-defined errors affect metric scores across languages.
• Developing bias-mitigation techniques for cross-lingual evaluation metrics based on the injected-error methodology.

Strengths

• Dataset is explicitly designed as a benchmark for a specific research problem: cross-lingual scoring bias.
• Construction methodology is described: injecting Multidimensional Quality Metric (MQM)-defined errors into high-quality translations.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count and overall data scale are unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality and specific language coverage require manual inspection.

Provenance

Source

naist-nlp (likely Nara Institute of Science and Technology, Natural Language Processing lab)

Collection Method

Constructed by injecting varying numbers of MQM-defined errors into high-quality translations.

Freshness

Last updated 2026-05-08 12:31:26; freshness should be verified.