WMT DA Human Evaluation: Human Annotations for News Translation Tasks

RicardoRei's dataset aggregates all Direct Assessment human annotations from previous WMT News Translation shared tasks. The data is organized into 8 columns including language pair, source text, machine translation, reference translation, and human scores. It was last updated on the Hugging Face platform in February 2023.

Use Cases

• Training or benchmarking machine translation quality estimation models based on human-provided Direct Assessment scores.
• Analyzing translation quality across different language pairs and domains based on the 'lp' and 'domain' columns.
• Comparing machine translation outputs ('mt') against human reference translations ('ref') for error analysis.
• Studying the correlation between raw human judgments ('raw') and standardized scores ('score') for evaluation methodologies.

Strengths

• Data originates from the authoritative WMT (Workshop on Machine Translation) shared tasks, a standard benchmark in the field.
• Includes multiple years of annotations, suggesting longitudinal coverage for analysis.
• Contains structured human judgments with scores, reference translations, and source texts for direct comparison.

Limitations

• Row count, file formats, and license information are unknown, which may limit suitability assessment.
• Column-level documentation is absent; field semantics must be inferred from the description or original sources.
• Last updated 2023-02-17 10:41:18; freshness should be verified against the original WMT results pages.

Provenance

Source

WMT (Workshop on Machine Translation) News Translation shared tasks.

Collection Method

Aggregated from human Direct Assessment annotations collected for the annual WMT competitions.

Time Range

Covers multiple years from previous WMT tasks; specific range is not stated.

Freshness

Last updated 2023-02-17 10:41:18.

Geography

null