Post-editing effort and linguistically motivated evaluation of machine translation
- Autores: Sergi Alvarez Vidal
- Directores de la Tesis: Toni Badia Cardús (dir. tes.), Antonio Oliver González (codir. tes.)
- Lectura: En la Universitat Pompeu Fabra ( España ) en 2021
- Idioma: español
- Tribunal Calificador de la Tesis: Constantin Orasan (presid.), Pilar Sánchez-Gijón (secret.), Jesús Ángel Giménez Linares (voc.)
- Programa de doctorado: Programa de Doctorado en Traducción y Ciencias del Lenguaje por la Universidad Pompeu Fabra
- Materias:
- Enlaces
- Tesis en acceso abierto en: TDX
- Resumen
The recent improvements in neural MT (NMT) have driven a shift from statistical MT (SMT) to NMT, which has propelled the use of post-editing (PE) in translation workflows. However, many professional translators state that if the quality of the MT output is not good enough, they delete the remaining segments and translate everything from scratch. The problem is that usual automatic measurements do not always indicate the quality of the MT output, especially with high quality outputs, and there is still no clear correlation between PE effort and productivity scores.
We combine quantitative and qualitative methods to study some of the usual automatic metrics used to evaluate the quality of MT output, and compare them to measures of post-editing effort. Then, we study in detail different direct and indirect measures of effort in order to establish a correlation among them. We complement this study with the analysis of translators’ perceptions of the task.
Finally, we conduct a fine-grained analysis of MT errors based on postediting corrections and suggest an error-based approach to evaluate raw MT output which includes the use of challenge sets.
