Authors
Nicola Bertoldi, Fondazione Bruno Kessler
Mauro Cettolo, Fondazione Bruno Kessler
Marcello Federico, Fondazione Bruno Kessler
Christian Buck, The University of Edinburgh
Abstract
The new frontier of computer assisted translation technology is the effective integration of statistical MT within the translation workflow. In this respect, the SMT ability of incrementally learning from the translations produced by users plays a central role.
A still open problem is the evaluation of SMT systems that evolve over time. In this paper, the authors propose a new metric for assessing the quality of an adaptive MT component that is derived from the theory of learning curves: the percentage slope.
For assessing its effectiveness, it was developed a simple but effective adapting SMT system suitable to work in the context of a CAT tool supported by MT. The authors have compared several ways to plot the change in error rate over time for different systems and identified the most suitable for computing the percentage slope. Finally, it was shown that the percentage slope well exposes the paradigmatic behaviors of evolving SMT systems.
