Imperial College London
Portrait Picture

Professor Lucia Specia

Faculty of Engineering, Department of Computing

Chair in Natural Language Processing
 
 
 
//

Contact

 

l.specia Website

 
 
//

Location

 

572aHuxley BuildingSouth Kensington Campus

//

Summary

 

Publications

Citation

BibTex format

@article{Caglayan:2020:10.1162/tacl_a_00330,
author = {Caglayan, O and Ive, J and Haralampieva, V and Madhyastha, P and Barrault, L and Specia, L},
doi = {10.1162/tacl_a_00330},
journal = {Transactions of the Association for Computational Linguistics},
pages = {539--555},
title = {Simultaneous machine translation with visual context},
url = {http://dx.doi.org/10.1162/tacl_a_00330},
volume = {8},
year = {2020}
}
Download

RIS format (EndNote, RefMan)

TY  - JOUR
AB  - Simultaneous machine translation (SiMT) aims to translate a continuous inputtext stream into another language with the lowest latency and highest qualitypossible. The translation thus has to start with an incomplete source text,which is read progressively, creating the need for anticipation. In this paper,we seek to understand whether the addition of visual information can compensatefor the missing source context. To this end, we analyse the impact of differentmultimodal approaches and visual features on state-of-the-art SiMT frameworks.Our results show that visual context is helpful and that visually-groundedmodels based on explicit object region information are much better thancommonly used global features, reaching up to 3 BLEU points improvement underlow latency scenarios. Our qualitative analysis illustrates cases where onlythe multimodal systems are able to translate correctly from English intogender-marked languages, as well as deal with differences in word order, suchas adjective-noun placement between English and French.
AU  - Caglayan,O
AU  - Ive,J
AU  - Haralampieva,V
AU  - Madhyastha,P
AU  - Barrault,L
AU  - Specia,L
DO  - 10.1162/tacl_a_00330
EP  - 555
PY  - 2020///
SN  - 2307-387X
SP  - 539
TI  - Simultaneous machine translation with visual context
T2  - Transactions of the Association for Computational Linguistics
UR  - http://dx.doi.org/10.1162/tacl_a_00330
UR  - http://arxiv.org/abs/2009.07310v3
UR  - https://www.mitpressjournals.org/doi/full/10.1162/tacl_a_00330
UR  - http://hdl.handle.net/10044/1/83292
VL  - 8
ER  -
Download