Abstract
Language understanding often entails the translation of natural language utterances to a symbolic knowledge representation that is used by a machine in a subsequent task. This process is illustrated with the research of the EU FP7-FET MUSE project, where we have translated stories written in natural language into actions happening in a 3D virtual world. In this talk we elaborate on the challenges encountered during the MUSE project, the most important being the lack of sufficient training examples when learning the mapping of language to a knowledge representation that steers the graphical engine, and the lack of world knowledge to fully understand language and to truthfully render the actions told in the story. We focus on the role of deep learning approaches to alleviate these challenges. Deep learning, which is mostly accomplished by the use of deep neural networks, capture contextual knowledge, for instance, obtained from textual or multimodal sources. We show the limitations of current representations trained with deep learning for modelling the semantic relations needed in language understanding. We also show promising results of capturing world knowledge from multimodal (textual and visual) data as is done in the EU CHIST-ERA MUSTER project on MUltimodal processing of Spatial and TEmporal expRessions.
Bio
Prof. Marie-Francine Moens heads the Language Intelligence and Information Retrieval research group at the Department of Computer Science of KU Leuven, Belgium. Her research expertise includes: content recognition in text, information extraction, discourse understanding, text mining, knowledge acquisition, machine reading of text, processing of noisy text such as user-generated content and speech transcripts, information retrieval, search models, and machine learning for natural language processing. She is author of more than 300 international peer-reviewed publications and of several books. She is involved in the organization or program committee (as program chair, area chair or reviewer) of major conferences on computational linguistics, information retrieval and machine learning. In 2011 and 2012 she was appointed as chair of the European Chapter of the Association for Computational Linguistics (EACL) and was a member of the executive board of the Association for Computational Linguistics (ACL). From 2010 until 2014 she was a member of the Research Council of KU Leuven and is currently a member of the Council of the Industrial Research Fund of KU Leuven. She is the scientific manager of the EU COST action iV&L (The European Network on Integrating Vision and Language). She was appointed as Scottish Informatics and Computer Science Alliance (SICSA) Distinguished Visiting Fellow in 2014.