Imperial College London

DrKrystianMikolajczyk

Faculty of EngineeringDepartment of Electrical and Electronic Engineering

Reader in Computer Vision
 
 
 
//

Contact

 

+44 (0)20 7594 6220k.mikolajczyk

 
 
//

Location

 

Electrical EngineeringSouth Kensington Campus

//

Summary

 

Publications

Citation

BibTex format

@article{Ramisa:2017:10.1109/TPAMI.2017.2721945,
author = {Ramisa, A and Yan, F and Moreno-Noguer, F and Mikolajczyk, K},
doi = {10.1109/TPAMI.2017.2721945},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
pages = {1072--1085},
title = {BreakingNews: article annotation by image and text processing},
url = {http://dx.doi.org/10.1109/TPAMI.2017.2721945},
volume = {40},
year = {2017}
}

RIS format (EndNote, RefMan)

TY  - JOUR
AB - Current approaches lying in the intersection of computer vision and NLP have achieved unprecedented breakthroughs in tasks like automatic captioning or image retrieval. Most of these methods, though, rely on training sets of images associated with annotations that specifically describe the visual content. This paper proposes going a step further and explores more complex cases where textual descriptions are loosely related to images. We focus on the particular domain of News. We introduce new deep learning methods that address source and popularity prediction, article illustration, and article geolocation. An adaptive CNN is proposed, that shares most of the structure for all tasks, and is suitable for multitask and transfer learning. Deep CCA is deployed for article illustration, and a new loss function based on Great Circle Distance is proposed for geolocation. Furthermore, we present BreakingNews, a novel dataset with approximately 100K news articles including images, text, captions, and enriched with heterogeneous meta-data. BreakingNews allows exploring all aforementioned problems, for which we provide baseline performances using various CNN architectures, and different representations of the textual and visual features. We report promising results and bring to light several limitations of current state-of-the-art, which we hope will help spur progress in the field.
AU - Ramisa,A
AU - Yan,F
AU - Moreno-Noguer,F
AU - Mikolajczyk,K
DO - 10.1109/TPAMI.2017.2721945
EP - 1085
PY - 2017///
SN - 0162-8828
SP - 1072
TI - BreakingNews: article annotation by image and text processing
T2 - IEEE Transactions on Pattern Analysis and Machine Intelligence
UR - http://dx.doi.org/10.1109/TPAMI.2017.2721945
UR - http://hdl.handle.net/10044/1/52699
VL - 40
ER -