Imperial College London
Portrait Picture

ProfessorPier LuigiDragotti

Faculty of Engineering, Department of Electrical and Electronic Engineering

Professor of Signal Processing
 
 
 
//

Contact

 

+44 (0)20 7594 6192p.dragotti

 
 
//

Location

 

814Electrical EngineeringSouth Kensington Campus

//

Summary

 

Publications

Citation

BibTex format

@article{Deng:2019:10.1109/TIP.2019.2944270,
author = {Deng, X and Dragotti, PL},
doi = {10.1109/TIP.2019.2944270},
journal = {IEEE Transactions on Image Processing},
pages = {1683--1698},
title = {Deep coupled ISTA network for multi-modal image super-resolution},
url = {http://dx.doi.org/10.1109/TIP.2019.2944270},
volume = {29},
year = {2019}
}
Download

RIS format (EndNote, RefMan)

TY  - JOUR
AB  - Given a low-resolution (LR) image, multi-modal image super-resolution (MISR) aims to find the high-resolution (HR) version of this image with the guidance of an HR image from another modality. In this paper, we use a model-based approach to design a new deep network architecture for MISR. We first introduce a novel joint multi-modal dictionary learning (JMDL) algorithm to model cross-modality dependency. In JMDL, we simultaneously learn three dictionaries and two transform matrices to combine the modalities. Then, by unfolding the iterative shrinkage and thresholding algorithm (ISTA), we turn the JMDL model into a deep neural network, called deep coupled ISTA network. Since the network initialization plays an important role in deep network training, we further propose a layer-wise optimization algorithm (LOA) to initialize the parameters of the network before running back-propagation strategy. Specifically, we model the network initialization as a multi-layer dictionary learning problem, and solve it through convex optimization. The proposed LOA is demonstrated to effectively decrease the training loss and increase the reconstruction accuracy. Finally, we compare our method with other state-of-the-art methods in the MISR task. The numerical results show that our method consistently outperforms others both quantitatively and qualitatively at different upscaling factors for various multi-modal scenarios.
AU  - Deng,X
AU  - Dragotti,PL
DO  - 10.1109/TIP.2019.2944270
EP  - 1698
PY  - 2019///
SN  - 1057-7149
SP  - 1683
TI  - Deep coupled ISTA network for multi-modal image super-resolution
T2  - IEEE Transactions on Image Processing
UR  - http://dx.doi.org/10.1109/TIP.2019.2944270
UR  - https://www.ncbi.nlm.nih.gov/pubmed/31603781
UR  - http://hdl.handle.net/10044/1/74161
VL  - 29
ER  -
Download