Imperial College London
Portrait Picture

ProfessorPier LuigiDragotti

Faculty of Engineering, Department of Electrical and Electronic Engineering

Professor of Signal Processing
 
 
 
//

Contact

 

+44 (0)20 7594 6192p.dragotti

 
 
//

Location

 

814Electrical EngineeringSouth Kensington Campus

//

Summary

 

Publications

Citation

BibTex format

@article{Deng:2020:10.1109/TPAMI.2020.2984244,
author = {Deng, X and Dragotti, PL},
doi = {10.1109/TPAMI.2020.2984244},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
pages = {3333--3348},
title = {Deep convolutional neural network for multi-modal image restoration and fusion},
url = {http://dx.doi.org/10.1109/TPAMI.2020.2984244},
volume = {43},
year = {2020}
}
Download

RIS format (EndNote, RefMan)

TY  - JOUR
AB  - In this paper, we propose a novel deep convolutional neural network to solve the general multi-modal image restoration (MIR) and multi-modal image fusion (MIF) problems. Different from other methods based on deep learning, our network architecture is designed by drawing inspirations from a new proposed multi-modal convolutional sparse coding (MCSC) model. The key feature of the proposed network is that it can automatically split the common information shared among different modalities, from the unique information that belongs to each single modality, and is therefore denoted with CU-Net, i.e., Common and Unique information splitting network. Specifically, the CU-Net is composed of three modules, i.e., the unique feature extraction module (UFEM), common feature preservation module (CFPM), and image reconstruction module (IRM). The architecture of each module is derived from the corresponding part in the MCSC model, which consists of several learned convolutional sparse coding (LCSC) blocks. Extensive numerical results verify the effectiveness of our method on a variety of MIR and MIF tasks, including RGB guided depth image super-resolution, flash guided non-flash image denoising, multi-focus and multi-exposure image fusion.
AU  - Deng,X
AU  - Dragotti,PL
DO  - 10.1109/TPAMI.2020.2984244
EP  - 3348
PY  - 2020///
SN  - 0162-8828
SP  - 3333
TI  - Deep convolutional neural network for multi-modal image restoration and fusion
T2  - IEEE Transactions on Pattern Analysis and Machine Intelligence
UR  - http://dx.doi.org/10.1109/TPAMI.2020.2984244
UR  - https://www.ncbi.nlm.nih.gov/pubmed/32248098
UR  - https://ieeexplore.ieee.org/document/9055063
UR  - http://hdl.handle.net/10044/1/77889
VL  - 43
ER  -
Download