Imperial College London
Portrait Picture

ProfessorGaryFrost

Faculty of Medicine, Department of Metabolism, Digestion and Reproduction

Chair in Nutrition & Dietetics
 
 
 
//

Contact

 

+44 (0)20 7594 0959g.frost Website

 
 
//

Location

 

Commonwealth BiuldingHammersmith HospitalHammersmith Campus

//

Summary

 

Publications

Citation

BibTex format

@article{Qiu:2023:10.1109/TCYB.2023.3243999,
author = {Qiu, J and Lo, FP-W and Gu, X and Jobarteh, ML and Jia, W and Baranowski, T and Steiner-Asiedu, M and Anderson, AK and Mccrory, MA and Sazonov, E and Sun, M and Frost, G and Lo, B},
doi = {10.1109/TCYB.2023.3243999},
journal = {IEEE Transactions on Cybernetics},
pages = {1--14},
title = {Egocentric image captioning for privacy-preserved passive dietary intake monitoring},
url = {http://dx.doi.org/10.1109/TCYB.2023.3243999},
volume = {PP},
year = {2023}
}
Download

RIS format (EndNote, RefMan)

TY  - JOUR
AB  - Camera-based passive dietary intake monitoring is able to continuously capture the eating episodes of a subject, recording rich visual information, such as the type and volume of food being consumed, as well as the eating behaviors of the subject. However, there currently is no method that is able to incorporate these visual clues and provide a comprehensive context of dietary intake from passive recording (e.g., is the subject sharing food with others, what food the subject is eating, and how much food is left in the bowl). On the other hand, privacy is a major concern while egocentric wearable cameras are used for capturing. In this article, we propose a privacy-preserved secure solution (i.e., egocentric image captioning) for dietary assessment with passive monitoring, which unifies food recognition, volume estimation, and scene understanding. By converting images into rich text descriptions, nutritionists can assess individual dietary intake based on the captions instead of the original images, reducing the risk of privacy leakage from images. To this end, an egocentric dietary image captioning dataset has been built, which consists of in-the-wild images captured by head-worn and chest-worn cameras in field studies in Ghana. A novel transformer-based architecture is designed to caption egocentric dietary images. Comprehensive experiments have been conducted to evaluate the effectiveness and to justify the design of the proposed architecture for egocentric dietary image captioning. To the best of our knowledge, this is the first work that applies image captioning for dietary intake assessment in real-life settings.
AU  - Qiu,J
AU  - Lo,FP-W
AU  - Gu,X
AU  - Jobarteh,ML
AU  - Jia,W
AU  - Baranowski,T
AU  - Steiner-Asiedu,M
AU  - Anderson,AK
AU  - Mccrory,MA
AU  - Sazonov,E
AU  - Sun,M
AU  - Frost,G
AU  - Lo,B
DO  - 10.1109/TCYB.2023.3243999
EP  - 14
PY  - 2023///
SN  - 1083-4419
SP  - 1
TI  - Egocentric image captioning for privacy-preserved passive dietary intake monitoring
T2  - IEEE Transactions on Cybernetics
UR  - http://dx.doi.org/10.1109/TCYB.2023.3243999
UR  - https://www.ncbi.nlm.nih.gov/pubmed/37028043
UR  - https://ieeexplore.ieee.org/document/10059226
UR  - http://hdl.handle.net/10044/1/103639
VL  - PP
ER  -
Download