Imperial College London
Portrait Picture

Dr Ben Glocker

Faculty of Engineering, Department of Computing

Professor in Machine Learning for Imaging
 
 
 
//

Contact

 

+44 (0)20 7594 8334b.glocker Website CV

 
 
//

Location

 

377Huxley BuildingSouth Kensington Campus

//

Summary

 

Publications

Citation

BibTex format

@article{Mackay:2023:10.1016/j.clon.2023.01.016,
author = {Mackay, K and Bernstein, D and Glocker, B and Kamnitsas, K and Taylor, A},
doi = {10.1016/j.clon.2023.01.016},
journal = {Clinical Oncology},
pages = {354--369},
title = {A review of the metrics used to assess auto-contouring systems in radiotherapy},
url = {http://dx.doi.org/10.1016/j.clon.2023.01.016},
volume = {35},
year = {2023}
}
Download

RIS format (EndNote, RefMan)

TY  - JOUR
AB  - Auto-contouring could revolutionise future planning of radiotherapy treatment. The lack of consensus on how to assess and validate auto-contouring systems currently limits clinical use. This review formally quantifies the assessment metrics used in studies published during one calendar year and assesses the need for standardised practice. A PubMed literature search was undertaken for papers evaluating radiotherapy auto-contouring published during 2021. Papers were assessed for types of metric and the methodology used to generate ground-truth comparators. Our PubMed search identified 212 studies, of which 117 met the criteria for clinical review. Geometric assessment metrics were used in 116 of 117 studies (99.1%). This includes the Dice Similarity Coefficient used in 113 (96.6%) studies. Clinically relevant metrics, such as qualitative, dosimetric and time-saving metrics, were less frequently used in 22 (18.8%), 27 (23.1%) and 18 (15.4%) of 117 studies, respectively. There was heterogeneity within each category of metric. Over 90 different names for geometric measures were used. Methods for qualitative assessment were different in all but two papers. Variation existed in the methods used to generate radiotherapy plans for dosimetric assessment. Consideration of editing time was only given in 11 (9.4%) papers. A single manual contour as a ground-truth comparator was used in 65 (55.6%) studies. Only 31 (26.5%) studies compared auto-contours to usual inter- and/or intra-observer variation. In conclusion, significant variation exists in how research papers currently assess the accuracy of automatically generated contours. Geometric measures are the most popular, however their clinical utility is unknown. There is heterogeneity in the methods used to perform clinical assessment. Considering the different stages of system implementation may provide a framework to decide the most appropriate metrics. This analysis supports the need for a consensus on the clinical implement
AU  - Mackay,K
AU  - Bernstein,D
AU  - Glocker,B
AU  - Kamnitsas,K
AU  - Taylor,A
DO  - 10.1016/j.clon.2023.01.016
EP  - 369
PY  - 2023///
SN  - 0936-6555
SP  - 354
TI  - A review of the metrics used to assess auto-contouring systems in radiotherapy
T2  - Clinical Oncology
UR  - http://dx.doi.org/10.1016/j.clon.2023.01.016
UR  - https://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:001007726800001&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=a2bf6146997ec60c407a63945d4e92bb
UR  - https://doi.org/10.1016/j.clon.2023.01.016
UR  - http://hdl.handle.net/10044/1/107862
VL  - 35
ER  -
Download