Publications
202 results found
Pawlowski N, Ktena SI, Lee MCH, et al., 2017, DLTK: State of the Art Reference Implementations for Deep Learning on Medical Images
We present DLTK, a toolkit providing baseline implementations for efficientexperimentation with deep learning methods on biomedical images. It builds ontop of TensorFlow and its high modularity and easy-to-use examples allow for alow-threshold access to state-of-the-art implementations for typical medicalimaging problems. A comparison of DLTK's reference implementations of popularnetwork architectures for image segmentation demonstrates new top performanceon the publicly available challenge data "Multi-Atlas Labeling Beyond theCranial Vault". The average test Dice similarity coefficient of $81.5$ exceedsthe previously best performing CNN ($75.7$) and the accuracy of the challengewinning method ($79.0$).
Baumgartner C, Kamnitsas K, Matthew J, et al., 2017, SonoNet: real-time detection and localisation of fetal standard scan planes in freehand ultrasound, IEEE Transactions on Medical Imaging, Vol: 36, Pages: 2204-2215, ISSN: 1558-254X
Identifying and interpreting fetal standard scan planes during 2D ultrasound mid-pregnancy examinations are highly complex tasks which require years of training. Apart from guiding the probe to the correct location, it can be equally difficult for a non-expert to identify relevant structures within the image. Automatic image processing can provide tools to help experienced as well as inexperienced operators with these tasks. In this paper, we propose a novel method based on convolutional neural networks which can automatically detect 13 fetal standard views in freehand 2D ultrasound data as well as provide a localisation of the fetal structures via a bounding box. An important contribution is that the network learns to localise the target anatomy using weak supervision based on image-level labels only. The network architecture is designed to operate in real-time while providing optimal output for the localisation task. We present results for real-time annotation, retrospective frame retrieval from saved videos, and localisation on a very large and challenging dataset consisting of images and video recordings of full clinical anomaly screenings. We found that the proposed method achieved an average F1-score of 0.798 in a realistic classification experiment modelling real-time detection, and obtained a 90.09% accuracy for retrospective frame retrieval. Moreover, an accuracy of 77.8% was achieved on the localisation task.
McDonagh S, Hou B, Kamnitsas K, et al., 2017, Context-sensitive super-resolution for fast fetal magnetic resonance imaging, Fifth International Workshop, CMMI 2017, Second International Workshop, RAMBO 2017, and First International Workshop, SWITCH 2017, Publisher: Springer Verlag, ISSN: 0302-9743
3D Magnetic Resonance Imaging (MRI) is often a trade-off between fast but low-resolution image acquisition and highly detailed but slow image acquisition. Fast imaging is required for targets that move to avoid motion artefacts. This is in particular difficult for fetal MRI. Spatially independent upsampling techniques, which are the state-of-the-art to address this problem, are error prone and disregard contextual information. In this paper we propose a context-sensitive upsampling method based on a residual convolutional neural network model that learns organ specific appearance and adopts semantically to input data allowing for the generation of high resolution images with sharp edges and fine scale detail. By making contextual decisions about appearance and shape, present in different parts of an image, we gain a maximum of structural detail at a similar contrast as provided by high-resolution data. We experiment on 145 fetal scans and show that our approach yields an increased PSNR of 1.25 dB when applied to under-sampled fetal data cf. baseline upsampling. Furthermore, our method yields an increased PSNR of 1.73 dB when utilizing under-sampled fetal data to perform brain volume reconstruction on motion corrupted captured data.
Cerrolaza JJ, Oktay O, Gomez A, et al., 2017, Fetal skull segmentation in 3D ultrasound via structured geodesic random forest, Fetal, Infant and Ophthalmic Medical Image Analysis: International Workshop, FIFI 2017, and 4th International Workshop, OMIA 2017, Held in Conjunction with MICCAI 2017, Pages: 25-32, ISSN: 0302-9743
© Springer International Publishing AG 2017. Ultrasound is the primary imaging method for prenatal screening and diagnosis of fetal anomalies. Thanks to its non-invasive and non-ionizing properties, ultrasound allows quick, safe and detailed evaluation of the unborn baby, including the estimation of the gestational age, brain and cranium development. However, the accuracy of traditional 2D fetal biometrics is dependent on operator expertise and subjectivity in 2D plane finding and manual marking. 3D ultrasound has the potential to reduce the operator dependence. In this paper, we propose a new random forest-based segmentation framework for fetal 3D ultrasound volumes, able to efficiently integrate semantic and structural information in the classification process. We introduce a new semantic features space able to encode spatial context via generalized geodesic distance transform. Unlike alternative auto-context approaches, this new set of features is efficiently integrated into the same forest using contextual trees. Finally, we use a new structured labels space as alternative to the traditional atomic class labels, able to capture morphological variability of the target organ. Here, we show the potential of this new general framework segmenting the skull in 3D fetal ultrasound volumes, significantly outperforming alternative random forest-based approaches.
Hou B, Alansary A, McDonagh S, et al., 2017, Predicting slice-to-volume transformation in presence of arbitrary subject motion, 20th International Conference on Medical Image Computing and Computer Assisted Intervention 2017, Publisher: Springer, Pages: 296-304, ISSN: 0302-9743
This paper aims to solve a fundamental problem in intensity-based 2D/3D registration, which concerns the limited capture range andneed for very good initialization of state-of-the-art image registrationmethods. We propose a regression approach that learns to predict rota-tion and translations of arbitrary 2D image slices from 3D volumes, withrespect to a learned canonical atlas co-ordinate system. To this end,we utilize Convolutional Neural Networks (CNNs) to learn the highlycomplex regression function that maps 2D image slices into their cor-rect position and orientation in 3D space. Our approach is attractivein challenging imaging scenarios, where significant subject motion com-plicates reconstruction performance of 3D volumes from 2D slice data.We extensively evaluate the effectiveness of our approach quantitativelyon simulated MRI brain data with extreme random motion. We furtherdemonstrate qualitative results on fetal MRI where our method is in-tegrated into a full reconstruction and motion compensation pipeline.With our CNN regression approach we obtain an average prediction er-ror of 7mm on simulated data, and convincing reconstruction quality ofimages of very young fetuses where previous methods fail. We furtherdiscuss applications to Computed Tomography (CT) and X-Ray pro-jections. Our approach is a general solution to the 2D/3D initializationproblem. It is computationally efficient, with prediction times per sliceof a few milliseconds, making it suitable for real-time scenarios.
Alansary A, Rajchl M, McDonagh S, et al., 2017, PVR: Patch-to-Volume Reconstruction for Large Area Motion Correction of Fetal MRI, IEEE Transactions on Medical Imaging, Vol: 36, Pages: 2031-2044, ISSN: 1558-254X
In this paper we present a novel method for the correction of motion artifacts that are present in fetal Magnetic Resonance Imaging (MRI) scans of the whole uterus. Contrary to current slice-to-volume registration (SVR) methods, requiring an inflexible anatomical enclosure of a \emph{single} investigated organ, the proposed patch-to-volume reconstruction (PVR) approach is able to reconstruct a large field of view of non-rigidly deforming structures. It relaxes rigid motion assumptions by introducing a specific amount of redundant information that is exploited with parallelized patch-wise optimization, super-resolution, and automatic outlier rejection. We further describe and provide an efficient parallel implementation of PVR allowing its execution within reasonable time on commercially available graphics processing units (GPU), enabling its use in the clinical practice. We evaluate PVR's computational overhead compared to standard methods and observe improved reconstruction accuracy in the presence of affine motion artifacts compared to conventional SVR in synthetic experiments. Furthermore, we have evaluated our method qualitatively and quantitatively on real fetal MRI data subject to maternal breathing and sudden fetal movements. We evaluate peak-signal-to-noise ratio (PSNR), structural similarity index (SSIM), and cross correlation (CC) with respect to the originally acquired data and provide a method for visual inspection of reconstruction uncertainty. We further evaluate the distance error for selected anatomical landmarks in the fetal head, as well as calculating the mean and maximum displacements resulting from automatic non-rigid registration to a motion-free ground truth image. These experiments demonstrate a successful application of PVR motion compensation to the whole fetal body, uterus and placenta.
Miao H, Mistelbauer G, Karimov A, et al., 2017, Placenta maps: in utero placental health assessment of the human fetus, IEEE Transactions on Visualization and Computer Graphics, Vol: 23, Pages: 1612-1623, ISSN: 1077-2626
The human placenta is essential for the supply of the fetus. To monitor the fetal development, imaging data is acquired usingultrasound (US). Although it is currently the gold-standard in fetal imaging, it might not capture certain abnormalities of the placenta.Magnetic resonance imaging (MRI) is a safe alternative for the in utero examination while acquiring the fetus data in higher detail.Nevertheless, there is currently no established procedure for assessing the condition of the placenta and consequently the fetal health.Due to maternal respiration and inherent movements of the fetus during examination, a quantitative assessment of the placenta requiresfetal motion compensation, precise placenta segmentation and a standardized visualization, which are challenging tasks. Utilizingadvanced motion compensation and automatic segmentation methods to extract the highly versatile shape of the placenta, we introducea novel visualization technique that presents the fetal and maternal side of the placenta in a standardized way. Our approach enablesphysicians to explore the placenta even in utero. This establishes the basis for a comparative assessment of multiple placentas to analyzepossible pathologic arrangements and to support the research and understanding of this vital organ. Additionally, we propose athree-dimensional structure-aware surface slicing technique in order to explore relevant regions inside the placenta. Finally, to survey theapplicability of our approach, we consulted clinical experts in prenatal diagnostics and i
Toisoul A, Rueckert D, Kainz B, 2017, Accessible GLSL Shader programming, EuroGraphics 2017, Publisher: Eurographics Association, ISSN: 1017-4656
Teaching fundamental principles of Computer Graphics requires a thoroughly prepared lecture alongside practical training.Modern graphics programming rarely provides a straightforward application programming interface (API) and the availableAPIs pose high entry barriers to students. Shader-based programming of standard graphics pipelines is often inaccessiblethrough complex setup procedures and convoluted programming environments. In this paper we discuss an undergraduateentry level lecture with its according lab exercises. We present a programming framework that makes interactive graphicsprogramming accessible while allowing to design individual tasks as instructive exercises to solidify the content of individuallecture units. The discussed teaching framework provides a well defined programmable graphics pipeline with geometry shadingstages and image-based post processing functionality based on framebuffer objects. It is open-source and available online.
Miao H, Mistelbauer G, Kainz B, 2017, Placenta Maps: In Utero Placental Health Assessment of the Human Fetus, IEEE Pacific Visualization Symposium, ISSN: 2165-8765
Kainz B, 2017, Reconstruction, Segmentation, and Analysis of Medical Images:First International Workshops, RAMBO 2016 and HVSMR 2016, Held in Conjunction with MICCAI 2016, Athens, Greece, October 17, 2016, Revised Selected Papers, Publisher: Springer, ISBN: 978-3-319-52279-1
McDonagh SG, Hou B, Alansary A, et al., 2017, Context-Sensitive Super-Resolution for Fast Fetal Magnetic Resonance Imaging., Publisher: Springer, Pages: 116-126
Rajchl M, Lee MCH, Oktay O, et al., 2016, DeepCut: object segmentation from bounding box annotations using convolutional neural networks, IEEE Transactions on Medical Imaging, Vol: 36, Pages: 674-683, ISSN: 0278-0062
In this paper, we propose DeepCut, a method to obtain pixelwise object segmentations given an image dataset labelled weak annotations, in our case bounding boxes. It extends the approach of the well-known GrabCut[1] method to include machine learning by training a neural network classifier from bounding box annotations. We formulate the problem as an energy minimisation problem over a densely-connected conditional random field and iteratively update the training targets to obtain pixelwise object segmentations. Additionally, we propose variants of the DeepCut method and compare those to a naìˆve approach to CNN training under weak supervision. We test its applicability to solve brain and lung segmentation problems on a challenging fetal magnetic resonance dataset and obtain encouraging results in terms of accuracy.
Kainz B, Alansary A, McDonagh ST, et al., 2016, Fast motion compensation and super-resolution from multiple stacks of 2D slices
This tool implements a novel method for the correction of motion artifacts as acquired in fetal Magnetic Resonance Imaging (MRI) scans of the whole uterus. Contrary to current slice-to-volume registration (SVR) methods, requiring an inflexible enclosure of a single investigated organ, the proposed patch-to-volume reconstruction (PVR) approach is able to reconstruct a large field of view of non-rigidly deforming structures. It relaxes rigid motion assumptions by introducing a defined amount of redundant information that is addressed with parallelized patch-wise optimization and automatic outlier rejection. We further describe and provide an efficient parallel implementation of PVR allowing its execution within reasonable time on commercially available graphics processing units (GPU), enabling its use in the clinical practice. We evaluate PVR’s computational overhead compared to standard methods and observe improved reconstruction accuracy in presence of affine motion artifacts of approximately 30% compared to conventional SVR in synthetic experiments.Furthermore, we have verified our method qualitatively and quantitatively on real fetal MRI data subject to maternal breathing and sudden fetal movements. We evaluate peak-signal-to-noise ratio (PSNR), structural similarity index (SSIM), and cross correlation (CC) with respect to the originally acquired data and provide a method for visual inspection of reconstruction uncertainty. With these experiments we demonstrate successful application of PVR motion compensation to the whole uterus, the human fetus, and the human placenta.
Kainz B, Toisoul A, 2016, ShaderLab Framework
ShaderLab is a teaching tool to solidify the fundamentals of Computer Graphics. The ShaderLab framework is based on Qt5, CMake, OpenGL 4.0, and GLSL and allows the student to modify GLSL shaders in an IDE-like environment. The framework is able to render shaded polyhedral geometry (.off/.obj), supports image-based post-processing, and allows to implement simple ray-tracing algorithms. This tool will be intensively tested by 140 CO317 Computer Graphics students in Spring 2017.
Steinnberger M, Kenzel M, Kainz B, 2016, ScatterAlloc
ScatterAlloc is a dynamic memory allocator for the GPU. It is designed concerning the requirements of massively parallel execution. ScatterAlloc greatly reduces collisions and congestion by scattering memory requests based on hashing. It can deal with thousands of GPU-threads concurrently allocating memory and its execution time is almost independent of the thread count. ScatterAlloc is open source and easy to use in your CUDA projects.
Alansary A, Kamnitsas K, Davidson A, et al., 2016, Fast Fully Automatic Segmentation of the Human Placenta from Motion Corrupted MRI, 19th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2016), Publisher: Springer Verlag, ISSN: 0302-9743
Baumgartner CH, Kamnitsas K, Matthew J, et al., 2016, Real-time standard scan plane detection and localisation in fetal ultrasound using fully convolutional neural networks, International Conference on Medical Image Computing and Computer-Assisted Intervention MICCAI 206, Publisher: Springer, Pages: 203-211
Fetal mid-pregnancy scans are typically carried out accordingto fixed protocols. Accurate detection of abnormalities and correctbiometric measurements hinge on the correct acquisition of clearlydefined standard scan planes. Locating these standard planes requires ahigh level of expertise. However, there is a worldwide shortage of expertsonographers. In this paper, we consider a fully automated system basedon convolutional neural networks which can detect twelve standard scanplanes as defined by the UK fetal abnormality screening programme. Thenetwork design allows real-time inference and can be naturally extendedto provide an approximate localisation of the fetal anatomy in the image.Such a framework can be used to automate or assist with scan planeselection, or for the retrospective retrieval of scan planes from recordedvideos. The method is evaluated on a large database of 1003 volunteermid-pregnancy scans. We show that standard planes acquired in a clinicalscenario are robustly detected with a precision and recall of 69 %and 80 %, which is superior to the current state-of-the-art. Furthermore,we show that it can retrospectively retrieve correct scan planes with anaccuracy of 71 % for cardiac views and 81 % for non-cardiac views.
Rueckert D, Glocker B, Kainz B, 2016, Learning clinically useful information from images: Past, present and future, Medical Image Analysis, Vol: 33, Pages: 13-18, ISSN: 1361-8423
Over the last decade, research in medical imaging has made significantprogress in addressing challenging tasks such as image registration and imagesegmentation. In particular, the use of model-based approaches has been keyin numerous, successful advances in methodology. The advantage of modelbasedapproaches is that they allow the incorporation of prior knowledgeacting as a regularisation that favours plausible solutions over implausibleones. More recently, medical imaging has moved away from hand-crafted, andoften explicitly designed models towards data-driven, implicit models thatare constructed using machine learning techniques. This has led to majorimprovements in all stages of the medical imaging pipeline, from acquisitionand reconstruction to analysis and interpretation. As more and more imagingdata is becoming available, e.g., from large population studies, this trend islikely to continue and accelerate. At the same time new developments inmachine learning, e.g., deep learning, as well as significant improvementsin computing power, e.g., parallelisation on graphics hardware, offer newpotential for data-driven, semantic and intelligent medical imaging. Thisarticle outlines the work of the BioMedIA group in this area and highlightssome of the challenges and opportunities for future work.
Rajchl M, Lee M, Schrans F, et al., 2016, Learning under Distributed Weak Supervision
The availability of training data for supervision is a frequently encountered bottleneck of medical image analysis methods. While typically established by a clinical expert rater, the increase in acquired imaging data renders traditional pixel-wise segmentations less feasible. In this paper, we examine the use of a crowdsourcing platform for the distribution of super-pixel weak annotation tasks and collect such annotations from a crowd of non-expert raters. The crowd annotations are subsequently used for training a fully convolutional neural network to address the problem of fetal brain segmentation in T2-weighted MR images. Using this approach we report encouraging results compared to highly targeted, fully supervised methods and potentially address a frequent problem impeding image analysis research.
Rajchl M, Lee M, Oktay O, et al., 2016, DeepCut: object segmentation from bounding box annotations using convolutional neural networks, Publisher: arXiv
In this paper, we propose DeepCut, a method to obtain pixelwise object segmentations given an image dataset labelled with bounding box annotations. It extends the approach of the well-known GrabCut method to include machine learning by training a neural network classifier from bounding box annotations. We formulate the problem as an energy minimisation problem over a densely-connected conditional random field and iteratively update the training targets to obtain pixelwise object segmentations. Additionally, we propose variants of the DeepCut method and compare those to a naive approach to CNN training under weak supervision. We test its applicability to solve brain and lung segmentation problems on a challenging fetal magnetic resonance dataset and obtain encouraging results in terms of accuracy.
Kainz B, Lloyd D, Alansary A, et al., 2016, High-Performance Motion Correction of Fetal MRI, EuroRVVV -- EuroVis Workshop on Reproducibility, Verification, and Validation in Visualization, Publisher: Eurographics Digital Library
Fetal Magnetic Resonance Imaging (MRI) shows promising results for pre-natal diagnostics. The detection of potentially lifethreateningabnormalities in the fetus can be difficult with ultrasound alone. MRI is one of the few safe alternative imagingmodalities in pregnancy. However, to date it has been limited by unpredictable fetal and maternal motion during acquisition.Motion between the acquisitions of individual slices of a 3D volume results in spatial inconsistencies that can be resolved byslice-to-volume reconstruction (SVR) methods to provide high quality 3D image data. Existing algorithms to solve this problemhave evolved from very slow implementations targeting a single organ to general high-performance solutions to reconstruct thewhole uterus. In this paper we give a brief overview over the current state-of-the art in fetal motion compensation methods andshow currently emerging clinical applications of these techniques
Lloyd D, Kainz B, van Amerom J, et al., 2016, Three-Dimensional Modelling of the Fetal Vasculature from Prenatal MRI using Motion-Corrected Slice-to-Volume Registration, International Society for Magnetic Resonance in Medicine Annual Meeting 2016
Lloyd D, Kainz B, van Amerom J, et al., 2016, Prenatal MRI visualisation of the aortic arch and fetal vasculature using motion-corrected slice-to-volume reconstruction
Alansary A, Kainz B, Rajchl M, et al., 2016, PVR: Patch-to-Volume Reconstruction for Large Area Motion Correction of Fetal MRI., CoRR, Vol: abs/1611.07289
Baumgartner CF, Kamnitsas K, Matthew J, et al., 2016, Real-Time Standard Scan Plane Detection and Localisation in Fetal Ultrasound Using Fully Convolutional Neural Networks, Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II, Editors: Ourselin, Joskowicz, Sabuncu, Unal, Wells, Publisher: Springer International Publishing, Pages: 203-211, ISBN: 978-3-319-46723-8
Alansary A, Lee M, Kainz B, et al., 2015, Automatic Brain Localisation in Foetal MRI using Superpixel Graphs, ICML Workshop on Machine Learning meets Medical Imaging, Publisher: Springer, ISSN: 0302-9743
Egger J, Voglreiter P, Dokter M, et al., 2015, In-depth Multicenter Workflow Analysis of Liver Tumor Ablations for the Development of a Novel Computer-aided Software Tool, 101th Annual Meeting of The Radiological Society of North America (RSNA)
Kainz B, Alansary A, Malamateniou CH, et al., 2015, Flexible reconstruction and correction of unpredictable motion from stacks of 2D images, 18th International Conference, Munich, Germany, October 5-9, 2015, Publisher: Springer International Publishing, Pages: 555-562, ISSN: 0302-9743
We present a method to correct motion in fetal in-utero scan sequences. The proposed approach avoids previously necessary manual segmentation of a region of interest. We solve the problem of non-rigid motion by splitting motion corrupted slices into overlapping patches of finite size. In these patches the assumption of rigid motion approximately holds and they can thus be used to perform a slice-to-volume-based (SVR) reconstruction during which their consistency with the other patches is learned. The learned information is used to reject patches that are not conform with the motion corrected reconstruction in their local areas. We evaluate rectangular and evenly distributed patches for the reconstruction as well as patches that have been derived from super-pixels. Both approaches achieve on 29 subjects aged between 22–37 weeks a sufficient reconstruction quality and facilitate following 3D segmentation of fetal organs and the placenta.
Egger J, Busse H, Brandmaier P, et al., 2015, Interactive volumetry of liver ablation zones, Scientific Reports, Vol: 5, ISSN: 2045-2322
Percutaneous radiofrequency ablation (RFA) is a minimally invasive technique that destroys cancer cells by heat. The heat results from focusing energy in the radiofrequency spectrum through a needle. Amongst others, this can enable the treatment of patients who are not eligible for an open surgery. However, the possibility of recurrent liver cancer due to incomplete ablation of the tumor makes post-interventional monitoring via regular follow-up scans mandatory. These scans have to be carefully inspected for any conspicuousness. Within this study, the RF ablation zones from twelve post-interventional CT acquisitions have been segmented semi-automatically to support the visual inspection. An interactive, graph-based contouring approach, which prefers spherically shaped regions, has been applied. For the quantitative and qualitative analysis of the algorithm’s results, manual slice-by-slice segmentations produced by clinical experts have been used as the gold standard (which have also been compared among each other). As evaluation metric for the statistical validation, the Dice Similarity Coefficient (DSC) has been calculated. The results show that the proposed tool provides lesion segmentation with sufficient accuracy much faster than manual segmentation. The visual feedback and interactivity make the proposed tool well suitable for the clinical workflow.
This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.