Publications
291 results found
Yunxiao R, Keshavarz M, Salzitsa A, et al., 2022, Machine Learning-Based Real-Time Localisation and Automatic Trapping of Multiple Microrobots in Optical Tweezer, International Conference on Manipulation, Automation and Robotics at Small Scales (MARSS2022)
Zhang D, Wu Z, Chen J, et al., 2022, Human-robot shared control for surgical robot based on context-aware sim-to-real adaptation, 2022 IEEE International Conference on Robotics and Automation (ICRA), Publisher: IEEE, Pages: 7701-7707
Human-robot shared control, which integrates the advantages of both humans and robots, is an effective approach to facilitate efficient surgical operation. Learning from demonstration (LfD) techniques can be used to automate some of the surgical sub tasks for the construction of the shared control mechanism. However, a sufficient amount of data is required for the robot to learn the manoeuvres. Using a surgical simulator to collect data is a less resource-demanding approach. With sim-to-real adaptation, the manoeuvres learned from a simulator can be transferred to a physical robot. To this end, we propose a sim-to-real adaptation method to construct a human-robot shared control framework for robotic surgery. In this paper, a desired trajectory is generated from a simulator using LfD method, while dynamic motion primitives (DMP) is used to transfer the desired trajectory from the simulator to the physical robotic platform. Moreover, a role adaptation mechanism is developed such that the robot can adjust its role according to the surgical operation contexts predicted by a neural network model. The effectiveness of the proposed framework is validated on the da Vinci Research Kit (dVRK). Results of the user studies indicated that with the adaptive human-robot shared control framework, the path length of the remote controller, the total clutching number and the task completion time can be reduced significantly. The proposed method outperformed the traditional manual control via teleoperation.
Gil B, Anastasova S, Lo B, 2022, Graphene field-effect transistors array for detection of liquid conductivities in the physiological range through novel time- multiplexed impedance measurements, CARBON, Vol: 193, Pages: 394-403, ISSN: 0008-6223
- Author Web Link
- Cite
- Citations: 2
Gil B, Lo B, Yang G-Z, et al., 2022, Smart implanted access port catheter for therapy intervention with pH and lactate biosensors., Materials Today Bio, Vol: 15, Pages: 1-9, ISSN: 2590-0064
Totally implanted access ports (TIAP) are widely used with oncology patients requiring long term central venous access for the delivery of chemotherapeutic agents, infusions, transfusions, blood sample collection and parenteral nutrition. Such devices offer a significant improvement to the quality of life for patients and reduced complication rates, particularly infection, in contrast to the classical central venous catheters. Nevertheless, infections do occur, with biofilm formation bringing difficulties to the treatment of infection-related complications that can ultimately lead to the explantation of the device. A smart TIAP device that is sensor-enabled to detect infection prior to extensive biofilm formation would reduce the cases for potential device explantation, whereas biomarkers detection within body fluids such as pH or lactate would provide vital information regarding metabolic processes occurring inside the body. In this paper, we propose a novel batteryless and wireless device suitable for the interrogation of such markers in an embodiment model of an TIAP, with miniature biochemical sensing needles. Device readings can be carried out by a smartphone equipped with Near Field Communication (NFC) interface at relative short distances off-body, while providing radiofrequency energy harvesting capability to the TIAP, useful for assessing patient's health and potential port infection on demand.
Bai W, Cursi F, Guo X, et al., 2022, Task-Based LSTM Kinematic Modeling for a Tendon-Driven Flexible Surgical Robot, IEEE TRANSACTIONS ON MEDICAL ROBOTICS AND BIONICS, Vol: 4, Pages: 339-342
- Author Web Link
- Cite
- Citations: 4
Zhang D, Barbot A, Seichepine F, et al., 2022, Micro-object pose estimation with sim-to-real transfer learning using small dataset, Communications Physics, Vol: 5, ISSN: 2399-3650
Lam K, Chen J, Wang Z, et al., 2022, Machine learning for technical skill assessment in surgery: a systematic review, npj Digital Medicine, Vol: 5, ISSN: 2398-6352
Accurate and objective performance assessment is essential for both trainees and certified surgeons. However, existing methods can be time consuming, labor intensive and subject to bias. Machine learning (ML) has the potential to provide rapid, automated and reproducible feedback without the need for expert reviewers. We aimed to systematically review the literature and determine the ML techniques used for technical surgical skill assessment and identify challenges and barriers in the field. A systematic literature search, in accordance with the PRISMA statement, was performed to identify studies detailing the use of ML for technical skill assessment in surgery. Of the 1896 studies that were retrieved, 66 studies were included. The most common ML methods used were Hidden Markov Models (HMM, 14/66), Support Vector Machines (SVM, 17/66) and Artificial Neural Networks (ANN, 17/66). 40/66 studies used kinematic data, 19/66 used video or image data, and 7/66 used both. Studies assessed performance of benchtop tasks (48/66), simulator tasks (10/66), and real-life surgery (8/66). Accuracy rates of over 80% were achieved, although tasks and participants varied between studies. Barriers to progress in the field included a focus on basic tasks, lack of standardization between studies, and lack of datasets. ML has the potential to produce accurate and objective surgical skill assessment through the use of methods including HMM, SVM, and ANN. Future ML-based assessment tools should move beyond the assessment ofbasic tasks and towards real-life surgery and provide interpretable feedback with clinical value for the surgeon.
Gu X, Guo Y, Yang G-Z, et al., 2022, Cross-domain self-supervised complete geometric representation learning for real-scanned point cloud based pathological gait analysis, IEEE Journal of Biomedical and Health Informatics, Vol: 26, Pages: 1034-1044, ISSN: 2168-2194
Accurate lower-limb pose estimation is a prereq-uisite of skeleton based pathological gait analysis. To achievethis goal in free-living environments for long-term monitoring,single depth sensor has been proposed in research. However,the depth map acquired from a single viewpoint encodes onlypartial geometric information of the lower limbs and exhibitslarge variations across different viewpoints. Existing off-the-shelfthree-dimensional (3D) pose tracking algorithms and publicdatasets for depth based human pose estimation are mainlytargeted at activity recognition applications. They are relativelyinsensitive to skeleton estimation accuracy, especially at thefoot segments. Furthermore, acquiring ground truth skeletondata for detailed biomechanics analysis also requires consid-erable efforts. To address these issues, we propose a novelcross-domain self-supervised complete geometric representationlearning framework, with knowledge transfer from the unlabelledsynthetic point clouds of full lower-limb surfaces. The proposedmethod can significantly reduce the number of ground truthskeletons (with only 1%) in the training phase, meanwhileensuring accurate and precise pose estimation and capturingdiscriminative features across different pathological gait patternscompared to other methods.
Jia W, Ren Y, Li B, et al., 2022, A Novel Approach to Dining Bowl Reconstruction for Image-Based Food Volume Estimation, SENSORS, Vol: 22
- Author Web Link
- Cite
- Citations: 2
Li W, Zhang D, Yang G-Z, et al., 2022, Design and Modelling of A Spring-Like Continuum Joint with Variable Pitch for Endoluminal Surgery, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Publisher: IEEE, Pages: 41-47, ISSN: 2153-0858
Rosa BMG, Lo B, Yeatman E, 2022, Prototype smartwatch device for prolonged physiological monitoring in remote environments, IEEE-EMBS International Conference on Wearable and Implantable Body Sensor Networks (BSN) / 18th IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), Publisher: IEEE, ISSN: 2376-8886
Yang Y, Bai W, Lo B, 2022, A Customized Artificial Ear Based on Vibrotactile Feedback: A Pilot Study, IEEE-EMBS International Conference on Wearable and Implantable Body Sensor Networks (BSN) / 18th IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), Publisher: IEEE, ISSN: 2376-8886
Zhou X, Bai W, Ren Y, et al., 2022, An LSTM-based Bilateral Active Estimation Model for Robotic Teleoperation with Varying Time Delay, 7th IEEE International Conference on Advanced Robotics and Mechatronics, Publisher: IEEE, Pages: 725-730
- Author Web Link
- Cite
- Citations: 2
Akbarzadeh S, Gu X, Wu Z, et al., 2022, A Novel Active Human Echolocation Device, IEEE-EMBS International Conference on Wearable and Implantable Body Sensor Networks (BSN) / 18th IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), Publisher: IEEE, ISSN: 2376-8886
Peng J, Shi P, Qiu J, et al., 2022, Clustering Egocentric Images in Passive Dietary Monitoring with Self-Supervised Learning, 4th IEEE-EMBS International Conference on Wearable and Implantable Body Sensor Networks (BSN) / 18th IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), Publisher: IEEE
Chen J, Wang Z, Zhu R, et al., 2022, Path Generation with Reinforcement Learning for Surgical Robot Control, 4th IEEE-EMBS International Conference on Wearable and Implantable Body Sensor Networks (BSN) / 18th IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), Publisher: IEEE
Qiu J, Lo FP-W, Gu X, et al., 2021, Indoor future person localization from an egocentric wearable camera, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Publisher: IEEE, Pages: 8586-8592
Accurate prediction of future person location and movement trajectory from an egocentric wearable camera can benefit a wide range of applications, such as assisting visually impaired people in navigation, and the development of mobility assistance for people with disability. In this work, a new egocentric dataset was constructed using a wearable camera, with 8,250 short clips of a targeted person either walking 1) toward, 2) away, or 3) across the camera wearer in indoor environments, or 4) staying still in the scene, and 13,817 person bounding boxes were manually labelled. Apart from the bounding boxes, the dataset also contains the estimated pose of the targeted person as well as the IMU signal of the wearable camera at each time point. An LSTM-based encoder-decoder framework was designed to predict the future location and movement trajectory of the targeted person in this egocentric setting. Extensive experiments have been conducted on the new dataset, and have shown that the proposed method is able to reliably and better predict future person location and trajectory in egocentric videos captured by the wearable camera compared to three baselines.
Zhang D, Wang R, Lo B, 2021, Surgical gesture recognition based on bidirectional multi-layer independently RNN with explainable spatial feature extraction, IEEE International Conference on Robotics and Automation (ICRA) 2021, Publisher: IEEE, Pages: 1350-1356
Minimally invasive surgery mainly consists of a series of sub-tasks, which can be decomposed into basic gestures or contexts. As a prerequisite of autonomic operation, surgical gesture recognition can assist motion planning and decision-making, and build up context-aware knowledge to improve the surgical robot control quality. In this work, we aim to develop an effective surgical gesture recognition approach with an explainable feature extraction process. A Bidirectional Multi-Layer independently RNN (BML-indRNN) model is proposed in this paper, while spatial feature extraction is implemented via fine-tuning of a Deep Convolutional Neural Network (DCNN) model constructed based on the VGG architecture. To eliminate the black-box effects of DCNN, Gradient-weighted Class Activation Mapping (Grad-CAM) is employed. It can provide explainable results by showing the regions of the surgical images that have a strong relationship with the surgical gesture classification results. The proposed method was evaluated based on the suturing task with data obtained from the public available JIGSAWS database. Comparative studies were conducted to verify the pro-posed framework. Results indicated that the testing accuracy for the suturing task based on our proposed method is 87.13%,which outperforms most of the state-of-the-art algorithms
Han J, Gu X, Lo B, 2021, Semi-supervised contrastive learning for generalizable motor imagery eeg classification, 17th IEEE International Conference on Wearable and Implantable Body Sensor Networks, Publisher: IEEE
Electroencephalography (EEG) is one of the most widely used brain-activity recording methods in non-invasive brain-machine interfaces (BCIs). However, EEG data is highly nonlinear, and its datasets often suffer from issues such as data heterogeneity, label uncertainty and data/label scarcity. To address these, we propose a domain independent, end-to-end semi-supervised learning framework with contrastive learning and adversarial training strategies. Our method was evaluated in experiments with different amounts of labels and an ablation study in a motor imagery EEG dataset. The experiments demonstrate that the proposed framework with two different backbone deep neural networks show improved performance over their supervised counterparts under the same condition.
Yang X, Zhang Y, Lo B, et al., 2021, DBAN: Adversarial Network With Multi-Scale Features for Cardiac MRI Segmentation, IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, Vol: 25, Pages: 2018-2028, ISSN: 2168-2194
- Author Web Link
- Cite
- Citations: 15
Jiang S, Kang P, Song X, et al., 2021, Emerging wearable interfaces and algorithms for hand gesture recognition: a survey., IEEE Reviews in Biomedical Engineering, Vol: PP, ISSN: 1941-1189
Hands are vital in a wide range of fundamental daily activities, and neurological diseases that impede hand function can significantly affect quality of life. Wearable hand gesture interfaces hold promise to restore and assist hand function and to enhance human-human and human-computer communication. The purpose of this review is to synthesize current novel sensing interfaces and algorithms for hand gesture recognition, and the scope of applications covers rehabilitation, prosthesis control, sign language recognition, and human-computer interaction. Results showed that electrical, dynamic, acoustical/vibratory, and optical sensing were the primary input modalities in gesture recognition interfaces. Two categories of algorithms were identified: 1) classification algorithms for predefined, fixed hand poses and 2) regression algorithms for continuous finger and wrist joint angles. Conventional machine learning algorithms, including linear discriminant analysis, support vector machines, random forests, and non-negative matrix factorization, have been widely used for a variety of gesture recognition applications, and deep learning algorithms have more recently been applied to further facilitate the complex relationship between sensor signals and multi-articulated hand postures. Future research should focus on increasing recognition accuracy with larger hand gesture datasets, improving reliability and robustness for daily use outside of the laboratory, and developing softer, less obtrusive interfaces.
Qiu J, Lo FP-W, Jiang S, et al., 2021, Counting bites and recognizing consumed food from videos for passive dietary monitoring., IEEE Journal of Biomedical and Health Informatics, Vol: 25, Pages: 1471-1482, ISSN: 2168-2194
Assessing dietary intake in epidemiological studies are predominantly based on self-reports, which are subjective, inefficient, and also prone to error. Technological approaches are therefore emerging to provide objective dietary assessments. Using only egocentric dietary intake videos, this work aims to provide accurate estimation on individual dietary intake through recognizing consumed food items and counting the number of bites taken. This is different from previous studies that rely on inertial sensing to count bites, and also previous studies that only recognize visible food items but not consumed ones. As a subject may not consume all food items visible in a meal, recognizing those consumed food items is more valuable. A new dataset that has 1,022 dietary intake video clips was constructed to validate our concept of bite counting and consumed food item recognition from egocentric videos. 12 subjects participated and 52 meals were captured. A total of 66 unique food items, including food ingredients and drinks, were labelled in the dataset along with a total of 2,039 labelled bites. Deep neural networks were used to perform bite counting and food item recognition in an end-to-end manner. Experiments have shown that counting bites directly from video clips can reach 74.15% top-1 accuracy (classifying between 0-4 bites in 20-second clips), and a MSE value of 0.312 (when using regression). Our experiments on video-based food recognition also show that recognizing consumed food items is indeed harder than recognizing visible ones, with a drop of 25% in F1 score.
Chen G, Jia W, Zhao Y, et al., 2021, Food/non-food classification of real-life egocentric images in low- and middle-income countries based on image tagging features, Frontiers in Artificial Intelligence, Vol: 4, ISSN: 2624-8212
Malnutrition, including both undernutrition and obesity, is a significant problem in low- and middle-income countries (LMICs). In order to study malnutrition and develop effective intervention strategies, it is crucial to evaluate nutritional status in LMICs at the individual, household, and community levels. In a multinational research project supported by the Bill & Melinda Gates Foundation, we have been using a wearable technology to conduct objective dietary assessment in sub-Saharan Africa. Our assessment includes multiple diet-related activities in urban and rural families, including food sources (e.g., shopping, harvesting, and gathering), preservation/storage, preparation, cooking, and consumption (e.g., portion size and nutrition analysis). Our wearable device ("eButton" worn on the chest) acquires real-life images automatically during wake hours at preset time intervals. The recorded images, in amounts of tens of thousands per day, are post-processed to obtain the information of interest. Although we expect future Artificial Intelligence (AI) technology to extract the information automatically, at present we utilize AI to separate the acquired images into two binary classes: images with (Class 1) and without (Class 0) edible items. As a result, researchers need only to study Class-1 images, reducing their workload significantly. In this paper, we present a composite machine learning method to perform this classification, meeting the specific challenges of high complexity and diversity in the real-world LMIC data. Our method consists of a deep neural network (DNN) and a shallow learning network (SLN) connected by a novel probabilistic network interface layer. After presenting the details of our method, an image dataset acquired from Ghana is utilized to train and evaluate the machine learning system. Our comparative experiment indicates that the new composite method performs better than the conventional deep learning method assessed by integra
Zhang C, Liu S, Han F, et al., 2021, Hybrid manifold-deep convolutional neural network for sleep staging, Methods, Pages: 1-9, ISSN: 1046-2023
Analysis of electroencephalogram (EEG) is a crucial diagnostic criterion for many sleep disorders, of which sleep staging is an important component. Manual stage classification is a labor-intensive process and usually suffered from many subjective factors. Recently, more and more computer-aided techniques have been applied to this task, among which deep convolutional neural network has been performing well as an effective automatic classification model. Despite some comprehensive models have been developed to improve classification results, the accuracy for clinical applications has not been reached due to the lack of sufficient labeled data and the limitation of extracting latent discriminative EEG features. Therefore, we propose a novel hybrid manifold-deep convolutional neural network with hyperbolic attention. To overcome the shortage of labeled data, we update the semi-supervised training scheme as an optimal solution. In order to extract the latent feature representation, we introduce the manifold learning module and the hyperbolic module to extract more discriminative information. Eight subjects from the public dataset are utilized to evaluate our pipeline, and the model achieved 89% accuracy, 70% precision, 80% sensitivity, 72% f1-score and kappa coefficient of 78%, respectively. The proposed model demonstrates powerful ability in extracting feature representation and achieves promising results by using semi-supervised training scheme. Therefore, our approach shows strong potential for future clinical development.
Lei J, Qiu J, Lo FP-W, et al., 2021, Assessing individual dietary intake in food sharing scenarios with food and human pose detection, 6th International Workshop on Multimedia Assisted Dietary Management (MADiMa 2020), Publisher: Springer International Publishing, Pages: 549-557, ISSN: 0302-9743
Food sharing and communal eating are very common in some countries. To assess individual dietary intake in food sharing scenarios, this work proposes a vision-based approach to first capturing the food sharing scenario with a 360-degree camera, and then using a neural network to infer different eating states of each individual based on their body pose and relative positions to the dishes. The number of bites each individual has taken of each dish is then deduced by analyzing the inferred eating states. A new dataset with 14 panoramic food sharing videos was constructed to validate our approach. The results show that our approach is able to reliably predict different eating states as well as individual’s bite count with respect to each dish in food sharing scenarios.
Li W, Tsai Y-Y, Yang G-Z, et al., 2021, A novel endoscope design using spiral technique for robotic-assisted endoscopy insertion, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Publisher: IEEE, Pages: 3119-3124
Gastrointestinal (GI) endoscopy is a conventional and prevalent procedure used to diagnose and treat diseases in the digestive tract. This procedure requires inserting an endoscope equipped with a camera and instruments inside a patient to the target of interest. To manoeuvre the endoscope, an endoscopist would rotate the knob at the handle to change the direction of the distal tip and apply the feeding force to advance the endoscope. However, due to the nature of the design, this often causes a looping problem during insertion making it difficult to be further advanced to the deeper section of the tract such as the transverse and ascending colon. To this end, in this paper, we propose a novel robotic endoscope which is covered by a rotating screw-like sheath and uses a spiral insertion technique to generate 'pull' forces at the distal tip of the endoscope to facilitate insertion. The whole shaft of the endoscope can be actively rotated, providing the crawling ability from the attached spiral sheath. With the redundant control on a spring-like continuum joint, the bending tip is capable of maintaining its orientation to assist endoscope navigation. To test its functions and feasibility to address the looping problem, three experiments were carried out. The first two experiments were to analyse the kinematic of the device and test the ability of the device to hold its distal tip at different orientation angles during spiral insertion. In the third experiment, we inserted the device in the bent colon phantom to evaluate the effectiveness of the proposed design against looping when advancing through a curved section of a colon. Results show the moving ability using spiral technique and verify its potential of clinical application.
Gu X, Guo Y, Deligianni F, et al., 2021, Cross-subject and cross-modal transfer for generalized abnormal gait pattern recognition, IEEE Transactions on Neural Networks and Learning Systems, Vol: 32, Pages: 546-560, ISSN: 1045-9227
For abnormal gait recognition, pattern-specific features indicating abnormalities are interleaved with the subject-specific differences representing biometric traits. Deep representations are, therefore, prone to overfitting, and the models derived cannot generalize well to new subjects. Furthermore, there is limited availability of abnormal gait data obtained from precise Motion Capture (Mocap) systems because of regulatory issues and slow adaptation of new technologies in health care. On the other hand, data captured from markerless vision sensors or wearable sensors can be obtained in home environments, but noises from such devices may prevent the effective extraction of relevant features. To address these challenges, we propose a cascade of deep architectures that can encode cross-modal and cross-subject transfer for abnormal gait recognition. Cross-modal transfer maps noisy data obtained from RGBD and wearable sensors to accurate 4-D representations of the lower limb and joints obtained from the Mocap system. Subsequently, cross-subject transfer allows disentangling subject-specific from abnormal pattern-specific gait features based on a multiencoder autoencoder architecture. To validate the proposed methodology, we obtained multimodal gait data based on a multicamera motion capture system along with synchronized recordings of electromyography (EMG) data and 4-D skeleton data extracted from a single RGBD camera. Classification accuracy was improved significantly in both Mocap and noisy modalities.
Li W, Shen M, Gao A, et al., 2021, Towards a Snake-Like Flexible Robot for Endoscopic Submucosal Dissection, IEEE TRANSACTIONS ON MEDICAL ROBOTICS AND BIONICS, Vol: 3, Pages: 257-260
- Author Web Link
- Cite
- Citations: 8
Wang R, Zhang D, Li Q, et al., 2021, Real-time Surgical Environment Enhancement for Robot-Assisted Minimally Invasive Surgery Based on Super-Resolution, IEEE International Conference on Robotics and Automation (ICRA), Publisher: IEEE, Pages: 3434-3440, ISSN: 1050-4729
- Author Web Link
- Cite
- Citations: 4
Rosa BG, Anastasova S, Lo B, 2021, Small-form wearable device for long-term monitoring of cardiac sounds on the body surface, 17th International Conference on Wearable and Implantable Body Sensor Networks (BSN), Publisher: IEEE, ISSN: 2376-8886
This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.