Publications
Below is a list of all relevant publications authored by Robotics Forum members.
Results
- Showing results for:
- Reset all filters
Search results
-
Journal articleCursi F, Mylonas GP, Kormushev P, 2020,
Adaptive kinematic modelling for multiobjective control of a redundant surgical robotic tool
, Robotics, Vol: 9, Pages: 68-68, ISSN: 2218-6581Accurate kinematic models are essential for effective control of surgical robots. For tendon driven robots, which are common for minimally invasive surgery, the high nonlinearities in the transmission make modelling complex. Machine learning techniques are a preferred approach to tackle this problem. However, surgical environments are rarely structured, due to organs being very soft and deformable, and unpredictable, for instance, because of fluids in the system, wear and break of the tendons that lead to changes of the system’s behaviour. Therefore, the model needs to quickly adapt. In this work, we propose a method to learn the kinematic model of a redundant surgical robot and control it to perform surgical tasks both autonomously and in teleoperation. The approach employs Feedforward Artificial Neural Networks (ANN) for building the kinematic model of the robot offline, and an online adaptive strategy in order to allow the system to conform to the changing environment. To prove the capabilities of the method, a comparison with a simple feedback controller for autonomous tracking is carried out. Simulation results show that the proposed method is capable of achieving very small tracking errors, even when unpredicted changes in the system occur, such as broken joints. The method proved effective also in guaranteeing accurate tracking in teleoperation.
-
Journal articleEscribano Macias J, Goldbeck N, Hsu P-Y, et al., 2020,
Endogenous stochastic optimisation for relief distribution assisted with unmanned aerial vehicles
, OR SPECTRUM, Vol: 42, Pages: 1089-1125, ISSN: 0171-6468Unmanned aerial vehicles (UAVs) have been increasingly viewed as useful tools to assist humanitarian response in recent years. While organisations already employ UAVs for damage assessment during relief delivery, there is a lack of research into formalising a problem that considers both aspects simultaneously. This paper presents a novel endogenous stochastic vehicle routing problem that coordinates UAV and relief vehicle deployments to minimise overall mission cost. The algorithm considers stochastic damage levels in a transport network, with UAVs surveying the network to determine the actual network damages. Ground vehicles are simultaneously routed based on the information gathered by the UAVs. A case study based on the Haiti road network is solved using a greedy solution approach and an adapted genetic algorithm. Both methods provide a significant improvement in vehicle travel time compared to a deterministic approach and a non-assisted relief delivery operation, demonstrating the benefits of UAV-assisted response.
-
Journal articleFalck F, Doshi S, Tormento M, et al., 2020,
Robot DE NIRO: a human-centered, autonomous, mobile research platform for cognitively-enhanced manipulation
, Frontiers in Robotics and AI, Vol: A17, ISSN: 2296-9144We introduceRobot DE NIRO, an autonomous, collaborative, humanoid robot for mobilemanipulation. We built DE NIRO to perform a wide variety of manipulation behaviors, with afocus on pick-and-place tasks. DE NIRO is designed to be used in a domestic environment,especially in support of caregivers working with the elderly. Given this design focus, DE NIRO caninteract naturally, reliably, and safely with humans, autonomously navigate through environmentson command, intelligently retrieve or move target objects, and avoid collisions efficiently. Wedescribe DE NIRO’s hardware and software, including an extensive vision sensor suite of 2Dand 3D LIDARs, a depth camera, and a 360-degree camera rig; two types of custom grippers;and a custom-built exoskeleton called DE VITO. We demonstrate DE NIRO’s manipulationcapabilities in three illustrative challenges: First, we have DE NIRO perform a fetch-an-objectchallenge. Next, we add more cognition to DE NIRO’s object recognition and grasping abilities,confronting it with small objects of unknown shape. Finally, we extend DE NIRO’s capabilitiesinto dual-arm manipulation of larger objects. We put particular emphasis on the features thatenable DE NIRO to interact safely and naturally with humans. Our contribution is in sharinghow a humanoid robot with complex capabilities can be designed and built quickly with off-the-shelf hardware and open-source software. Supplementary material including our code, adocumentation, videos and the CAD models of several hardware parts are openly availableavailable athttps://www.imperial.ac.uk/robot-intelligence/software/
-
Conference paperFlageat M, Cully A, 2020,
Fast and stable MAP-Elites in noisy domains using deep grids
, 2020 Conference on Artificial Life, Publisher: Massachusetts Institute of Technology, Pages: 273-282Quality-Diversity optimisation algorithms enable the evolutionof collections of both high-performing and diverse solutions.These collections offer the possibility to quickly adapt andswitch from one solution to another in case it is not workingas expected. It therefore finds many applications in real-worlddomain problems such as robotic control. However, QD algo-rithms, like most optimisation algorithms, are very sensitive touncertainty on the fitness function, but also on the behaviouraldescriptors. Yet, such uncertainties are frequent in real-worldapplications. Few works have explored this issue in the spe-cific case of QD algorithms, and inspired by the literature inEvolutionary Computation, mainly focus on using samplingto approximate the ”true” value of the performances of a solu-tion. However, sampling approaches require a high number ofevaluations, which in many applications such as robotics, canquickly become impractical.In this work, we propose Deep-Grid MAP-Elites, a variantof the MAP-Elites algorithm that uses an archive of similarpreviously encountered solutions to approximate the perfor-mance of a solution. We compare our approach to previouslyexplored ones on three noisy tasks: a standard optimisationtask, the control of a redundant arm and a simulated Hexapodrobot. The experimental results show that this simple approachis significantly more resilient to noise on the behavioural de-scriptors, while achieving competitive performances in termsof fitness optimisation, and being more sample-efficient thanother existing approaches.
-
Conference paperLu Q, Baron N, Clark A, et al., 2020,
The RUTH Gripper: systematic object-invariant prehensile in-hand manipulation via reconfigurable underactuation
, Robotics: Science and Systems, Publisher: RSSWe introduce a reconfigurable underactuated robothand able to perform systematic prehensile in-hand manipu-lations regardless of object size or shape. The hand utilisesa two-degree-of-freedom five-bar linkage as the palm of thegripper, with three three-phalanx underactuated fingers—jointlycontrolled by a single actuator—connected to the mobile revolutejoints of the palm. Three actuators are used in the robot handsystem, one for controlling the force exerted on objects by thefingers and two for changing the configuration of the palm.This novel layout allows decoupling grasping and manipulation,facilitating the planning and execution of in-hand manipulationoperations. The reconfigurable palm provides the hand withlarge grasping versatility, and allows easy computation of amap between task space and joint space for manipulation basedon distance-based linkage kinematics. The motion of objects ofdifferent sizes and shapes from one pose to another is thenstraightforward and systematic, provided the objects are keptgrasped. This is guaranteed independently and passively by theunderactuated fingers using a custom tendon routing method,which allows no tendon length variation when the relative fingerbase position changes with palm reconfigurations. We analysethe theoretical grasping workspace and manipulation capabilityof the hand, present algorithms for computing the manipulationmap and in-hand manipulation planning, and evaluate all theseexperimentally. Numerical and empirical results of several ma-nipulation trajectories with objects of different size and shapeclearly demonstrate the viability of the proposed concept.
-
Journal articleKinross JM, Mason SE, Mylonas G, et al., 2020,
Next-generation robotics in gastrointestinal surgery
, Nature Reviews Gastroenterology and Hepatology, Vol: 17, Pages: 430-440, ISSN: 1759-5045The global numbers of robotic gastrointestinal surgeries are increasing. However, the evidence base for robotic gastrointestinal surgery does not yet support its widespread adoption or justify its cost. The reasons for its continued popularity are complex, but a notable driver is the push for innovation — robotic surgery is seen as a compelling solution for delivering on the promise of minimally invasive precision surgery — and a changing commercial landscape delivers the promise of increased affordability. Novel systems will leverage the robot as a data-driven platform, integrating advances in imaging, artificial intelligence and machine learning for decision support. However, if this vision is to be realized, lessons must be heeded from current clinical trials and translational strategies, which have failed to demonstrate patient benefit. In this Perspective, we critically appraise current research to define the principles on which the next generation of gastrointestinal robotics trials should be based. We also discuss the emerging commercial landscape and define existing and new technologies.
-
Conference paperCarvalho EDC, Clark R, Nicastro A, et al., 2020,
Scalable uncertainty for computer vision with functional variationalinference
, CVPR 2020, Publisher: IEEE, Pages: 12003-12013As Deep Learning continues to yield successful applications in ComputerVision, the ability to quantify all forms of uncertainty is a paramountrequirement for its safe and reliable deployment in the real-world. In thiswork, we leverage the formulation of variational inference in function space,where we associate Gaussian Processes (GPs) to both Bayesian CNN priors andvariational family. Since GPs are fully determined by their mean and covariancefunctions, we are able to obtain predictive uncertainty estimates at the costof a single forward pass through any chosen CNN architecture and for anysupervised learning task. By leveraging the structure of the induced covariancematrices, we propose numerically efficient algorithms which enable fasttraining in the context of high-dimensional tasks such as depth estimation andsemantic segmentation. Additionally, we provide sufficient conditions forconstructing regression loss functions whose probabilistic counterparts arecompatible with aleatoric uncertainty quantification.
-
Conference paperLu Q, Liang H, Nanayakkara DPT, et al., 2020,
Precise in-hand manipulation of soft objects using soft fingertips with tactile sensing and active deformation
, IEEE International Conference on Soft Robotics, Publisher: IEEE, Pages: 52-57While soft fingertips have shown significant development for grasping tasks, its ability to facilitate the manipulation of objects within the hand is still limited. Thanks to elasticity, soft fingertips enhance the ability to grasp soft objects. However, the in-hand manipulation of these objects has proved to be challenging, with both soft fingertips and traditional designs, as the control of coordinated fine fingertip motions and uncertainties for soft materials are intricate. This paper presents a novel technique for in-hand manipulating soft objects with precision. The approach is based on enhancing the dexterity of robot hands via soft fingertips with tactile sensing and active shape changing; such that pressurized air cavities act as soft tactile sensors to provide closed loop control of fingertip position and avoid object’s damage, and pneumatic-tuned positive-pressure deformations act as a localized soft gripper to perform additional translations and rotations. We model the deformation of the soft fingertips to predict the in-hand manipulation of soft objects and experimentally demonstrate the resulting in-hand manipulationcapabilities of a gripper of limited dexterity with an algorithm based on the proposed dual abilities. Results show that the introduced approach can ease and enhance the prehensile in-hand translation and rotation of soft objects for precision tasks across the hand workspace, without damage.
-
Journal articleBaron N, Philippides A, Rojas N, 2020,
On the false positives and false negatives of the Jacobian matrix in kinematically redundant parallel mechanisms
, IEEE Transactions on Robotics, Vol: 36, ISSN: 1552-3098The Jacobian matrix is a highly popular tool for the control and performance analysis of closed-loop robots. Its usefulness in parallel mechanisms is certainly apparent, and its application to solve motion planning problems, or other higher level questions, has been seldom queried, or limited to non-redundant systems. In this paper, we discuss the shortcomings of the use of the Jacobian matrix under redundancy, in particular when applied to kinematically redundant parallel architectures with non-serially connected actuators. These architectures have become fairly popular recently as they allow the end-effector to achieve full rotations, which is an impossible task with traditional topologies. The problems with the Jacobian matrix in these novel systems arise from the need to eliminate redundant variables when forming it, resulting in both situations where the Jacobian incorrectly identifies singularities (false positive), and where it fails to identify singularities (false negative). These issues have thus far remained unaddressed in the literature. We highlight these limitations herein by demonstrating several cases using numerical examples of both planar and spatial architectures.
-
Conference paperZhang F, Demiris Y, 2020,
Learning grasping points for garment manipulation in robot-assisted dressing
, 2020 IEEE International Conference on Robotics and Automation (ICRA), Publisher: IEEE, Pages: 9114-9120Assistive robots have the potential to provide tremendous support for disabled and elderly people in their daily dressing activities. Recent studies on robot-assisted dressing usually simplify the setup of the initial robot configuration by manually attaching the garments on the robot end-effector and positioning them close to the user's arm. A fundamental challenge in automating such a process for robots is computing suitable grasping points on garments that facilitate robotic manipulation. In this paper, we address this problem by introducing a supervised deep neural network to locate a predefined grasping point on the garment, using depth images for their invariance to color and texture. To reduce the amount of real data required, which is costly to collect, we leverage the power of simulation to produce large amounts of labeled data. The network is jointly trained with synthetic datasets of depth images and a limited amount of real data. We introduce a robot-assisted dressing system that combines the grasping point prediction method, with a grasping and manipulation strategy which takes grasping orientation computation and robot-garment collision avoidance into account. The experimental results demonstrate that our method is capable of yielding accurate grasping point estimations. The proposed dressing system enables the Baxter robot to autonomously grasp a hospital gown hung on a rail, bring it close to the user and successfully dress the upper-body.
-
Journal articleLu Q, Clark A, Shen M, et al., 2020,
An origami-inspired variable friction surface for increasing the dexterity of robotic grippers
, IEEE Robotics and Automation Letters, Vol: 5, Pages: 2538-2545, ISSN: 2377-3766While the grasping capability of robotic grippers has shown significant development, the ability to manipulate objects within the hand is still limited. One explanation for this limitation is the lack of controlled contact variation between the grasped object and the gripper. For instance, human hands have the ability to firmly grip object surfaces, as well as slide over object faces, an aspect that aids the enhanced manipulation of objects within the hand without losing contact. In this letter, we present a parametric, origami-inspired thin surface capable of transitioning between a high friction and a low friction state, suitable for implementation as an epidermis in robotic fingers. A numerical analysis of the proposed surface based on its design parameters, force analysis, and performance in in-hand manipulation tasks is presented. Through the development of a simple two-fingered two-degree-of-freedom gripper utilizing the proposed variable-friction surfaces with different parameters, we experimentally demonstrate the improved manipulation capabilities of the hand when compared to the same gripper without changeable friction. Results show that the pattern density and valley gap are the main parameters that effect the in-hand manipulation performance. The origami-inspired thin surface with a higher pattern density generated a smaller valley gap and smaller height change, producing a more stable improvement of the manipulation capabilities of the hand.
-
Journal articleHe L, Lu Q, Abad S-A, et al., 2020,
Soft fingertips with tactile sensing and active deformation for robust grasping of delicate objects
, IEEE Robotics and Automation Letters, Vol: 5, Pages: 2714-2721, ISSN: 2377-3766Soft fingertips have shown significant adaptability for grasping a wide range of object shapes, thanks to elasticity. This ability can be enhanced to grasp soft, delicate objects by adding touch sensing. However, in these cases, the complete restraint and robustness of the grasps have proved to be challenging, as the exertion of additional forces on the fragile object can result in damage. This letter presents a novel soft fingertip design for delicate objects based on the concept of embedded air cavities, which allow the dual ability of tactile sensing and active shape-changing. The pressurized air cavities act as soft tactile sensors to control gripper position from internal pressure variation; and active fingertip deformation is achieved by applying positive pressure to these cavities, which then enable a delicate object to be kept securely in position, despite externally applied forces, by form closure. We demonstrate this improved grasping capability by comparing the displacement of grasped delicate objects exposed to high-speed motions. Results show that passive soft fingertips fail to restrain fragile objects at accelerations as low as 0.1 m/s 2 , in contrast, with the proposed fingertips delicate objects are completely secure even at accelerations of more than 5 m/s 2 .
-
Journal articleTsai Y-Y, Xiao B, Johns E, et al., 2020,
Constrained-space optimization and reinforcement learning for complex tasks
, IEEE Robotics and Automation Letters, Vol: 5, Pages: 683-690, ISSN: 2377-3766Learning from demonstration is increasingly used for transferring operator manipulation skills to robots. In practice, it is important to cater for limited data and imperfect human demonstrations, as well as underlying safety constraints. This article presents a constrained-space optimization and reinforcement learning scheme for managing complex tasks. Through interactions within the constrained space, the reinforcement learning agent is trained to optimize the manipulation skills according to a defined reward function. After learning, the optimal policy is derived from the well-trained reinforcement learning agent, which is then implemented to guide the robot to conduct tasks that are similar to the experts' demonstrations. The effectiveness of the proposed method is verified with a robotic suturing task, demonstrating that the learned policy outperformed the experts' demonstrations in terms of the smoothness of the joint motion and end-effector trajectories, as well as the overall task completion time.
-
Journal articleZhao M, Oude Vrielink TJC, Kogkas A, et al., 2020,
LaryngoTORS: a novel cable-driven parallel robotic system for transoral laser phonosurgery
, IEEE Robotics and Automation Letters, Vol: 5, Pages: 1516-1523, ISSN: 2377-3766Transoral laser phonosurgery is a commonly used surgical procedure in which a laser beam is used to perform incision, ablation or photocoagulation of laryngeal tissues. Two techniques are commonly practiced: free beam and fiber delivery. For free beam delivery, a laser scanner is integrated into a surgical microscope to provide an accurate laser scanning pattern. This approach can only be used under direct line of sight, which may cause increased postoperative pain to the patient and injury, is uncomfortable for the surgeon during prolonged operations, the manipulability is poor and extensive training is required. In contrast, in the fiber delivery technique, a flexible fiber is used to transmit the laser beam and therefore does not require direct line of sight. However, this can only achieve manual level accuracy, repeatability and velocity, and does not allow for pattern scanning. Robotic systems have been developed to overcome the limitations of both techniques. However, these systems offer limited workspace and degrees-of-freedom (DoF), limiting their clinical applicability. This work presents the LaryngoTORS, a robotic system that aims at overcoming the limitations of the two techniques, by using a cable-driven parallel mechanism (CDPM) attached at the end of a curved laryngeal blade for controlling the end tip of the laser fiber. The system allows autonomous generation of scanning patterns or user driven freepath scanning. Path scan validation demonstrated errors as low as 0.054±0.028 mm and high repeatability of 0.027±0.020 mm (6×2 mm arc line). Ex vivo tests on chicken tissue have been carried out. The results show the ability of the system to overcome limitations of current methods with high accuracy and repeatability using the superior fiber delivery approach.
-
Journal articleLiow L, Clark A, Rojas N, 2020,
OLYMPIC: a modular, tendon-driven prosthetic hand with novel finger and wrist coupling mechanisms
, IEEE Robotics and Automation Letters, Vol: 5, Pages: 299-306, ISSN: 2377-3766Prosthetic hands, while having shown significant progress in affordability, typically suffer from limited repairability, specifically by the user themselves. Several modular hands have been proposed to address this, but these solutions require handling of intricate components or are unsuitable for prosthetic use due to the large volume and weight resulting from added mechanical complexity to achieve this modularity. In this paper, we propose a fully modular design for a prosthetic hand with finger and wrist level modularity, allowing the removal and attachment of tendon-driven fingers without the need for tools, retendoning, and rewiring. Our innovative design enables placement of the motors behind the hand for remote actuation of the tendons, which are contained solely within the fingers. Details of the novel coupling-transmission mechanisms enabling this are presented; and the capabilities of a prototype using a control-independent grasping benchmark are discussed. The modular detachment torque of the fingers is also computed to analyse the trade-off between intentional removal and the ability to withstand external loads. Experiment results demonstrate that the prosthetic hand is able to grasp a wide range of household and food items, of different shape, size, and weight, without resulting in the ejection of fingers, while allowing a user to remove them easily using a single hand.
-
Journal articleGao Y, Chang HJ, Demiris Y, 2020,
User modelling using multimodal information for personalised dressing assistance
, IEEE Access, Vol: 8, Pages: 45700-45714, ISSN: 2169-3536 -
Conference paperNunes UM, Demiris Y, 2020,
Online unsupervised learning of the 3D kinematic structure of arbitrary rigid bodies
, IEEE/CVF International Conference on Computer Vision (ICCV), Publisher: IEEE Computer Soc, Pages: 3808-3816, ISSN: 1550-5499This work addresses the problem of 3D kinematic structure learning of arbitrary articulated rigid bodies from RGB-D data sequences. Typically, this problem is addressed by offline methods that process a batch of frames, assuming that complete point trajectories are available. However, this approach is not feasible when considering scenarios that require continuity and fluidity, for instance, human-robot interaction. In contrast, we propose to tackle this problem in an online unsupervised fashion, by recursively maintaining the metric distance of the scene's 3D structure, while achieving real-time performance. The influence of noise is mitigated by building a similarity measure based on a linear embedding representation and incorporating this representation into the original metric distance. The kinematic structure is then estimated based on a combination of implicit motion and spatial properties. The proposed approach achieves competitive performance both quantitatively and qualitatively in terms of estimation accuracy, even compared to offline methods.
-
Conference paperPardo F, Levdik V, Kormushev P, 2020,
Scaling all-goals updates in reinforcement learning using convolutional neural networks
, 34th AAAI Conference on Artificial Intelligence (AAAI 2020), Publisher: Association for the Advancement of Artificial Intelligence, Pages: 5355-5362, ISSN: 2374-3468Being able to reach any desired location in the environmentcan be a valuable asset for an agent. Learning a policy to nav-igate between all pairs of states individually is often not fea-sible. Anall-goals updatingalgorithm uses each transitionto learn Q-values towards all goals simultaneously and off-policy. However the expensive numerous updates in parallellimited the approach to small tabular cases so far. To tacklethis problem we propose to use convolutional network archi-tectures to generate Q-values and updates for a large numberof goals at once. We demonstrate the accuracy and generaliza-tion qualities of the proposed method on randomly generatedmazes and Sokoban puzzles. In the case of on-screen goalcoordinates the resulting mapping from frames todistance-mapsdirectly informs the agent about which places are reach-able and in how many steps. As an example of applicationwe show that replacing the random actions inε-greedy ex-ploration by several actions towards feasible goals generatesbetter exploratory trajectories on Montezuma’s Revenge andSuper Mario All-Stars games.
-
Conference paperChacon-Quesada R, Demiris Y, 2020,
Augmented reality controlled smart wheelchair using dynamic signifiers for affordance representation
, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Publisher: IEEEThe design of augmented reality interfaces for people with mobility impairments is a novel area with great potential, as well as multiple outstanding research challenges. In this paper we present an augmented reality user interface for controlling a smart wheelchair with a head-mounted display to provide assistance for mobility restricted people. Our motivation is to reduce the cognitive requirements needed to control a smart wheelchair. A key element of our platform is the ability to control the smart wheelchair using the concepts of affordances and signifiers. In addition to the technical details of our platform, we present a baseline study by evaluating our platform through user-trials of able-bodied individuals and two different affordances: 1) Door Go Through and 2) People Approach. To present these affordances to the user, we evaluated fixed symbol based signifiers versus our novel dynamic signifiers in terms of ease to understand the suggested actions and its relation with the objects. Our results show a clear preference for dynamic signifiers. In addition, we show that the task load reported by participants is lower when controlling the smart wheelchair with our augmented reality user interface compared to using the joystick, which is consistent with their qualitative answers.
-
Conference paperSaputra RP, Rakicevic N, Kormushev P, 2020,
Sim-to-real learning for casualty detection from ground projected point cloud data
, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2019), Publisher: IEEEThis paper addresses the problem of human body detection-particularly a human body lying on the ground (a.k.a. casualty)-using point cloud data. This ability to detect a casualty is one of the most important features of mobile rescue robots, in order for them to be able to operate autonomously. We propose a deep-learning-based casualty detection method using a deep convolutional neural network (CNN). This network is trained to be able to detect a casualty using a point-cloud data input. In the method we propose, the point cloud input is pre-processed to generate a depth image-like ground-projected heightmap. This heightmap is generated based on the projected distance of each point onto the detected ground plane within the point cloud data. The generated heightmap-in image form-is then used as an input for the CNN to detect a human body lying on the ground. To train the neural network, we propose a novel sim-to-real approach, in which the network model is trained using synthetic data obtained in simulation and then tested on real sensor data. To make the model transferable to real data implementations, during the training we adopt specific data augmentation strategies with the synthetic training data. The experimental results show that data augmentation introduced during the training process is essential for improving the performance of the trained model on real data. More specifically, the results demonstrate that the data augmentations on raw point-cloud data have contributed to a considerable improvement of the trained model performance.
This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.