Below is a list of all relevant publications authored by Robotics Forum members.
- Showing results for:
- Reset all filters
Conference paperGoncalves Nunes UM, Demiris Y, 2020,
Entropy minimisation framework for event-based vision model estimation, 16th European Conference on Computer Vision 2020, Publisher: Springer, Pages: 161-176
We propose a novel Entropy Minimisation (EMin) frame-work for event-based vision model estimation. The framework extendsprevious event-based motion compensation algorithms to handle modelswhose outputs have arbitrary dimensions. The main motivation comesfrom estimating motion from events directly in 3D space (e.g.eventsaugmented with depth), without projecting them onto an image plane.This is achieved by modelling the event alignment according to candidateparameters and minimising the resultant dispersion. We provide a familyof suitable entropy loss functions and an efficient approximation whosecomplexity is only linear with the number of events (e.g.the complexitydoes not depend on the number of image pixels). The framework is eval-uated on several motion estimation problems, including optical flow androtational motion. As proof of concept, we also test our framework on6-DOF estimation by performing the optimisation directly in 3D space.
Conference paperLiu S, Lin Z, Wang Y, et al., 2020,
Shape adaptor: a learnable resizing module, European Conference on Computer Vision 2020, Publisher: Springer Verlag, Pages: 661-677, ISSN: 0302-9743
We present a novel resizing module for neural networks: shape adaptor, a drop-in enhancement built on top of traditional resizing layers, such as pooling, bilinear sampling, and strided convolution. Whilst traditional resizing layers have fixed and deterministic reshaping factors, our module allows for a learnable reshaping factor. Our implementation enables shape adaptors to be trained end-to-end without any additional supervision, through which network architectures can be optimised for each individual task, in a fully automated way. We performed experiments across seven image classification datasets, and results show that by simply using a set of our shape adaptors instead of the original resizing layers, performance increases consistently over human-designed networks, across all datasets. Additionally, we show the effectiveness of shape adaptors on two other applications: network compression and transfer learning.
Journal articleRussell F, Kormushev P, Vaidyanathan R, et al., 2020,
The impact of ACL laxity on a bicondylar robotic knee and implications in human joint biomechanics, IEEE Transactions on Biomedical Engineering, Vol: 67, Pages: 2817-2827, ISSN: 0018-9294
Objective: Elucidating the role of structural mechanisms in the knee can improve joint surgeries, rehabilitation, and understanding of biped locomotion. Identification of key features, however, is challenging due to limitations in simulation and in-vivo studies. In particular the coupling of the patello-femoral and tibio-femoral joints with ligaments and its impact on joint mechanics and movement is not understood. We investigate this coupling experimentally through the design and testing of a robotic sagittal plane model. Methods: We constructed a sagittal plane robot comprised of: 1) elastic links representing cruciate ligaments; 2) a bi-condylar joint; 3) a patella; and 4) actuator hamstrings and quadriceps. Stiffness and geometry were derived from anthropometric data. 10° - 110° squatting tests were executed at speeds of 0.1 - 0.25Hz over a range of anterior cruciate ligament (ACL) slack lengths. Results: Increasing ACL length compromised joint stability, yet did not impact quadriceps mechanical advantage and force required for squat. The trend was consistent through varying condyle contact point and ligament force changes. Conclusion: The geometry of the condyles allows the ratio of quadriceps to patella tendon force to compensate for contact point changes imparted by the removal of the ACL. Thus the system maintains a constant mechanical advantage. Significance: The investigation uncovers critical features of human knee biomechanics. Findings contribute to understanding of knee ligament damage, inform procedures for knee surgery and orthopaedic implant design, and support design of trans-femoral prosthetics and walking robots. Results further demonstrate the utility of robotics as a powerful means of studying human joint biomechanics.
Conference paperWang K, Marsh DM, Saputra RP, et al., 2020,
Design and control of SLIDER: an ultra-lightweight, knee-less, low-cost bipedal walking robot, Las Vegas, USA, International Conference on Intelligence Robots and Systems (IROS), Publisher: IEEE, Pages: 3488-3495
Most state-of-the-art bipedal robots are designedto be highly anthropomorphic and therefore possess legs withknees. Whilst this facilitates more human-like locomotion, thereare implementation issues that make walking with straight ornear-straight legs difficult. Most bipedal robots have to movewith a constant bend in the legs to avoid singularities at theknee joints, and to keep the centre of mass at a constant heightfor control purposes. Furthermore, having a knee on the legincreases the design complexity as well as the weight of the leg,hindering the robot’s performance in agile behaviours such asrunning and jumping.We present SLIDER, an ultra-lightweight, low-cost bipedalwalking robot with a novel knee-less leg design. This nonanthropomorphic straight-legged design reduces the weight ofthe legs significantly whilst keeping the same functionality asanthropomorphic legs. Simulation results show that SLIDER’slow-inertia legs contribute to less vertical motion in the centerof mass (CoM) than anthropomorphic robots during walking,indicating that SLIDER’s model is closer to the widely usedInverted Pendulum (IP) model. Finally, stable walking onflat terrain is demonstrated both in simulation and in thephysical world, and feedback control is implemented to addresschallenges with the physical robot.
Journal articleAlAttar A, Kormushev P, 2020,
Kinematic-model-free orientation control for robot manipulation using locally weighted dual quaternions, Robotics, Vol: 9, Pages: 1-12, ISSN: 2218-6581
Conventional control of robotic manipulators requires prior knowledge of their kinematic structure. Model-learning controllers have the advantage of being able to control robots without requiring a complete kinematic model and work well in less structured environments. Our recently proposed Encoderless controller has shown promising ability to control a manipulator without requiring any prior kinematic model whatsoever. However, this controller is only limited to position control, leaving orientation control unsolved. The research presented in this paper extends the state-of-the-art kinematic-model-free controller to handle orientation control to manipulate a robotic arm without requiring any prior model of the robot or any joint angle information during control. This paper presents a novel method to simultaneously control the position and orientation of a robot’s end effector using locally weighted dual quaternions. The proposed novel controller is also scaled up to control three-degrees-of-freedom robots.
Conference paperDing Z, Lepora N, Johns E, 2020,
Sim-to-real transfer for optical tactile sensing, IEEE International Conference on Robotics and Automation, Publisher: IEEE, Pages: 1639-1645, ISSN: 2152-4092
Deep learning and reinforcement learning meth-ods have been shown to enable learning of flexible and complexrobot controllers. However, the reliance on large amounts oftraining data often requires data collection to be carried outin simulation, with a number of sim-to-real transfer methodsbeing developed in recent years. In this paper, we study thesetechniques for tactile sensing using the TacTip optical tactilesensor, which consists of a deformable tip with a cameraobserving the positions of pins inside this tip. We designeda model for soft body simulation which was implemented usingthe Unity physics engine, and trained a neural network topredict the locations and angles of edges when in contact withthe sensor. Using domain randomisation techniques for sim-to-real transfer, we show how this framework can be used toaccurately predict edges with less than 1 mm prediction errorin real-world testing, without any real-world data at all.
Conference paperClark A, Rojas N, 2020,
Design and workspace characterisation of malleable robots, IEEE International Conference on Robotics and Automation, Publisher: IEEE, Pages: 9021-9027
For the majority of tasks performed by traditionalserial robot arms, such as bin picking or pick and place, onlytwo or three degrees of freedom (DOF) are required for motion;however, by augmenting the number of degrees of freedom,further dexterity of robot arms for multiple tasks can beachieved. Instead of increasing the number of joints of a robotto improve flexibility and adaptation, which increases controlcomplexity, weight, and cost of the overall system, malleablerobots utilise a variable stiffness link between joints allowing therelative positioning of the revolute pairs at each end of the linkto vary, thus enabling a low DOF serial robot to adapt acrosstasks by varying its workspace. In this paper, we present thedesign and prototyping of a 2-DOF malleable robot, calculatethe general equation of its workspace using a parameterisationbased on distance geometry—suitable for robot arms of variabletopology, and characterise the workspace categories that theend effector of the robot can trace via reconfiguration. Throughthe design and construction of the malleable robot we exploredesign considerations, and demonstrate the viability of theoverall concept. By using motion tracking on the physical robot,we show examples of the infinite number of workspaces thatthe introduced 2-DOF malleable robot can achieve.
Journal articleBaron N, Philippides A, Rojas N, 2020,
A robust geometric method of singularity avoidance for kinematically redundant planar parallel robot manipulators, Mechanism and Machine Theory, Vol: 151, Pages: 1-14, ISSN: 0094-114X
Jacobian-based methods of singularity analysis are known to be unreliable when applied to kinematically redundant parallel robot manipulators, due to their potential to miss certain singularities and incorrectly identify others in the manipulator’s workspace. In this paper, a geometric method of singularity avoidance for kinematically redundant planar parallel robot manipulators is presented, which firstly determines the manipulator’s proximity to a singularity and then computes how the kinematically redundant degree(s) of freedom should be optimised for the given pose of the end-effector. The singularity analysis is conducted by examining the mechanism in terms of the instantaneous centres of rotation of its corresponding mobility one sub-mechanisms when all but one of the actuators are locked, where the manipulator is in a type-II singularity when these points either are indeterminable or coincide with one another, and an index, rmin, is introduced which describes the minimum normalised distance from such conditions being met. A predictor-corrector method is employed to compute the configuration for which rmin is optimised, and is reachable without crossing a singularity. Finally, the advantages of the geometric method of singularity analysis are shown in comparison to traditional Jacobian-based methods when applied to kinematically redundant parallel robot manipulators.
Journal articleCursi F, Mylonas GP, Kormushev P, 2020,
Adaptive kinematic modelling for multiobjective control of a redundant surgical robotic tool, Robotics, Vol: 9, Pages: 68-68, ISSN: 2218-6581
Accurate kinematic models are essential for effective control of surgical robots. For tendon driven robots, which are common for minimally invasive surgery, the high nonlinearities in the transmission make modelling complex. Machine learning techniques are a preferred approach to tackle this problem. However, surgical environments are rarely structured, due to organs being very soft and deformable, and unpredictable, for instance, because of fluids in the system, wear and break of the tendons that lead to changes of the system’s behaviour. Therefore, the model needs to quickly adapt. In this work, we propose a method to learn the kinematic model of a redundant surgical robot and control it to perform surgical tasks both autonomously and in teleoperation. The approach employs Feedforward Artificial Neural Networks (ANN) for building the kinematic model of the robot offline, and an online adaptive strategy in order to allow the system to conform to the changing environment. To prove the capabilities of the method, a comparison with a simple feedback controller for autonomous tracking is carried out. Simulation results show that the proposed method is capable of achieving very small tracking errors, even when unpredicted changes in the system occur, such as broken joints. The method proved effective also in guaranteeing accurate tracking in teleoperation.
Journal articleEscribano Macias J, Goldbeck N, Hsu P-Y, et al., 2020,
Endogenous stochastic optimisation for relief distribution assisted with unmanned aerial vehicles, OR SPECTRUM, Vol: 42, Pages: 1089-1125, ISSN: 0171-6468
Unmanned aerial vehicles (UAVs) have been increasingly viewed as useful tools to assist humanitarian response in recent years. While organisations already employ UAVs for damage assessment during relief delivery, there is a lack of research into formalising a problem that considers both aspects simultaneously. This paper presents a novel endogenous stochastic vehicle routing problem that coordinates UAV and relief vehicle deployments to minimise overall mission cost. The algorithm considers stochastic damage levels in a transport network, with UAVs surveying the network to determine the actual network damages. Ground vehicles are simultaneously routed based on the information gathered by the UAVs. A case study based on the Haiti road network is solved using a greedy solution approach and an adapted genetic algorithm. Both methods provide a significant improvement in vehicle travel time compared to a deterministic approach and a non-assisted relief delivery operation, demonstrating the benefits of UAV-assisted response.
Journal articleFalck F, Doshi S, Tormento M, et al., 2020,
Robot DE NIRO: a human-centered, autonomous, mobile research platform for cognitively-enhanced manipulation, Frontiers in Robotics and AI, Vol: A17, ISSN: 2296-9144
We introduceRobot DE NIRO, an autonomous, collaborative, humanoid robot for mobilemanipulation. We built DE NIRO to perform a wide variety of manipulation behaviors, with afocus on pick-and-place tasks. DE NIRO is designed to be used in a domestic environment,especially in support of caregivers working with the elderly. Given this design focus, DE NIRO caninteract naturally, reliably, and safely with humans, autonomously navigate through environmentson command, intelligently retrieve or move target objects, and avoid collisions efficiently. Wedescribe DE NIRO’s hardware and software, including an extensive vision sensor suite of 2Dand 3D LIDARs, a depth camera, and a 360-degree camera rig; two types of custom grippers;and a custom-built exoskeleton called DE VITO. We demonstrate DE NIRO’s manipulationcapabilities in three illustrative challenges: First, we have DE NIRO perform a fetch-an-objectchallenge. Next, we add more cognition to DE NIRO’s object recognition and grasping abilities,confronting it with small objects of unknown shape. Finally, we extend DE NIRO’s capabilitiesinto dual-arm manipulation of larger objects. We put particular emphasis on the features thatenable DE NIRO to interact safely and naturally with humans. Our contribution is in sharinghow a humanoid robot with complex capabilities can be designed and built quickly with off-the-shelf hardware and open-source software. Supplementary material including our code, adocumentation, videos and the CAD models of several hardware parts are openly availableavailable athttps://www.imperial.ac.uk/robot-intelligence/software/
Conference paperFlageat M, Cully A, 2020,
Fast and stable MAP-Elites in noisy domains using deep grids, 2020 Conference on Artificial Life, Publisher: Massachusetts Institute of Technology, Pages: 273-282
Quality-Diversity optimisation algorithms enable the evolutionof collections of both high-performing and diverse solutions.These collections offer the possibility to quickly adapt andswitch from one solution to another in case it is not workingas expected. It therefore finds many applications in real-worlddomain problems such as robotic control. However, QD algo-rithms, like most optimisation algorithms, are very sensitive touncertainty on the fitness function, but also on the behaviouraldescriptors. Yet, such uncertainties are frequent in real-worldapplications. Few works have explored this issue in the spe-cific case of QD algorithms, and inspired by the literature inEvolutionary Computation, mainly focus on using samplingto approximate the ”true” value of the performances of a solu-tion. However, sampling approaches require a high number ofevaluations, which in many applications such as robotics, canquickly become impractical.In this work, we propose Deep-Grid MAP-Elites, a variantof the MAP-Elites algorithm that uses an archive of similarpreviously encountered solutions to approximate the perfor-mance of a solution. We compare our approach to previouslyexplored ones on three noisy tasks: a standard optimisationtask, the control of a redundant arm and a simulated Hexapodrobot. The experimental results show that this simple approachis significantly more resilient to noise on the behavioural de-scriptors, while achieving competitive performances in termsof fitness optimisation, and being more sample-efficient thanother existing approaches.
Conference paperLu Q, Baron N, Clark A, et al., 2020,
The RUTH Gripper: systematic object-invariant prehensile in-hand manipulation via reconfigurable underactuation, Robotics: Science and Systems, Publisher: RSS
We introduce a reconfigurable underactuated robothand able to perform systematic prehensile in-hand manipu-lations regardless of object size or shape. The hand utilisesa two-degree-of-freedom five-bar linkage as the palm of thegripper, with three three-phalanx underactuated fingers—jointlycontrolled by a single actuator—connected to the mobile revolutejoints of the palm. Three actuators are used in the robot handsystem, one for controlling the force exerted on objects by thefingers and two for changing the configuration of the palm.This novel layout allows decoupling grasping and manipulation,facilitating the planning and execution of in-hand manipulationoperations. The reconfigurable palm provides the hand withlarge grasping versatility, and allows easy computation of amap between task space and joint space for manipulation basedon distance-based linkage kinematics. The motion of objects ofdifferent sizes and shapes from one pose to another is thenstraightforward and systematic, provided the objects are keptgrasped. This is guaranteed independently and passively by theunderactuated fingers using a custom tendon routing method,which allows no tendon length variation when the relative fingerbase position changes with palm reconfigurations. We analysethe theoretical grasping workspace and manipulation capabilityof the hand, present algorithms for computing the manipulationmap and in-hand manipulation planning, and evaluate all theseexperimentally. Numerical and empirical results of several ma-nipulation trajectories with objects of different size and shapeclearly demonstrate the viability of the proposed concept.
Journal articleKinross JM, Mason SE, Mylonas G, et al., 2020,
Next-generation robotics in gastrointestinal surgery, Nature Reviews Gastroenterology and Hepatology, Vol: 17, Pages: 430-440, ISSN: 1759-5045
The global numbers of robotic gastrointestinal surgeries are increasing. However, the evidence base for robotic gastrointestinal surgery does not yet support its widespread adoption or justify its cost. The reasons for its continued popularity are complex, but a notable driver is the push for innovation — robotic surgery is seen as a compelling solution for delivering on the promise of minimally invasive precision surgery — and a changing commercial landscape delivers the promise of increased affordability. Novel systems will leverage the robot as a data-driven platform, integrating advances in imaging, artificial intelligence and machine learning for decision support. However, if this vision is to be realized, lessons must be heeded from current clinical trials and translational strategies, which have failed to demonstrate patient benefit. In this Perspective, we critically appraise current research to define the principles on which the next generation of gastrointestinal robotics trials should be based. We also discuss the emerging commercial landscape and define existing and new technologies.
Journal articleChappell D, Wang K, Kormushev P, 2020,
Asynchronous Real-Time Optimization of Footstep Placement and Timing in Bipedal Walking Robots
Online footstep planning is essential for bipedal walking robots to be ableto walk in the presence of disturbances. Until recently this has been achievedby only optimizing the placement of the footstep, keeping the duration of thestep constant. In this paper we introduce a footstep planner capable ofoptimizing footstep placement and timing in real-time by asynchronouslycombining two optimizers, which we refer to as asynchronous real-timeoptimization (ARTO). The first optimizer which runs at approximately 25 Hz,utilizes a fourth-order Runge-Kutta (RK4) method to accurately approximate thedynamics of the linear inverted pendulum (LIP) model for bipedal walking, thenuses non-linear optimization to find optimal footsteps and duration at a lowerfrequency. The second optimizer that runs at approximately 250 Hz, usesanalytical gradients derived from the full dynamics of the LIP model andconstraint penalty terms to perform gradient descent, which finds approximatelyoptimal footstep placement and timing at a higher frequency. By combining thetwo optimizers asynchronously, ARTO has the benefits of fast reactions todisturbances from the gradient descent optimizer, accurate solutions that avoidlocal optima from the RK4 optimizer, and increases the probability that afeasible solution will be found from the two optimizers. Experimentally, weshow that ARTO is able to recover from considerably larger pushes and producesfeasible solutions to larger reference velocity changes than a standardfootstep location optimizer, and outperforms using just the RK4 optimizeralone.
Conference paperCarvalho EDC, Clark R, Nicastro A, et al., 2020,
Scalable uncertainty for computer vision with functional variationalinference, CVPR 2020, Publisher: IEEE, Pages: 12003-12013
As Deep Learning continues to yield successful applications in ComputerVision, the ability to quantify all forms of uncertainty is a paramountrequirement for its safe and reliable deployment in the real-world. In thiswork, we leverage the formulation of variational inference in function space,where we associate Gaussian Processes (GPs) to both Bayesian CNN priors andvariational family. Since GPs are fully determined by their mean and covariancefunctions, we are able to obtain predictive uncertainty estimates at the costof a single forward pass through any chosen CNN architecture and for anysupervised learning task. By leveraging the structure of the induced covariancematrices, we propose numerically efficient algorithms which enable fasttraining in the context of high-dimensional tasks such as depth estimation andsemantic segmentation. Additionally, we provide sufficient conditions forconstructing regression loss functions whose probabilistic counterparts arecompatible with aleatoric uncertainty quantification.
Conference paperLu Q, Liang H, Nanayakkara DPT, et al., 2020,
Precise in-hand manipulation of soft objects using soft fingertips with tactile sensing and active deformation, IEEE International Conference on Soft Robotics, Publisher: IEEE, Pages: 52-57
While soft fingertips have shown significant development for grasping tasks, its ability to facilitate the manipulation of objects within the hand is still limited. Thanks to elasticity, soft fingertips enhance the ability to grasp soft objects. However, the in-hand manipulation of these objects has proved to be challenging, with both soft fingertips and traditional designs, as the control of coordinated fine fingertip motions and uncertainties for soft materials are intricate. This paper presents a novel technique for in-hand manipulating soft objects with precision. The approach is based on enhancing the dexterity of robot hands via soft fingertips with tactile sensing and active shape changing; such that pressurized air cavities act as soft tactile sensors to provide closed loop control of fingertip position and avoid object’s damage, and pneumatic-tuned positive-pressure deformations act as a localized soft gripper to perform additional translations and rotations. We model the deformation of the soft fingertips to predict the in-hand manipulation of soft objects and experimentally demonstrate the resulting in-hand manipulationcapabilities of a gripper of limited dexterity with an algorithm based on the proposed dual abilities. Results show that the introduced approach can ease and enhance the prehensile in-hand translation and rotation of soft objects for precision tasks across the hand workspace, without damage.
Journal articleWong MZ, Guillard B, Murai R, et al., 2020,
AnalogNet: Convolutional Neural Network Inference on Analog Focal Plane Sensor Processors
We present a high-speed, energy-efficient Convolutional Neural Network (CNN)architecture utilising the capabilities of a unique class of devices known asanalog Focal Plane Sensor Processors (FPSP), in which the sensor and theprocessor are embedded together on the same silicon chip. Unlike traditionalvision systems, where the sensor array sends collected data to a separateprocessor for processing, FPSPs allow data to be processed on the imagingdevice itself. This unique architecture enables ultra-fast image processing andhigh energy efficiency, at the expense of limited processing resources andapproximate computations. In this work, we show how to convert standard CNNs toFPSP code, and demonstrate a method of training networks to increase theirrobustness to analog computation errors. Our proposed architecture, coinedAnalogNet, reaches a testing accuracy of 96.9% on the MNIST handwritten digitsrecognition task, at a speed of 2260 FPS, for a cost of 0.7 mJ per frame.
Journal articleBaron N, Philippides A, Rojas N, 2020,
On the false positives and false negatives of the Jacobian matrix in kinematically redundant parallel mechanisms, IEEE Transactions on Robotics, Vol: 36, ISSN: 1552-3098
The Jacobian matrix is a highly popular tool for the control and performance analysis of closed-loop robots. Its usefulness in parallel mechanisms is certainly apparent, and its application to solve motion planning problems, or other higher level questions, has been seldom queried, or limited to non-redundant systems. In this paper, we discuss the shortcomings of the use of the Jacobian matrix under redundancy, in particular when applied to kinematically redundant parallel architectures with non-serially connected actuators. These architectures have become fairly popular recently as they allow the end-effector to achieve full rotations, which is an impossible task with traditional topologies. The problems with the Jacobian matrix in these novel systems arise from the need to eliminate redundant variables when forming it, resulting in both situations where the Jacobian incorrectly identifies singularities (false positive), and where it fails to identify singularities (false negative). These issues have thus far remained unaddressed in the literature. We highlight these limitations herein by demonstrating several cases using numerical examples of both planar and spatial architectures.
Conference paperZhang F, Demiris Y, 2020,
Learning grasping points for garment manipulation in robot-assisted dressing, 2020 IEEE International Conference on Robotics and Automation (ICRA), Publisher: IEEE, Pages: 9114-9120
Assistive robots have the potential to provide tremendous support for disabled and elderly people in their daily dressing activities. Recent studies on robot-assisted dressing usually simplify the setup of the initial robot configuration by manually attaching the garments on the robot end-effector and positioning them close to the user's arm. A fundamental challenge in automating such a process for robots is computing suitable grasping points on garments that facilitate robotic manipulation. In this paper, we address this problem by introducing a supervised deep neural network to locate a predefined grasping point on the garment, using depth images for their invariance to color and texture. To reduce the amount of real data required, which is costly to collect, we leverage the power of simulation to produce large amounts of labeled data. The network is jointly trained with synthetic datasets of depth images and a limited amount of real data. We introduce a robot-assisted dressing system that combines the grasping point prediction method, with a grasping and manipulation strategy which takes grasping orientation computation and robot-garment collision avoidance into account. The experimental results demonstrate that our method is capable of yielding accurate grasping point estimations. The proposed dressing system enables the Baxter robot to autonomously grasp a hospital gown hung on a rail, bring it close to the user and successfully dress the upper-body.
This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.