Imperial College London


Faculty of EngineeringDyson School of Design Engineering

Research Postgraduate



n.rakicevic Website




Dyson BuildingSouth Kensington Campus





Publication Type

8 results found

Rakicevic N, Cully A, Kormushev P, 2021, Policy manifold search: exploring the manifold hypothesis for diversity-based neuroevolution, Proceedings of the 2021 Genetic and Evolutionary Computation Conference

Neuroevolution is an alternative to gradient-based optimisation that has thepotential to avoid local minima and allows parallelisation. The main limitingfactor is that usually it does not scale well with parameter spacedimensionality. Inspired by recent work examining neural network intrinsicdimension and loss landscapes, we hypothesise that there exists alow-dimensional manifold, embedded in the policy network parameter space,around which a high-density of diverse and useful policies are located. Thispaper proposes a novel method for diversity-based policy search viaNeuroevolution, that leverages learned representations of the policy networkparameters, by performing policy search in this learned representation space.Our method relies on the Quality-Diversity (QD) framework which provides aprincipled approach to policy search, and maintains a collection of diversepolicies, used as a dataset for learning policy representations. Further, weuse the Jacobian of the inverse-mapping function to guide the search in therepresentation space. This ensures that the generated samples remain in thehigh-density regions, after mapping back to the original space. Finally, weevaluate our contributions on four continuous-control tasks in simulatedenvironments, and compare to diversity-based baselines.

Conference paper

Saputra RP, Rakicevic N, Kuder I, Bilsdorfer J, Gough A, Dakin A, Cocker ED, Rock S, Harpin R, Kormushev Pet al., 2021, ResQbot 2.0: an improved design of a mobile rescue robot with an inflatable neck securing device for safe casualty extraction, Applied Sciences, Vol: 11, Pages: 1-18, ISSN: 2076-3417

Despite the fact that a large number of research studies have been conducted in the field of searchand rescue robotics, significantly little attention has been given to the development of rescue robotscapable of performing physical rescue interventions, including loading and transporting victims toa safe zone—i.e. casualty extraction tasks. The aim of this study is to develop a mobile rescue robotthat could assist first responders when saving casualties from a danger area by performing a casualty extraction procedure, whilst ensuring that no additional injury is caused by the operation andno additional lives are put at risk. In this paper, we present a novel design of ResQbot 2.0—a mobilerescue robot designed for performing the casualty extraction task. This robot is a stretcher-type casualty extraction robot, which is a significantly improved version of the initial proof-of-concept prototype, ResQbot (retrospectively referred to as ResQbot 1.0), that has been developed in our previous work. The proposed designs and development of the mechanical system of ResQbot 2.0, as wellas the method for safely loading a full body casualty onto the robot’s ‘stretcher bed’, are describedin detail based on the conducted literature review, evaluation of our previous work and feedbackprovided by medical professionals. To verify the proposed design and the casualty extraction procedure, we perform simulation experiments in Gazebo physics engine simulator. The simulationresults demonstrate the capability of ResQbot 2.0 to successfully carry out safe casualty extractions

Journal article

Saputra RP, Rakicevic N, Chappell D, Wang K, Kormushev Pet al., 2021, Hierarchical decomposed-objective model predictive control for autonomous casualty extraction, IEEE Access, Vol: 9, Pages: 39656-39679, ISSN: 2169-3536

In recent years, several robots have been developed and deployed to perform casualty extraction tasks. However, the majority of these robots are overly complex, and require teleoperation via either a skilled operator or a specialised device, and often the operator must be present at the scene to navigate safely around the casualty. Instead, improving the autonomy of such robots can reduce the reliance on expert operators and potentially unstable communication systems, while still extracting the casualty in a safe manner. There are several stages in the casualty extraction procedure, from navigating to the location of the emergency, safely approaching and loading the casualty, to finally navigating back to the medical assistance location. In this paper, we propose a Hierarchical Decomposed-Objective based Model Predictive Control (HiDO-MPC) method for safely approaching and manoeuvring around the casualty. We implement this controller on ResQbot — a proof-of-concept mobile rescue robot we previously developed — capable of safely rescuing an injured person lying on the ground, i.e. performing the casualty extraction procedure. HiDO-MPC achieves the desired casualty extraction behaviour by decomposing the main objective into multiple sub-objectives with a hierarchical structure. At every time step, the controller evaluates this hierarchical decomposed objective and generates the optimal control decision. We have conducted a number of experiments both in simulation and using the real robot to evaluate the proposed method’s performance, and compare it with baseline approaches. The results demonstrate that the proposed control strategy gives significantly better results than baseline approaches in terms of accuracy, robustness, and execution time, when applied to casualty extraction scenarios.

Journal article

Rakicevic N, Cully A, Kormushev P, 2020, Policy manifold search for improving diversity-based neuroevolution, Publisher: arXiv

Diversity-based approaches have recently gained popularity as an alternativeparadigm to performance-based policy search. A popular approach from thisfamily, Quality-Diversity (QD), maintains a collection of high-performingpolicies separated in the diversity-metric space, defined based on policies'rollout behaviours. When policies are parameterised as neural networks, i.e.Neuroevolution, QD tends to not scale well with parameter space dimensionality.Our hypothesis is that there exists a low-dimensional manifold embedded in thepolicy parameter space, containing a high density of diverse and feasiblepolicies. We propose a novel approach to diversity-based policy search viaNeuroevolution, that leverages learned latent representations of the policyparameters which capture the local structure of the data. Our approachiteratively collects policies according to the QD framework, in order to (i)build a collection of diverse policies, (ii) use it to learn a latentrepresentation of the policy parameters, (iii) perform policy search in thelearned latent space. We use the Jacobian of the inverse transformation(i.e.reconstruction function) to guide the search in the latent space. Thisensures that the generated samples remain in the high-density regions of theoriginal space, after reconstruction. We evaluate our contributions on threecontinuous control tasks in simulated environments, and compare todiversity-based baselines. The findings suggest that our approach yields a moreefficient and robust policy search process.

Conference paper

Saputra RP, Rakicevic N, Kormushev P, 2020, Sim-to-real learning for casualty detection from ground projected point cloud data, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2019), Publisher: IEEE

This paper addresses the problem of human body detection-particularly a human body lying on the ground (a.k.a. casualty)-using point cloud data. This ability to detect a casualty is one of the most important features of mobile rescue robots, in order for them to be able to operate autonomously. We propose a deep-learning-based casualty detection method using a deep convolutional neural network (CNN). This network is trained to be able to detect a casualty using a point-cloud data input. In the method we propose, the point cloud input is pre-processed to generate a depth image-like ground-projected heightmap. This heightmap is generated based on the projected distance of each point onto the detected ground plane within the point cloud data. The generated heightmap-in image form-is then used as an input for the CNN to detect a human body lying on the ground. To train the neural network, we propose a novel sim-to-real approach, in which the network model is trained using synthetic data obtained in simulation and then tested on real sensor data. To make the model transferable to real data implementations, during the training we adopt specific data augmentation strategies with the synthetic training data. The experimental results show that data augmentation introduced during the training process is essential for improving the performance of the trained model on real data. More specifically, the results demonstrate that the data augmentations on raw point-cloud data have contributed to a considerable improvement of the trained model performance.

Conference paper

Rakicevic N, Kormushev P, 2019, Active learning via informed search in movement parameter space for efficient robot task learning and transfer, Autonomous Robots, Vol: 43, Pages: 1917-1935, ISSN: 0929-5593

Learning complex physical tasks via trial-and-error is still challenging for high-degree-of-freedom robots. Greatest challenges are devising a suitable objective function that defines the task, and the high sample complexity of learning the task. We propose a novel active learning framework, consisting of decoupled task model and exploration components, which does not require an objective function. The task model is specific to a task and maps the parameter space, defining a trial, to the trial outcome space. The exploration component enables efficient search in the trial-parameter space to generate the subsequent most informative trials, by simultaneously exploiting all the information gained from previous trials and reducing the task model’s overall uncertainty. We analyse the performance of our framework in a simulation environment and further validate it on a challenging bimanual-robot puck-passing task. Results show that the robot successfully acquires the necessary skills after only 100 trials without any prior information about the task or target positions. Decoupling the framework’s components also enables efficient skill transfer to new environments which is validated experimentally.

Journal article

Rakicevic N, Rudovic O, Petridis S, Pantic Met al., 2017, Multi-modal neural conditional ordinal random fields for agreement level estimation, 23rd International Conference on Pattern Recognition (ICPR), Publisher: IEEE, Pages: 2228-2233, ISSN: 1051-4651

The ability to automatically detect the extent of agreement or disagreement a person expresses is an important indicator of inter-personal relations and emotion expression. Most of existing methods for automated analysis of human agreement from audio-visual data perform agreement detection using either audio or visual modality of human interactions. However, this is suboptimal as expression of different agreement levels is composed of various facial and vocal cues specific to the target level. To this end, we propose the first approach for multi-modal estimation of agreement intensity levels. Specifically, our model leverages the feature representation power of Multi-modal Neural Networks (NN) and discriminative power of Conditional Ordinal Random Fields (CORF) to achieve dynamic classification of agreement levels from videos. We show on the MAHNOB-Mimicry database of dyadic human interactions that the proposed approach outperforms its uni-modal and linear counterparts, and related models that can be applied to the target task.

Conference paper

Rakicevic N, Rudovic O, Petridis S, Pantic Met al., 2015, Neural Conditional Ordinal Random Fields for Agreement Level Estimation, 6th AAAC Affective Computing and Intelligent Interaction International Conference (ACII), Publisher: IEEE, Pages: 885-890, ISSN: 2156-8103

Conference paper

This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.

Request URL: Request URI: /respub/WEB-INF/jsp/search-html.jsp Query String: respub-action=search.html&id=00851572&limit=30&person=true