Publications

Search or filter publications

Filter by type:

Filter by publication type

Filter by year:

to

Results

  • Showing results for:
  • Reset all filters

Search results

  • CONFERENCE PAPER
    Elsdon J, Demiris Y,

    Assisted painting of 3D structures using shared control with a hand-held robot

    , IEEE International Conference on Robotics and Automation, Publisher: IEEE

    Abstract— We present a shared control method of painting3D geometries, using a handheld robot which has a singleautonomously controlled degree of freedom. The user scansthe robot near to the desired painting location, the singlemovement axis moves the spray head to achieve the requiredpaint distribution. A simultaneous simulation of the sprayingprocedure is performed, giving an open loop approximationof the current state of the painting. An online prediction ofthe best path for the spray nozzle actuation is calculated ina receding horizon fashion. This is calculated by producing amap of the paint required in the 2D space defined by nozzleposition on the gantry and the time into the future. A directedgraph then extracts its edge weights from this paint density mapand Dijkstra’s algorithm is then used to find the candidate forthe most effective path. Due to the heavy parallelisation of thisapproach and the majority of the calculations taking place on aGPU we can run the prediction loop in 32.6ms for a predictionhorizon of 1 second, this approach is computationally efficient,outperforming a greedy algorithm. The path chosen by theproposed method on average chooses a path in the top 15%of all paths as calculated by exhaustive testing. This approachenables development of real time path planning for assistedspray painting onto complicated 3D geometries. This methodcould be applied to applications such as assistive painting forpeople with disabilities, or accurate placement of liquid whenlarge scale positioning of the head is too expensive.
  • CONFERENCE PAPER
    Yoo YJ, Chang H, Yun S, Demiris Y, Choi JYet al.,

    Variational autoencoded regression: high dimensional regression of visual data on complex manifold

    , IEEE Conference on Computer Vision and Pattern Recognition, Publisher: IEEE

    This paper proposes a new high dimensional regression method by merging Gaussian process regression into a variational autoencoder framework. In contrast to other regression methods, the proposed method focuses on the case where output responses are on a complex high dimensional manifold, such as images. Our contributions are summarized as follows: (i) A new regression method estimating high dimensional image responses, which is not handled by existing regression algorithms, is proposed. (ii) The proposed regression method introduces a strategy to learn the latent space as well as the encoder and decoder so that the result of the regressed response in the latent space coincide with the corresponding response in the data space. (iii) The proposed regression is embedded into a generative model, and the whole procedure is developed by the variational autoencoder framework. We demonstrate the robustness and effectiveness of our method through a number of experiments on various visual data regression problems.
  • CONFERENCE PAPER
    Choi J, Chang HJ, Yun S, Fischer T, Demiris Y, Choi JYet al., 2017,

    Attentional correlation filter network for adaptive visual tracking

    , IEEE Conference on Computer Vision and Pattern Recognition, Publisher: IEEE

    We propose a new tracking framework with an attentional mechanism that chooses a subset of the associated correlation filters for increased robustness and computational efficiency. The subset of filters is adaptively selected by a deep attentional network according to the dynamic properties of the tracking target. Our contributions are manifold, and are summarised as follows: (i) Introducing the Attentional Correlation Filter Network which allows adaptive tracking of dynamic targets. (ii) Utilising an attentional network which shifts the attention to the best candidate modules, as well as predicting the estimated accuracy of currently inactive modules. (iii) Enlarging the variety of correlation filters which cover target drift, blurriness, occlusion, scale changes, and flexible aspect ratio. (iv) Validating the robustness and efficiency of the attentional mechanism for visual tracking through a number of experiments. Our method achieves similar performance to non real-time trackers, and state-of-the-art performance amongst real-time trackers.
  • JOURNAL ARTICLE
    Cully A, Demiris Y, Cully AHR, Demiris Yet al., 2017,

    Quality and Diversity Optimization: A Unifying Modular Framework

    , IEEE Transactions on Evolutionary Computation, Pages: 1-1, ISSN: 1089-778X

    The optimization of functions to find the best solution according to one or several objectives has a central role in many engineering and research fields. Recently, a new family of optimization algorithms, named Quality-Diversity optimization, has been introduced, and contrasts with classic algorithms. Instead of searching for a single solution, Quality-Diversity algorithms are searching for a large collection of both diverse and high-performing solutions. The role of this collection is to cover the range of possible solution types as much as possible, and to contain the best solution for each type. The contribution of this paper is threefold. Firstly, we present a unifying framework of Quality-Diversity optimization algorithms that covers the two main algorithms of this family (Multi-dimensional Archive of Phenotypic Elites and the Novelty Search with Local Competition), and that highlights the large variety of variants that can be investigated within this family. Secondly, we propose algorithms with a new selection mechanism for Quality-Diversity algorithms that outperforms all the algorithms tested in this paper. Lastly, we present a new collection management that overcomes the erosion issues observed when using unstructured collections. These three contributions are supported by extensive experimental comparisons of Quality-Diversity algorithms on three different experimental scenarios.
  • JOURNAL ARTICLE
    Georgiou T, Demiris Y, Georgiou T, Demiris Y, Georgiou T, Demiris Y, Georgiou T, Demiris Yet al., 2017,

    Adaptive user modelling in car racing games using behavioural and physiological data

    , USER MODELING AND USER-ADAPTED INTERACTION, Vol: 27, Pages: 267-311, ISSN: 0924-1868

    © 2017, The Author(s). Personalised content adaptation has great potential to increase user engagement in video games. Procedural generation of user-tailored content increases the self-motivation of players as they immerse themselves in the virtual world. An adaptive user model is needed to capture the skills of the player and enable automatic game content altering algorithms to fit the individual user. We propose an adaptive user modelling approach using a combination of unobtrusive physiological data to identify strengths and weaknesses in user performance in car racing games. Our system creates user-tailored tracks to improve driving habits and user experience, and to keep engagement at high levels. The user modelling approach adopts concepts from the Trace Theory framework; it uses machine learning to extract features from the user’s physiological data and game-related actions, and cluster them into low level primitives. These primitives are transformed and evaluated into higher level abstractions such as experience, exploration and attention. These abstractions are subsequently used to provide track alteration decisions for the player. Collection of data and feedback from 52 users allowed us to associate key model variables and outcomes to user responses, and to verify that the model provides statistically significant decisions personalised to the individual player. Tailored game content variations between users in our experiments, as well as the correlations with user satisfaction demonstrate that our algorithm is able to automatically incorporate user feedback in subsequent procedural content generation.
  • JOURNAL ARTICLE
    Korkinof D, Demiris Y, Korkinof D, Demiris Y, Korkinof D, Demiris Yet al., 2017,

    Multi-task and multi-kernel Gaussian process dynamical systems

    , PATTERN RECOGNITION, Vol: 66, Pages: 190-201, ISSN: 0031-3203

    In this work, we propose a novel method for rectifying damaged motion sequences in an unsupervised manner. In order to achieve maximal accuracy, the proposed model takes advantage of three key properties of the data: their sequential nature, the redundancy that manifests itself among repetitions of the same task, and the potential of knowledge transfer across different tasks. In order to do so, we formulate a factor model consisting of Gaussian Process Dynamical Systems (GPDS), where each factor corresponds to a single basic pattern in time and is able to represent their sequential nature. Factors collectively form a dictionary of fundamental trajectories shared among all sequences, thus able to capture recurrent patterns within the same or across different tasks. We employ variational inference to learn directly from incomplete sequences and perform maximum a-posteriori (MAP) estimates of the missing values. We have evaluated our model with a number of motion datasets, including robotic and human motion capture data. We have compared our approach to well-established methods in the literature in terms of their reconstruction error and our results indicate significant accuracy improvement across different datasets and missing data ratios. Concluding, we investigate the performance benefits of the multi-task learning scenario and how this improvement relates to the extent of component sharing that takes place.
  • JOURNAL ARTICLE
    Zambelli M, Demiris Y, Zambelli M, Demirisy Y, Zambelli M, Demiris Y, Zambelli M, Demiris Yet al., 2017,

    Online Multimodal Ensemble Learning Using Self-Learned Sensorimotor Representations

    , IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS, Vol: 9, Pages: 113-126, ISSN: 2379-8920

    © 2016 IEEE. Internal models play a key role in cognitive agents by providing on the one hand predictions of sensory consequences of motor commands (forward models), and on the other hand inverse mappings (inverse models) to realize tasks involving control loops, such as imitation tasks. The ability to predict and generate new actions in continuously evolving environments intrinsically requiring the use of different sensory modalities is particularly relevant for autonomous robots, which must also be able to adapt their models online. We present a learning architecture based on self-learned multimodal sensorimotor representations. To attain accurate forward models, we propose an online heterogeneous ensemble learning method that allows us to improve the prediction accuracy by leveraging differences of multiple diverse predictors. We further propose a method to learn inverse models on-the-fly to equip a robot with multimodal learning skills to perform imitation tasks using multiple sensory modalities. We have evaluated the proposed methods on an iCub humanoid robot. Since no assumptions are made on the robot kinematic/dynamic structure, the method can be applied to different robotic platforms.
  • CONFERENCE PAPER
    Chang HJ, Fischer T, Petit M, Zambelli M, Demiris Y, Chang HJ, Fischer T, Petit M, Zambelli M, Demiris Yet al., 2016,

    Kinematic structure correspondences via hypergraph matching

    , IEEE Conference on Computer Vision and Pattern Recognition, Publisher: IEEE, Pages: 4216-4225, ISSN: 1063-6919

    In this paper, we present a novel framework for finding the kinematic structure correspondence between two objects in videos via hypergraph matching. In contrast to prior appearance and graph alignment based matching methods which have been applied among two similar static images, the proposed method finds correspondences between two dynamic kinematic structures of heterogeneous objects in videos. Our main contributions can be summarised as follows: (i) casting the kinematic structure correspondence problem into a hypergraph matching problem, incorporating multi-order similarities with normalising weights, (ii) a structural topology similarity measure by a new topology constrained subgraph isomorphism aggregation, (iii) a kinematic correlation measure between pairwise nodes, and (iv) a combinatorial local motion similarity measure using geodesic distance on the Riemannian manifold. We demonstrate the robustness and accuracy of our method through a number of experiments on complex articulated synthetic and real data.
  • CONFERENCE PAPER
    Choi J, Chang HJ, Jeong J, Demiris Y, Choi JY, Choi J, Chang H, Jeong J, Demiris Y, Choi JYet al., 2016,

    Visual Tracking Using Attention-Modulated Disintegration and Integration

    , 29th IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Publisher: IEEE, Pages: 4321-4330, ISSN: 1063-6919

    In this paper, we present a novel attention-modulatedvisual tracking algorithm that decomposes an object intointo multiple cognitive units, and trains multiple elemen-tary trackers in order to modulate the distribution of at-tention according to various feature and kernel types. Inthe integration stage it recombines the units to memorizeand recognize the target object effectively. With respectto the elementary trackers, we present a novel attentionalfeature-based correlation filter (AtCF) that focuses on dis-tinctive attentional features. The effectiveness of the pro-posed algorithm is validated through experimental compar-ison with state-of-the-art methods on widely-used trackingbenchmark datasets.
  • CONFERENCE PAPER
    Fischer T, Demiris Y, Fischer T, Demiris Yet al., 2016,

    Markerless Perspective Taking for Humanoid Robots in Unconstrained Environments

    , IEEE International Conference on Robotics and Automation (ICRA), Publisher: IEEE, Pages: 3309-3316, ISSN: 1050-4729

    Perspective taking enables humans to imagine the world from another viewpoint. This allows reasoning about the state of other agents, which in turn is used to more accurately predict their behavior. In this paper, we equip an iCub humanoid robot with the ability to perform visuospatial perspective taking (PT) using a single depth camera mounted above the robot. Our approach has the distinct benefit that the robot can be used in unconstrained environments, as opposed to previous works which employ marker-based motion capture systems. Prior to and during the PT, the iCub learns the environment, recognizes objects within the environment, and estimates the gaze of surrounding humans. We propose a new head pose estimation algorithm which shows a performance boost by normalizing the depth data to be aligned with the human head. Inspired by psychological studies, we employ two separate mechanisms for the two different types of PT. We implement line of sight tracing to determine whether an object is visible to the humans (level 1 PT). For more complex PT tasks (level 2 PT), the acquired point cloud is mentally rotated, which allows algorithms to reason as if the input data was acquired from an egocentric perspective. We show that this can be used to better judge where object are in relation to the humans. The multifaceted improvements to the PT pipeline advance the state of the art, and move PT in robots to markerless, unconstrained environments.
  • CONFERENCE PAPER
    Gao Y, Chang HJ, Demiris Y, 2016,

    Personalised assistive dressing by humanoid robots using multi-modal information

    , Workshop on Human-Robot Interfaces for Enhanced Physical Interactions at ICRA

    In this paper, we present an approach to enable a humanoid robot to provide personalised dressing assistance for human users using multi-modal information. A depth sensor is mounted on top of the robot to provide visual information, and the robot end effectors are equipped with force sensors to provide haptic information. We use visual information to model the movement range of human upper-body parts. The robot plans the dressing motions using the movement rangemodels and real-time human pose. During assistive dressing, the force sensors are used to detect external force resistances. We present how the robot locally adjusts its motions based on the detected forces. In the experiments we show that the robot can assist human to wear a sleeveless jacket while reacting tothe force resistances.
  • CONFERENCE PAPER
    Gao Y, Chang HJ, Demiris Y, Gao Y, Chang HJ, Demiris Yet al., 2016,

    Iterative Path Optimisation for Personalised Dressing Assistance using Vision and Force Information

    , IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Publisher: IEEE, Pages: 4398-4403

    We propose an online iterative path optimisationmethod to enable a Baxter humanoid robot to assist humanusers to dress. The robot searches for the optimal personaliseddressing path using vision and force sensor information: visioninformation is used to recognise the human pose and model themovement space of upper-body joints; force sensor informationis used for the robot to detect external force resistance andto locally adjust its motion. We propose a new stochastic pathoptimisation method based on adaptive moment estimation. Wefirst compare the proposed method with other path optimisationalgorithms on synthetic data. Experimental results show thatthe performance of the method achieves the smallest error withfewer iterations and less computation time. We also evaluatereal-world data by enabling the Baxter robot to assist realhuman users with their dressing.
  • CONFERENCE PAPER
    Georgiou T, Demiris Y, Georgiou T, Demiris Y, Georgiou T, Demiris Yet al., 2016,

    Personalised Track Design in Car Racing Games

    , IEEE Conference on Computational Intelligence and Games (CIG), Publisher: IEEE, ISSN: 2325-4270

    © 2016 IEEE. Real-time adaptation of computer games' content to the users' skills and abilities can enhance the player's engagement and immersion. Understanding of the user's potential while playing is of high importance in order to allow the successful procedural generation of user-tailored content. We investigate how player models can be created in car racing games. Our user model uses a combination of data from unobtrusive sensors, while the user is playing a car racing simulator. It extracts features through machine learning techniques, which are then used to comprehend the user's gameplay, by utilising the educational theoretical frameworks of the Concept of Flow and Zone of Proximal Development. The end result is to provide at a next stage a new track that fits to the user needs, which aids both the training of the driver and their engagement in the game. In order to validate that the system is designing personalised tracks, we associated the average performance from 41 users that played the game, with the difficulty factor of the generated track. In addition, the variation in paths of the implemented tracks between users provides a good indicator for the suitability of the system.
  • CONFERENCE PAPER
    Petit M, Demiris Y, Petit M, Demiris Yet al., 2016,

    Hierarchical Action Learning by Instruction Through Interactive Grounding of Body Parts and Proto-actions

    , IEEE International Conference on Robotics and Automation (ICRA), Publisher: IEEE, Pages: 3375-3382, ISSN: 1050-4729

    Learning by instruction allows humans programming a robot to achieve a task using spoken language, without the requirement of being able to do the task themselves, which can be problematic for users with motor impairments. We provide a developmental framework to program the humanoid robot iCub without any hand-coded a-priori knowledge about any motor skills. Inspired by child development theories, the system involves hierarchical learning, starting with the human verbally labelling robot body parts. The robot can then focus its attention on a precise body part during robot motor babbling, and link the on-the-fly spoken descriptions of proto-actions to angle values of a specific joint. The direct grounding of proto-actions is possible through the use of a linear model which calculates the effects on the joint of the proto-action and the body part used, allowing a generalisation of the proto-action if the joint has never been used before. Eventually, transferring the grounding is allowed via learning by instructions where humans can combine the newly acquired proto-actions to build primitives and more complex actions by scaffolding them. The framework has been validated using a humanoid robot iCub, which is able to learn without any prior knowledge: 1) the name of its fingers and the corresponding joint number, 2) how to fold and unfold them and 3) how to close or open its hand and how to show numbers with its fingers.
  • CONFERENCE PAPER
    Petit M, Fischer T, Demiris Y, 2016,

    Towards the Emergence of Procedural Memories from Lifelong Multi-Modal Streaming Memories for Cognitive Robots

    , Workshop on Machine Learning Methods for High-Level Cognitive Capabilities in Robotics at IEEE/RSJ IROS

    Various research topics are emerging as the demand for intelligent lifelong interactions between robot and humans increases. Among them, we can find the examination of persistent storage, the continuous unsupervised annotation of memories and the usage of data at high-frequency over long periods of time. We recently proposed a lifelong autobiographical memory architecture tackling some of these challenges, allowing the iCub humanoid robot to 1) create new memories for both actions that are self-executed and observed from humans, 2) continuously annotate these actions in an unsupervised manner, and 3) use reasoning modules to augment these memories a-posteriori. In this paper, we present a reasoning algorithm which generalises the robots’ understanding of actions by finding the point of commonalities with the former ones. In particular, we generated and labelled templates of pointing actions in different directions. This represents a first step towards the emergence of a procedural memory within a long-term autobiographical memory framework for robots.
  • CONFERENCE PAPER
    Zambelli M, Demiris Y, Zambelli M, Demiris Yet al., 2016,

    Multimodal Imitation using Self-learned Sensorimotor Representations

    , IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Publisher: IEEE, Pages: 3953-3958, ISSN: 2153-0866

    Although many tasks intrinsically involve multiplemodalities, often only data from a single modality are used toimprove complex robots acquisition of new skills. We presenta method to equip robots with multimodal learning skills toachieve multimodal imitation on-the-fly on multiple concurrenttask spaces, including vision, touch and proprioception, onlyusing self-learned multimodal sensorimotor relations, withoutthe need of solving inverse kinematic problems or explicit analyticalmodels formulation. We evaluate the proposed methodon a humanoid iCub robot learning to interact with a pianokeyboard and imitating a human demonstration. Since noassumptions are made on the kinematic structure of the robot,the method can be also applied to different robotic platforms.
  • CONFERENCE PAPER
    Zambelli M, Fischer T, Petit M, Chang HJ, Cully A, Demiris Yet al., 2016,

    Towards Anchoring Self-Learned Representations to Those of Other Agents

    , Workshop on Bio-inspired Social Robot Learning in Home Scenarios IEEE/RSJ International Conference on Intelligent Robots and Systems, Publisher: Institute of Electrical and Electronics Engineers (IEEE)

    In the future, robots will support humans in their every day activities. One particular challenge that robots will face is understanding and reasoning about the actions of other agents in order to cooperate effectively with humans. We propose to tackle this using a developmental framework, where the robot incrementally acquires knowledge, and in particular 1) self-learns a mapping between motor commands and sensory consequences, 2) rapidly acquires primitives and complex actions by verbal descriptions and instructions from a human partner, 3) discoverscorrespondences between the robots body and other articulated objects and agents, and 4) employs these correspondences to transfer the knowledge acquired from the robots point of view to the viewpoint of the other agent. We show that our approach requires very little a-priori knowledge to achieve imitation learning, to find correspondent body parts of humans, and allows taking the perspective of another agent. This represents a step towards the emergence of a mirror neuron like system based on self-learned representations.
  • CONFERENCE PAPER
    Chang HJ, Demiris Y, Chang HJ, Demiris Yet al., 2015,

    Unsupervised Learning of Complex Articulated Kinematic Structures combining Motion and Skeleton Information

    , IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Publisher: IEEE, Pages: 3138-3146, ISSN: 1063-6919

    In this paper we present a novel framework for unsupervised kinematic structure learning of complex articulated objects from a single-view image sequence. In contrast to prior motion information based methods, which estimate relatively simple articulations, our method can generate arbitrarily complex kinematic structures with skeletal topology by a successive iterative merge process. The iterative merge process is guided by a skeleton distance function which is generated from a novel object boundary generation method from sparse points. Our main contributions can be summarised as follows: (i) Unsupervised complex articulated kinematic structure learning by combining motion and skeleton information. (ii) Iterative fine-to-coarse merging strategy for adaptive motion segmentation and structure smoothing. (iii) Skeleton estimation from sparse feature points. (iv) A new highly articulated object dataset containing multi-stage complexity with ground truth. Our experiments show that the proposed method out-performs state-of-the-art methods both quantitatively and qualitatively.
  • CONFERENCE PAPER
    Gao Y, Chang HJ, Demiris Y, Gao Y, Chang HJ, Demiris Yet al., 2015,

    User Modelling for Personalised Dressing Assistance by Humanoid Robots

    , IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Publisher: IEEE, Pages: 1840-1845, ISSN: 2153-0858

    Assistive robots can improve the well-being of disabled or frail human users by reducing the burden that activities of daily living impose on them. To enable personalised assistance, such robots benefit from building a user-specific model, so that the assistance is customised to the particular set of user abilities. In this paper, we present an end-to-end approach for home-environment assistive humanoid robots to provide personalised assistance through a dressing application for users who have upper-body movement limitations. We use randomised decision forests to estimate the upper-body pose of users captured by a top-view depth camera, and model the movement space of upper-body joints using Gaussian mixture models. The movement space of each upper-body joint consists of regions with different reaching capabilities. We propose a method which is based on real-time upper-body pose and user models to plan robot motions for assistive dressing. We validate each part of our approach and test the whole system, allowing a Baxter humanoid robot to assist human to wear a sleeveless jacket.
  • CONFERENCE PAPER
    Georgiou T, Demiris Y, Georgiou T, Demiris Y, Georgiou T, Demiris Yet al., 2015,

    Predicting car states through learned models of vehicle dynamics and user behaviours

    , Intelligent Vehicles Symposium (IV), Publisher: IEEE, Pages: 1240-1245, ISSN: 1931-0587

    The ability to predict forthcoming car states is crucial for the development of smart assistance systems. Forthcoming car states do not only depend on vehicle dynamics but also on user behaviour. In this paper, we describe a novel prediction methodology by combining information from both sources - vehicle and user - using Gaussian Processes. We then apply this method in the context of high speed car racing. Results show that the forthcoming position and speed of the car can be predicted with low Root Mean Square Error through the trained model.
  • CONFERENCE PAPER
    Kormushev P, Demiris Y, Caldwell DG, Kormushev P, Demiris Y, Caldwell DGet al., 2015,

    Encoderless Position Control of a Two-Link Robot Manipulator

    , IEEE International Conference on Robotics and Automation (ICRA), Publisher: IEEE COMPUTER SOC, Pages: 943-949, ISSN: 1050-4729

    Encoders have been an inseparable part of robotssince the very beginning of modern robotics in the 1950s. As aresult, the foundations of robot control are built on the conceptsof kinematics and dynamics of articulated rigid bodies, whichrely on explicitly measuring the robot configuration in termsof joint angles – done by encoders.In this paper, we propose a radically new concept forcontrolling robots called Encoderless Robot Control (EnRoCo).The concept is based on our hypothesis that it is possible tocontrol a robot without explicitly measuring its joint angles, bymeasuring instead the effects of the actuation on its end-effector.To prove the feasibility of this unconventional control approach,we propose a proof-of-concept control algorithm for encoderlessposition control of a robot’s end-effector in task space. Wedemonstrate a prototype implementation of this controller ina dynamics simulation of a two-link robot manipulator. Theprototype controller is able to successfully control the robot’send-effector to reach a reference position, as well as to trackcontinuously a desired trajectory.Notably, we demonstrate how this novel controller can copewith something that traditional control approaches fail to do:adapt on-the-fly to changes in the kinematics of the robot, suchas changing the lengths of the links.
  • CONFERENCE PAPER
    Kormushev P, Demiris Y, Caldwell DG, Kormushev P, Demiris Y, Caldwell DGet al., 2015,

    Kinematic-free Position Control of a 2-DOF Planar Robot Arm

    , IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Publisher: IEEE, Pages: 5518-5525, ISSN: 2153-0858

    This paper challenges the well-established as-sumption in robotics that in order to control a robot itis necessary to know its kinematic information, that is, thearrangement of links and joints, the link dimensions and thejoint positions. We propose a kinematic-free robot controlconcept that does not require any prior kinematic knowledge.The concept is based on our hypothesis that it is possible tocontrol a robot without explicitly measuring its joint angles, bymeasuring instead the effects of the actuation on its end-effector.We implement a proof-of-concept encoderless robot con-troller and apply it for the position control of a physical 2-DOF planar robot arm. The prototype controller is able tosuccessfully control the robot to reach a reference position, aswell as to track a continuous reference trajectory. Notably, wedemonstrate how this novel controller can cope with somethingthat traditional control approaches fail to do: adapt to drastickinematic changes such as 100% elongation of a link, 35-degreeangular offset of a joint, and even a complete overhaul of thekinematics involving the addition of new joints and links.
  • CONFERENCE PAPER
    Kucukyilmaz A, Demiris Y, Kucukyilmaz A, Demiris Y, Kucukyilmaz A, Demiris Y, Küçükyilmaz A, Demiris Yet al., 2015,

    One-shot assistance estimation from expert demonstrations for a shared control wheelchair system

    , International Symposium on Robot and Human Interactive Communication (RO-MAN), Publisher: IEEE, Pages: 438-443

    An emerging research problem in the field of assistive robotics is the design of methodologies that allow robots to provide human-like assistance to the users. Especially within the rehabilitation domain, a grand challenge is to program a robot to mimic the operation of an occupational therapist, intervening with the user when necessary so as to improve the therapeutic power of the assistive robotic system. We propose a method to estimate assistance policies from expert demonstrations to present human-like intervention during navigation in a powered wheelchair setup. For this purpose, we constructed a setting, where a human offers assistance to the user over a haptic shared control system. The robot learns from human assistance demonstrations while the user is actively driving the wheelchair in an unconstrained environment. We train a Gaussian process regression model to learn assistance commands given past and current actions of the user and the state of the environment. The results indicate that the model can estimate human assistance after only a single demonstration, i.e. in one-shot, so that the robot can help the user by selecting the appropriate assistance in a human-like fashion.
  • JOURNAL ARTICLE
    Lee K, Ognibene D, Chang HJ, Kim T-K, Demiris Y, Lee K, Ognibene D, Chang HJ, Kim TK, Demiris Y, Lee K, Ognibene D, Chang HJ, Kim T-K, Demiris Y, Lee K, Ognibene D, Chang H, Kim T-K, Demiris Yet al., 2015,

    STARE: Spatio-Temporal Attention Relocation for Multiple Structured Activities Detection

    , IEEE TRANSACTIONS ON IMAGE PROCESSING, Vol: 24, Pages: 5916-5927, ISSN: 1057-7149

    © 2015 IEEE. We present a spatiooral attention relocation (STARE) method, an information-theoretic approach for efficient detection of simultaneously occurring structured activities. Given multiple human activities in a scene, our method dynamically focuses on the currently most informative activity. Each activity can be detected without complete observation, as the structure of sequential actions plays an important role on making the system robust to unattended observations. For such systems, the ability to decide where and when to focus is crucial to achieving high detection performances under resource bounded condition. Our main contributions can be summarized as follows: 1) information-theoretic dynamic attention relocation framework that allows the detection of multiple activities efficiently by exploiting the activity structure information and 2) a new high-resolution data set of temporally-structured concurrent activities. Our experiments on applications show that the STARE method performs efficiently while maintaining a reasonable level of accuracy.
  • JOURNAL ARTICLE
    Petit M, Fischer T, Demiris Y, Petit M, Fischer T, Demiris Y, Petit M, Fischer T, Demiris Yet al., 2015,

    Lifelong Augmentation of Multi-Modal Streaming Autobiographical Memories

    , IEEE Transactions on Cognitive and Developmental Systems, Vol: 8, Pages: 201-213, ISSN: 2379-8920

    Robot systems that interact with humans over extended periods of time will benefit from storing and recalling large amounts of accumulated sensorimotor and interaction data. We provide a principled framework for the cumulative organisation of streaming autobiographical data so that data can be continuously processed and augmented as the processing and reasoning abilities of the agent develop and further interactions with humans take place. As an example, we show how a kinematic structure learning algorithm reasons a-posteriori about the skeleton of a human hand. A partner can be asked to provide feedback about the augmented memories, which can in turn be supplied to the reasoning processes in order to adapt their parameters. We employ active, multi-modal remembering, so the robot as well as humans can gain insights of both the original and augmented memories. Our framework is capable of storing discrete and continuous data in real-time. The data can cover multiple modalities and several layers of abstraction (e.g. from raw sound signals over sentences to extracted meanings). We show a typical interaction with a human partner using an iCub humanoid robot. The framework is implemented in a platform-independent manner. In particular, we validate its multi platform capabilities using the iCub, Baxter and NAO robots. We also provide an interface to cloud based services, which allow automatic annotation of episodes. Our framework is geared towards the developmental robotics community, as it 1) provides a variety of interfaces for other modules, 2) unifies previous works on autobiographical memory, and 3) is licensed as open source software.
  • CONFERENCE PAPER
    Sarabia M, Lee K, Demiris Y, Sarabia M, Lee K, Demiris Y, Sarabia M, Lee K, Demiris Y, Sarabia M, Lee K, Demiris Yet al., 2015,

    Towards a Synchronised Grammars Framework for Adaptive Musical Human-Robot Collaboration

    , IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Publisher: IEEE, Pages: 715-721

    We present an adaptive musical collaboration framework for interaction between a human and a robot. The aim of our work is to develop a system that receives feedback from the user in real time and learns the music progression style of the user over time. To tackle this problem, we represent a song as a hierarchically structured sequence of music primitives. By exploiting the sequential constraints of these primitives inferred from the structural information combined with user feedback, we show that a robot can play music in accordance with the user’s anticipated actions. We use Stochastic Context-Free Grammars augmented with the knowledge of the learnt user’s preferences.We provide synthetic experiments as well as a pilot study with a Baxter robot and a tangible music table. The synthetic results show the synchronisation and adaptivity features of our framework and the pilot study suggest these are applicable to create an effective musical collaboration experience.
  • JOURNAL ARTICLE
    Soh H, Demiris Y, Soh H, Demiris Y, Soh H, Demiris Y, Soh H, Demiris Y, Soh H, Demiris Yet al., 2015,

    Spatio-Temporal Learning With the Online Finite and Infinite Echo-State Gaussian Processes

    , IEEE Transactions on Neural Networks and Learning Systems, Vol: 26, Pages: 522-536, ISSN: 2162-237X

    Successful biological systems adapt to change. In this paper, we are principally concerned with adaptive systems that operate in environments where data arrives sequentially and is multivariate in nature, for example, sensory streams in robotic systems. We contribute two reservoir inspired methods: 1) the online echostate Gaussian process (OESGP) and 2) its infinite variant, the online infinite echostate Gaussian process (OIESGP) Both algorithms are iterative fixed-budget methods that learn from noisy time series. In particular, the OESGP combines the echo-state network with Bayesian online learning for Gaussian processes. Extending this to infinite reservoirs yields the OIESGP, which uses a novel recursive kernel with automatic relevance determination that enables spatial and temporal feature weighting. When fused with stochastic natural gradient descent, the kernel hyperparameters are iteratively adapted to better model the target system. Furthermore, insights into the underlying system can be gleamed from inspection of the resulting hyperparameters. Experiments on noisy benchmark problems (one-step prediction and system identification) demonstrate that our methods yield high accuracies relative to state-of-the-art methods, and standard kernels with sliding windows, particularly on problems with irrelevant dimensions. In addition, we describe two case studies in robotic learning-by-demonstration involving the Nao humanoid robot and the Assistive Robot Transport for Youngsters (ARTY) smart wheelchair.
  • CONFERENCE PAPER
    Zambelli M, Demiris Y, 2015,

    Online Ensemble Learning of Sensorimotor Contingencies

    , Workshop on Sensorimotor Contingencies For Robotics at IROS

    Forward models play a key role in cognitive agents by providing predictions of the sensory consequences of motor commands, also known as sensorimotor contingencies (SMCs). In continuously evolving environments, the ability to anticipate is fundamental in distinguishing cognitive from reactive agents, and it is particularly relevant for autonomous robots, that must be able to adapt their models in an online manner. Online learning skills, high accuracy of the forward models and multiple-step-ahead predictions are needed to enhance the robots’ anticipation capabilities. We propose an online heterogeneous ensemble learning method for building accurate forward models of SMCs relating motor commands to effects in robots’ sensorimotor system, in particular considering proprioception and vision. Our method achieves up to 98% higher accuracy both in short and long term predictions, compared to single predictors and other online and offline homogeneous ensembles. This method is validated on two different humanoid robots, namely the iCub and the Baxter.
  • JOURNAL ARTICLE
    Demiris Y, Aziz-Zadeh L, Bonaiuto J, Demiris Y, Aziz-Zadeh L, Bonaiuto J, Demiris Y, Aziz-Zadeh L, Bonaiuto J, Demiris Y, Aziz-Zadeh L, Bonaiuto J, Demiris Y, Aziz-Zadeh L, Bonaiuto Jet al., 2014,

    Information Processing in the Mirror Neuron System in Primates and Machines

    , Neuroinformatics, Vol: 12, Pages: 63-91, ISSN: 1539-2791

    The mirror neuron system in primates matches observations of actions with the motor representations used for their execution, and is a topic of intense research and debate in biological and computational disciplines. In robotics, models of this system have been used for enabling robots to imitate and learn how to perform tasks from human demonstrations. Yet, existing computational and robotic models of these systems are found in multiple levels of description, and although some models offer plausible explanations and testable predictions, the difference in the granularity of the experimental setups, methodologies, computational structures and selected modeled data make principled meta-analyses, common in other fields, difficult. In this paper, we adopt an interdisciplinary approach, using the BODB integrated environment in order to bring together several different but complementary computational models, by functionally decomposing them into brain operating principles (BOPs) which each capture a limited subset of the model’s functionality. We then explore links from these BOPs to neuroimaging and neurophysiological data in order to pinpoint complementary and conflicting explanations and compare predictions against selected sets of neurobiological data. The results of this comparison are used to interpret mirror system neuroimaging results in terms of neural network activity, evaluate the biological plausibility of mirror system models, and suggest new experiments that can shed light on the neural basis of mirror systems.
  • JOURNAL ARTICLE
    Ros R, Baroni I, Demiris Y, Ros R, Baroni I, Demiris Y, Ros R, Baroni I, Demiris Y, Ros R, Demiris Y, Baroni I, Ros R, Baroni I, Demiris Yet al., 2014,

    Adaptive human-robot interaction in sensorimotor task instruction: From human to robot dance tutors

    , Robotics and Autonomous Systems, Vol: 62, Pages: 707-720, ISSN: 1872-793X

    We explore the potential for humanoid robots to interact with children in a dance activity. In this context, the robot plays the role of an instructor to guide the child through several dance moves to learn a dance phrase. We participated in 30 dance sessions in schools to study human–human interaction between children and a human dance teacher, and to identify the applied methodologies. Based on the strategies observed, both social and task-dependent, we implemented a robotic system capable of autonomously instructing dance sequences to children while displaying basic social cues to engage the child in the task. Experiments were performed in a hospital with the Nao robot interacting with 12 children through multiple encounters, when possible (18 sessions, 236 min). Observational analysis through video recordings and survey evaluations were used to assess the quality of interaction. Moreover, we introduce an involvement measure based on the aggregation of observed behavioral cues to assess the level of interest in the interaction through time. The analysis revealed high levels of involvement, while highlighting the need for further research into social engagement and adaptation with robots over repeated sessions.
  • CONFERENCE PAPER
    Ros R, Coninx A, Demiris Y, Patsis G, Enescu V, Sahli H, Ros R, Coninx A, Demiris Y, Patsis G, Enescu V, Sahli Het al., 2014,

    Behavioral Accommodation towards a Dance Robot Tutor

    , International Conference on Human-Robot Interaction, Publisher: ACM/IEEE, Pages: 278-279

    We report first results on children adaptive behavior towards a dance tutoring robot. We can observe that children behavior rapidly evolves through few sessions in order to accommodate with the robotic tutor rhythm and instructions.
  • JOURNAL ARTICLE
    Soh H, Demiris Y, Soh H, Demiris Y, Soh H, Demiris Y, Soh H, Demiris Y, Soh H, Demiris Yet al., 2014,

    Incrementally Learning Objects by Touch: Online Discriminative and Generative Models for Tactile-Based Recognition

    , IEEE Transactions on Haptics, Vol: 7, Pages: 512-525, ISSN: 1939-1412

    Human beings not only possess the remarkable ability to distinguish objects through tactile feedback but are further able to improve upon recognition competence through experience. In this work, we explore tactile-based object recognition with learners capable of incremental learning. Using the sparse online infinite Echo-State Gaussian process (OIESGP), we propose and compare two novel discriminative and generative tactile learners that produce probability distributions over objects during object grasping/ palpation. To enable iterative improvement, our online methods incorporate training samples as they become available. We also describe incremental unsupervised learning mechanisms, based on novelty scores and extreme value theory, when teacher labels are not available. We present experimental results for both supervised and unsupervised learning tasks using the iCub humanoid, with tactile sensors on its five-fingered anthropomorphic hand, and 10 different object classes. Our classifiers perform comparably to state-of-the-art methods (C4.5 and SVM classifiers) and findings indicate that tactile signals are highly relevant for making accurate object classifications. We also show that accurate “early” classifications are possible using only 20-30 percent of the grasp sequence. For unsupervised learning, our methods generate high quality clusterings relative to the widely-used sequential k-means and self-organising map (SOM), and we present analyses into the differences between the approaches.
  • CONFERENCE PAPER
    Su Y, Dong W, Wu Y, Du Z, Demiris Y, Su Y, Dong W, Wu Y, Du Z, Demiris Yet al., 2014,

    Increasing the Accuracy and the Repeatability of Position Control for Micromanipulations Using Heteroscedastic Gaussian Processes

    , IEEE International Conference on Robotics and Automation (ICRA), Publisher: IEEE, Pages: 4692-4698, ISSN: 1050-4729

    Many recent studies describe micromanipulation systems by using complex Analytic Forward Models (AFM), but such models are difficult to build and incapable of describing unmodelable factors, such as manufacturing defects. In this work, we propose the Enhanced Analytic Forward Model (EAFM), an integrated model of the AFM and the Heteroscedastic Gaussian Processes (HGP). The EAFM can compensate the shortfalls of the AFM by training the HGP on the residual of the AFM. This also allows the HGP to learn the repeatability of the micromanipulation system. Based on the EAFM, we further contribute an optimal position controller for improving the accuracy and the repeatability. This optimal EAFM controller is implemented and tested on a three degree-of-freedom micromanipulator based micromanipulation system. Two sets of real-world experiments are carried out to verify our method. The results demonstrate that the controller using EAFM can statistically achieve higher accuracy and repeatability than solely using the AFM.
  • JOURNAL ARTICLE
    Wu Y, Su Y, Demiris Y, Wu Y, Su Y, Demiris Y, Wu Y, Su Y, Demiris Yet al., 2014,

    A morphable template framework for robot learning by demonstration: Integrating one-shot and incremental learning approaches

    , Robotics and Autonomous Systems, Vol: 62, Pages: 1517-1530, ISSN: 0921-8890

    Robot learning by demonstration is key to bringing robots into daily social environments to interact with and learn from human and other agents. However, teaching a robot to acquire new knowledge is a tedious and repetitive process and often restrictive to a specific setup of the environment. We propose a template-based learning framework for robot learning by demonstration to address both generalisation and adaptability. This novel framework is based upon a one-shot learning model integrated with spectral clustering and an online learning model to learn and adapt actions in similar scenarios. A set of statistical experiments is used to benchmark the framework components and shows that this approach requires no extensive training for generalisation and can adapt to environmental changes flexibly. Two real-world applications of an iCub humanoid robot playing the tic-tac-toe game and soldering a circuit board are used to demonstrate the relative merits of the framework.
  • JOURNAL ARTICLE
    Belpaeme T, Baxter PE, Read R, Wood R, Cuayáhuitl H, Kiefer B, Racioppa S, Kruijff-Korbayová I, Athanasopoulos G, Enescu V, Looije R, Neerincx M, Demiris Y, Ros-Espinoza R, Beck A, Cañamero L, Hiolle A, Lewis M, Baroni I, Nalin M, Cosi P, Paci G, Tesser F, Sommavilla G, Humbert R, Belpaeme T, Baxter PE, Read R, Wood R, Cuayáhuitl H, Kiefer B, Racioppa S, Kruijff-Korbayová I, Athanasopoulos G, Enescu V, Looije R, Neerincx M, Demiris Y, Ros-Espinoza R, Beck A, Cañamero L, Hiolle A, Lewis M, Baroni I, Nalin M, Cosi P, Paci G, Tesser F, Sommavilla G, Humbert Ret al., 2013,

    Multimodal Child-Robot Interaction: Building Social Bonds

    , Journal of Human-Robot Interaction, Vol: 1, Pages: 33-53

    For robots to interact effectively with human users they must be capable of coordinated, timely behavior in response to social context. The Adaptive Strategies for Sustainable Long-Term Social Interaction (ALIZ-E) project focuses on the design of long-term, adaptive social interaction between robots and child users in real-world settings. In this paper, we report on the iterative approach taken to scientific and technical developments toward this goal: advancing individual technical competen- cies and integrating them to form an autonomous robotic system for evaluation “in the wild.” The first evaluation iterations have shown the potential of this methodology in terms of adaptation of the robot to the interactant and the resulting influences on engagement. This sets the foundation for an ongoing research program that seeks to develop technologies for social robot companions.
  • JOURNAL ARTICLE
    Chatzis S, Demiris Y, Chatzis SP, Demiris Y, Chatzis SP, Demiris Y, Chatzis SP, Demiris Y, Chatzis SP, Demiris Yet al., 2013,

    The Infinite-Order Conditional Random Field Model for Sequential Data Modeling

    , IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol: 6, Pages: 1523-1534, ISSN: 0162-8828

    Sequential data labeling is a fundamental task in machine learning applications, with speech and natural language processing, activity recognition in video sequences, and biomedical data analysis being characteristic examples, to name just a few. The conditional random field (CRF), a log-linear model representing the conditional distribution of the observation labels, is one of the most successful approaches for sequential data labeling and classification, and has lately received significant attention in machine learning as it achieves superb prediction performance in a variety of scenarios. Nevertheless, existing CRF formulations can capture only one- or few-timestep interactions and neglect higher order dependences, which are potentially useful in many real-life sequential data modeling applications. To resolve these issues, in this paper we introduce a novel CRF formulation, based on the postulation of an energy function which entails infinitely long time-dependences between the modeled data. Building blocks of our novel approach are: 1) the sequence memoizer (SM), a recently proposed nonparametric Bayesian approach for modeling label sequences with infinitely long time dependences, and 2) a mean-field-like approximation of the model marginal likelihood, which allows for the derivation of computationally efficient inference algorithms for our model. The efficacy of the so-obtained infinite-order CRF model is experimentally demonstrated.
  • CONFERENCE PAPER
    Korkinof D, Demiris Y, 2013,

    Online Quantum Mixture Regression for Trajectory Learning by Demonstration

    , International Conference on Intelligent Systems and Robots (IROS), Publisher: IEEE, Pages: 3222-3229, ISSN: 2153-0858

    In this work, we present the online Quantum Mixture Model (oQMM), which combines the merits of quantum mechanics and stochastic optimization. More specifically it allows for quantum effects on the mixture states, which in turn become a superposition of conventional mixture states. We propose an efficient stochastic online learning algorithm based on the online Expectation Maximization (EM), as well as a generation and decay scheme for model components. Our method is suitable for complex robotic applications, where data is abundant or where we wish to iteratively refine our model and conduct predictions during the course of learning. With a synthetic example, we show that the algorithm can achieve higher numerical stability. We also empirically demonstrate the efficacy of our method in well-known regression benchmark datasets. Under a trajectory Learning by Demonstration setting we employ a multi-shot learning application in joint angle space, where we observe higher quality of learning and reproduction. We compare against popular and well-established methods, widely adopted across the robotics community.
  • JOURNAL ARTICLE
    Lee K, Su Y, Kim T-K, Demiris Y, Lee K, Su Y, Kim TK, Demiris Y, Lee K, Su Y, Kim T-K, Demiris Y, Lee K, Su Y, Kim T-K, Demiris Y, Lee K, Su Y, Kim T-K, Demiris Yet al., 2013,

    A syntactic approach to robot imitation learning using probabilistic activity grammars

    , ROBOTICS AND AUTONOMOUS SYSTEMS, Vol: 61, Pages: 1323-1334, ISSN: 0921-8890

    This paper describes a syntactic approach to imitation learning that captures important task structures in the form of probabilistic activity grammars from a reasonably small number of samples under noisy conditions. We show that these learned grammars can be recursively applied to help recognize unforeseen, more complicated tasks that share underlying structures. The grammars enforce an observation to be consistent with the previously observed behaviors which can correct unexpected, out-of-context actions due to errors of the observer and/or demonstrator. To achieve this goal, our method (1) actively searches for frequently occurring action symbols that are subsets of input samples to uncover the hierarchical structure of the demonstration, and (2) considers the uncertainties of input symbols due to imperfect low-level detectors. We evaluate the proposed method using both synthetic data and two sets of real-world humanoid robot experiments. In our Towers of Hanoi experiment, the robot learns the important constraints of the puzzle after observing demonstrators solving it. In our Dance Imitation experiment, the robot learns 3 types of dances from human demonstrations. The results suggest that under reasonable amount of noise, our method is capable of capturing the reusable task structures and generalizing them to cope with recursions. © 2013 Elsevier B.V. All rights reserved.
  • JOURNAL ARTICLE
    Ognibene D, Chinellato E, Sarabia M, Demiris Y, Ognibene D, Chinellato E, Sarabia M, Demiris Y, Ognibene D, Chinellato E, Sarabia M, Demiris Y, Ognibene D, Chinellato E, Sarabia M, Demiris Y, Ognibene D, Chinellato E, Sarabia M, Demiris Y, Ognibene D, Chinellato E, Sarabia M, Demiris Yet al., 2013,

    Contextual action recognition and target localization with an active allocation of attention on a humanoid robot

    , Bioinspiration & Biomimetics, Vol: 8, Pages: 035002-035002, ISSN: 1748-3182

    Exploratory gaze movements are fundamental for gathering the most relevant information regarding the partner during social interactions. Inspired by the cognitive mechanisms underlying human social behaviour, we have designed and implemented a system for a dynamic attention allocation which is able to actively control gaze movements during a visual action recognition task exploiting its own action execution predictions. Our humanoid robot is able, during the observation of a partner's reaching movement, to contextually estimate the goal position of the partner's hand and the location in space of the candidate targets. This is done while actively gazing around the environment, with the purpose of optimizing the gathering of information relevant for the task. Experimental results on a simulated environment show that active gaze control, based on the internal simulation of actions, provides a relevant advantage with respect to other action perception approaches, both in terms of estimation precision and of time required to recognize an action. Moreover, our model reproduces and extends some experimental results on human attention during an action perception.
  • CONFERENCE PAPER
    Ognibene D, Demiris Y, 2013,

    Towards Active Event Recognition

    , International Joint Conference on Artificial Intelligence (IJCAI), Publisher: AIII Press, Pages: 2495-2501

    Directing robot attention to recognise activities and to anticipate events like goal-directed actions is a crucial skill for human-robot interaction. Unfortunately, issues like intrinsic time constraints, the spatially distributed nature of the entailed information sources, and the existence of a multitude of unobservable states affecting the system, like latent intentions, have long rendered achievement of such skills a rather elusive goal. The problem tests the limits of current attention control systems. It requires an integrated solution for tracking, exploration and recognition, which traditionally have been seen as separate problems in active vision.We propose a probabilistic generative framework based on a mixture of Kalman filters and information gain maximisation that uses predictions in both recognition and attention-control. This framework can efficiently use the observations of one element in a dynamic environment to provide information on other elements, and consequently enables guided exploration.Interestingly, the sensors-control policy, directly derived from first principles, represents the intuitive trade-off between finding the most discriminative clues and maintaining overall awareness.Experiments on a simulated humanoid robot observing a human executing goal-oriented actions demonstrated improvement on recognition time and precision over baseline systems.
  • JOURNAL ARTICLE
    Petit M, Lallée S, Boucher J-D, Pointeau G, Cheminade P, Ognibene D, Chinellato E, Pattacini U, Gori I, Martinez-Hernandez U, Barron-Gonzalez H, Inderbitzin M, Luvizotto A, Vouloutsi V, Demiris Y, Metta G, Dominey PF, Petit M, Lallee S, Boucher J-D, Pointeau G, Cheminade P, Ognibene D, Chinellato E, Pattacini U, Gori I, Martinez-Hernandez U, Barron-Gonzalez H, Inderbitzin M, Luvizotto A, Vouloutsi V, Demiris Y, Metta G, Dominey PF, Petit M, Lallee S, Boucher J-D, Pointeau G, Cheminade P, Ognibene D, Chinellato E, Pattacini U, Gori I, Martinez-Hernandez U, Barron-Gonzalez H, Inderbitzin M, Luvizotto A, Vouloutsi V, Demiris Y, Metta G, Dominey PF, Petit M, Lallée S, Boucher J-D, Pointeau G, Cheminade P, Ognibene D, Chinellato E, Pattacini U, Gori I, Martinez-Hernandez U, Barron-Gonzalez H, Inderbitzin M, Luvizotto AL, Vouloutsi V, Demiris Y, Metta G, Dominey PFet al., 2013,

    The Coordinating Role of Language in Real-Time Multi-Modal Learning of Cooperative Tasks

    , IEEE Transactions on Autonomous Mental Development, Vol: 5, Pages: 3-17, ISSN: 1943-0604

    One of the defining characteristics of human cognition is our outstanding capacity to cooperate. A central requirement for cooperation is the ability to establish a “shared plan”-which defines the interlaced actions of the two cooperating agents-in real time, and even to negotiate this shared plan during its execution. In the current research we identify the requirements for cooperation, extending our earlier work in this area. These requirements include the ability to negotiate a shared plan using spoken language, to learn new component actions within that plan, based on visual observation and kinesthetic demonstration, and finally to coordinate all of these functions in real time. We present a cognitive system that implements these requirements, and demonstrate the system's ability to allow a Nao humanoid robot to learn a nontrivial cooperative task in real-time. We further provide a concrete demonstration of how the real-time learning capability can be easily deployed on a different platform, in this case the iCub humanoid. The results are considered in the context of how the development of language in the human infant provides a powerful lever in the development of cooperative plans from lower-level sensorimotor capabilities.
  • CONFERENCE PAPER
    Ros R, Demiris Y, Ros R, Demiris Y, Ros R, Demiris Y, Ros R, Demiris Yet al., 2013,

    Creative Dance: An Approach for Social Interaction between Robots and Children

    , 4th International Workshop on Human Behavior Understanding (HBU), Publisher: Springer, Pages: 40-51, ISSN: 0302-9743

    In this paper we discuss the potential of using a dance robot tutor with children in the context of creative dance to study child-robot interaction through several encounters. We have taken part of dance sessions in order to extract strategies and models to inspire and justify the design of a robot dance tutor. Moreover, we present implementation details and preliminary results on a pilot study to extract initial feedback to further improve and test our system with a broader children population.
  • CONFERENCE PAPER
    Sarabia M, Demiris Y, Sarabia M, Demiris Y, Sarabia M, Demiris Yet al., 2013,

    A Humanoid Robot Companion for Wheelchair Users

    , International Conference on Social Robotics (ICSR), Publisher: Springer, Pages: 432-441, ISSN: 0302-9743

    In this paper we integrate a humanoid robot with a powered wheelchair with the aim of lowering the cognitive requirements needed for powered mobility. We propose two roles for this companion: pointing out obstacles and giving directions. We show that children enjoyed driving with the humanoid companion by their side during a field-trial in an uncontrolled environment. Moreover, we present the results of a driving experiment for adults where the companion acted as a driving aid and conclude that participants preferred the humanoid companion to a simulated companion. Our results suggest that people will welcome a humanoid companion for their wheelchairs.
  • CONFERENCE PAPER
    Sarabia M, Le Mau T, Soh H, Naruse S, Poon C, Liao Z, Tan KC, Lai ZJ, Demiris Y, Sarabia M, Mau TL, Soh H, Naruse S, Poon C, Liao Z, Tan KC, Lai ZJ, Demiris Y, Sarabia M, Mau TL, Soh H, Naruse S, Poon C, Liao Z, Tan KC, Lai ZJ, Demiris Yet al., 2013,

    iCharibot : Design and Field Trials of a Fundraising Robot

    , International Conference on Social Robotics (ICSR 2013), Publisher: Springer, Pages: 412-421, ISSN: 0302-9743

    In this work, we address the problem of increasing charitable donations through a novel, engaging fundraising robot: the Imperial Charity Robot (iCharibot). To better understand how to engage passers-by, we conducted a field trial in outdoor locations at a busy area in London, spread across 9 sessions of 40 minutes each. During our experiments, iCharibot attracted 679 people and engaged with 386 individuals. Our results show that interactivity led to longer user engagement with the robot. Our data further suggests both saliency and interactivity led to an increase in the total donation amount. These findings should prove useful for future design of robotic fundraisers in particular and for social robots in general.
  • CONFERENCE PAPER
    Soh H, Demiris Y, Soh H, Demiris Yet al., 2013,

    When and how to help: An iterative probabilistic model for learning assistance by demonstration

    , International Conference on Intelligent Robots and Systems (IROS), Publisher: IEEE, Pages: 3230-3236, ISSN: 2153-0858

    Crafting a proper assistance policy is a difficult endeavour but essential for the development of robotic assistants. Indeed, assistance is a complex issue that depends not only on the task-at-hand, but also on the state of the user, environment and competing objectives. As a way forward, this paper proposes learning the task of assistance through observation; an approach we term Learning Assistance by Demonstration (LAD). Our methodology is a subclass of Learning-by-Demonstration (LbD), yet directly addresses difficult issues associated with proper assistance such as when and how to appropriately assist. To learn assistive policies, we develop a probabilistic model that explicitly captures these elements and provide efficient, online, training methods. Experimental results on smart mobility assistance — using both simulation and a real-world smart wheelchair platform — demonstrate the effectiveness of our approach; the LAD model quickly learns when to assist (achieving an AUC score of 0.95 after only one demonstration) and improves with additional examples. Results show that this translates into better task-performance; our LAD-enabled smart wheelchair improved participant driving performance (measured in lap seconds) by 20.6s (a speedup of 137%), after a single teacher demonstration.
  • JOURNAL ARTICLE
    Carlson T, Demiris Y, Carlson T, Demiris Y, Carlson T, Demiris Y, Carlson T, Demiris Y, Carlson T, Demiris Y, Carlson T, Demiris Yet al., 2012,

    Collaborative Control of a Robotic Wheelchair: Evaluation of Performance, Attention and Workload

    , IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics, Vol: 42, Pages: 876-888, ISSN: 1083-4419

    Powered wheelchair users often struggle to drive safely and effectively and, in more critical cases, can only get around when accompanied by an assistant. To address these issues, we propose a collaborative control mechanism that assists users as and when they require help. The system uses a multiple-hypothesis method to predict the driver's intentions and, if necessary, adjusts the control signals to achieve the desired goal safely. The main emphasis of this paper is on a comprehensive evaluation, where we not only look at the system performance but also, perhaps more importantly, characterize the user performance in an experiment that combines eye tracking with a secondary task. Without assistance, participants experienced multiple collisions while driving around the predefined route. Conversely, when they were assisted by the collaborative controller, not only did they drive more safely but also they were able to pay less attention to their driving, resulting in a reduced cognitive workload. We discuss the importance of these results and their implications for other applications of shared control, such as brain-machine interfaces, where it could be used to compensate for both the low frequency and the low resolution of the user input.
  • JOURNAL ARTICLE
    Chatzis SP, Demiris Y, Chatzis SP, Demiris Y, Chatzis S, Demiris Y, Chatzis S, Demiris Y, Chatzis SP, Demiris Yet al., 2012,

    Nonparametric mixtures of Gaussian processes with power-law behavior

    , IEEE Transactions on Neural Networks, Vol: 23, Pages: 1862-1871, ISSN: 2162-237X

    Gaussian processes (GPs) constitute one of the most important Bayesian machine learning approaches, based on a particularly effective method for placing a prior distribution overthe space of regression functions. Several researchers have considered postulating mixtures of Gaussian processes as a means ofdealing with non-stationary covariance functions, discontinuities, multi-modality, and overlapping output signals. In existing works, mixtures of Gaussian processes are based on the introduction of a gating function defined over the space of model input variables. This way, each postulated mixture component Gaussian process is effectively restricted in a limited subset of the input space. In this work, we follow a different approach: We consider a fully generative nonparametric Bayesian model with power-law behavior, generating Gaussian processes over the whole input space of the learned task. We provide an efficient algorithm for model inference, based on the variational Bayesian framework, and exhibit its efficacy using benchmark and real-world datasets.
  • JOURNAL ARTICLE
    Chatzis SP, Demiris Y, Chatzis SP, Demiris Y, Chatzis SP, Demiris Y, Chatzis SP, Demiris Yet al., 2012,

    The copula echo state network

    , Pattern Recognition, Vol: 45, Pages: 570-577, ISSN: 0031-3203

    Echo state networks (ESNs) constitute a novel approach to recurrent neural network (RNN) training, with an RNN (the reservoir) being generated randomly, and only a readout being trained using a simple, computationally efficient algorithm. ESNs have greatly facilitated the practical application of RNNs, outperforming classical approaches on a number of benchmark tasks. This paper studies the formulation of a class of copula-based semiparametric models for sequential data modeling, characterized by nonparametric marginal distributions modeled by postulating suitable echo state networks, and parametric copula functions that help capture all the scale-free temporal dependence of the modeled processes. We provide a simple algorithm for the data-driven estimation of the marginal distribution and the copula parameters of our model under the maximum-likelihood framework. We exhibit the merits of our approach by considering a number of applications; as we show, our method offers a significant enhancement in the dynamical data modeling capabilities of ESNs, without significant compromises in the algorithm's computational efficiency.
  • JOURNAL ARTICLE
    Chatzis SP, Demiris Y, Chatzis SP, Demiris Y, Chatzis SP, Demiris Y, Chatzis SP, Demiris Yet al., 2012,

    A Sparse Nonparametric Hierarchical Bayesian Approach Towards Inductive Transfer for Preference Modeling

    , Expert Systems with Applications, Vol: 39, Pages: 7235-7246, ISSN: 0957-4174

    In this paper, we present a novel methodology for preference learning based on the concept of inductive transfer. Specifically, we introduce a nonparametric hierarchical Bayesian multitask learning approach, based on the notion that human subjects may cluster together forming groups of individuals with similar preference rationale (but not identical preferences). Our approach is facilitated by the utilization of a Dirichlet process prior, which allows for the automatic inference of the most appropriate number of subject groups (clusters), as well as the employment of the automatic relevance determination (ARD) mechanism, giving rise to a sparse nature for our model, which significantly enhances its computational efficiency. We explore the efficacy of our novel approach by applying it to both a synthetic experiment and a real-world music recommendation application. As we show, our approach offers a significant enhancement in the effectiveness of knowledge transfer in statistical preference learning applications, being capable of correctly inferring the actual number of human subject groups in a modeled dataset, and limiting knowledge transfer only to subjects belonging to the same group (wherein knowledge transferability is more likely).
  • JOURNAL ARTICLE
    Chatzis SP, Demiris Y, Chatzis SP, Demiris Y, Chatzis SP, Demiris Y, Chatzis SP, Demiris Yet al., 2012,

    A Reservoir-Driven Non-Stationary Hidden Markov Model

    , Pattern Recognition, Vol: 45, Pages: 3985-3996, ISSN: 0031-3203

    In this work, we propose a novel approach towards sequential data modeling that leverages the strengths of hidden Markov models and echo-state networks (ESNs) in the context of non-parametric Bayesian inference approaches. We introduce a non-stationary hidden Markov model, the time-dependent state transition probabilities of which are driven by a high-dimensional signal that encodes the whole history of the modeled observations, namely the state vector of a postulated observations-driven ESN reservoir. We derive an efficient inference algorithm for our model under the variational Bayesian paradigm, and we examine the efficacy of our approach considering a number of sequential data modeling applications.

This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.

Request URL: http://wlsprd.imperial.ac.uk:80/respub/WEB-INF/jsp/search-t4-html.jsp Request URI: /respub/WEB-INF/jsp/search-t4-html.jsp Query String: id=559&limit=50&respub-action=search.html Current Millis: 1503270025902 Current Time: Mon Aug 21 00:00:25 BST 2017