3 Column Software
Code to accompany our Science Robotics paper on "Learning garment manipulation policies toward robot-assisted dressing". Download code.
Action-Conditioned Generation of Bimanual Object Manipulation Sequences
A novel method that receives an action label to generate a motion sequence of a person performing bimanual object manipulation. Published at AAAI 2023. Download Code.
Holo-Spot Unity App and ROS packages (IROS 2022)
This project includes the Unity application and ROS packages needed for controlling Boston Dynamics' Spot using the Microsoft HoloLens 2. Download Code
Using Eye Gaze to Forecast Human Pose in Everyday Pick and Place Actions
A novel method that uses gaze information to predict the object being fixated, as well as the human pose the instant the fixated object is picked or placed. Published at ICRA 2022. Download Code.
Message-Passing Framework for Vision Prediction Stability in Human-Robot Interaction
To stabilise vision prediction results used in HRI, we present a message-passing framework (MPF) that uses the memory of individual modules to correct each other's outputs. The framework includes pose estimation, object detection, hand-object detection, and gaze estimation. Download Code
What is the Patient Looking At? Robust Gaze-Scene Intersection Under Free-Viewing Conditions
For gaze tracking under free-viewing conditions where there is large parallax between the user's head pose and the camera, we propose a fast and robust method to intersect the participant's gaze with the scene. Download Code
Using a Single Input to Forecast Human Pose in Everyday Pick and Place Actions
A method that uses an input from a single timestep to directly forecast the human pose the instant a pick or place action is performed, as well as the time taken to complete the action. Published at ICASSP 2022. Download Code
Multitask Variational Autoencoding of Human-to-Human Handover
A method that directly forecasts the human pose and object orientation in human-to-human handover sequences. Published at IROS 2021. Download Code
Event-based vision model estimation
Entropy minimisation framework for event-based vision model estimation (see Nunes and Demiris ECCV2020). Download EventEMin.
RT-BENE blink estimation
A robust blink estimator based on the RT-GENE dataset (see Cortacero et al. ICCV2019W). Download RT-BENE (includes ROS code and link to the dataset).
RT-GENE gaze estimation
A new robust gaze estimation framework applicable in natural environments with large camera-subject distances (see Fischer et al. ECCV2018). Download RT-GENE (includes ROS code and link to the dataset).
An obstacle avoidance algorithm in C++ that has been adapted for shared control on differential-drive robots (see Zolotas and Demiris, IROS 2019 for description of its use on a robotic wheelchair). Download Shared Control Source (ROS package).
TRACA visual object tracking
A new context-aware correlation filter based tracking framework to achieve both high computational speed and state-of-the-art performance among real-time trackers (see Choi et al. CVPR2018). Download TRACA (includes ROS code).
A set of convenience wrappers related to perception, object manipulation and social interaction for the iCub humanoid robot (see Fischer et al. Frontiers in Robotics and AI 2018). Download iCub-HRI
Attentional Visual Tracking
Visual tracking with an attentional mechanism that chooses a subset of the associated correlation filters for increased robustness and computational efficiency (see Choi et al. CVPR2017). Download Attentional Visual Tracking Source. Please see also our TRACA CVPR2018 paper below.
Markerless Perspective Taking
C++ / Python framework which implements markerless perspective Taking (see Fischer and Demiris ICRA2016). Download Perspective Taking Source
Lifelong Augmentation of Multi-Modal Streaming Autobiographical Memories
C++ framework which implements an Autobiographical Memory geared towards the developmental robotics community (see Petit, Fischer and Demiris TCDS2016). Download Autobiographical Memory Source
Kinematic Structure Correspondences
Matlab package which implements learning correspondences between kinematic structures (see Chang et al. TPAMI). The download includes the dataset. Download Kinematic Structure Correspondences Source
Kinematic Structure Learning
Matlab package which implements unsupervised learning of kinematic structures (see Chang and Demiris CVPR2015). Download Kinematic Structure Learning Source
C++ package which implements offline and online unsupervised learning of kinematic structures based on 3D data (see Nunes and Demiris ICCV2020). Download 3D Kinematic Structure Learning Source
HAMMER is a C++ library to easily instantiate cognitive architectures based on the simulation theory of mind. Download HAMMER
Online Temporal Learning Library
The Online Temporal Learning C++ library implements several memory structures and includes two “high-level” online learning methods. Download OTL
RT-BENE Blink Estimation Dataset
This dataset contains blink labels for the RT-GENE dataset (see Cortacero et al. ICCV2019W). Download RT-BENE dataset (includes source code).
RT-GENE Gaze Estimation Dataset
This dataset contains face and eye images along with the ground truth head pose and eye gaze angles (see Fischer et al. ECCV2018). Download RT-GENE dataset (includes source code).
Kinematic Structure Correspondences Dataset
This dataset contains kinematic structures and their correspondences, see Chang et al. CVPR2016. The download includes the CVPR2016 source code. Download Kinematic Structure Correspondences Dataset
Kinematic Structure Learning Dataset
This dataset contains various kinematic structures which can be used for unsupervised complex kinematic structure learning (see Chang and Demiris CVPR2015). Download Kinematic Structure Dataset
iCub Grasp Dataset
This dataset contains grasping data for nine-everyday objects collected using the iCub humanoid platform. Download Grasp Dataset