My principal research interests are focused on :
- 3D information measurement, analysis, synthesis
- Movement Analysis
- Gamification of rehabilitation or learning exercice in virtual or augmented reality environment
- Virtual and Augmented Reality for clinician training and assistive technologies.
I merge this three domains during my principal research works like fall detection at home, gait analysis with one or multiple Kinects.
Virtual and Augmented reality for medical training and intra-operative assistance
Training clinician is a long and difficult process. My research interest is to look at how Virtual Reality technologies could be used to support medical training. With such media, exposure to knowledge could be enhanced as well as training exercise done in a more autonomous way.
Fall detection at home
Fall at home is a serious healthcare problematic. Sometimes, faller can not call emergency services and stay in distress situation. To tackle this problem, we designed a fall detection method using multiple cameras. But furniture could lead to occlusions of the person in camera's viewfield disturbing the fall detection. We used a volumic reconstruction method based on multiple cameras which is resitant to occlusion and a new fall detection feature based on vertical volume repartition versus the shoulder width.
Here is an exemple of volume reconstruction during a fall :
And the corresponding vertical volume repartition during the fall :
The Video Dataset is available at : Fall video dataset
Gait analysis with a Kinect
Walking is a complexe movement which solicit many human body systems like musculo-skeletal and nervous systems. Gait analysis gives precious information for physician to establish a diagnostic. To make such information quantitatively available, we designed a method using a Kinect placed in front of the subject walking on a treadmill to measure gait asymetry. A mean gait cycle model is then computed from many gait cycles recorded (nearly 120). This model permits to bypass measurement noise of the Kinect (1 cm of error) and to absorb the lateral and in depth movement of the subject on the treadmill. Finally it makes possible to measure the difference between the right leg movement versus the left leg one.
Gait analysis with multiple Kinects
The depth camera of the Kinect permits to measure tridimensional surfaces at a low price. It is very interesting to merge measures from multiple depth cameras for gait analysis. We developed a calibration method for depth cameras. This calibration permits the projection of measured surfaces in a common referencial. For body measurement during walking exercice on a treadmill, we showed that reconstruction with this method using 3 kinects was better than Visual Hull reconstruction using more than twenty cameras.
This method is detailed in the article Multiple Depth Cameras Calibration and Body Volume Reconstruction for Gait Analysis and summed up in the following video :
For body measurement during walking exercice on a treadmill, we showed that reconstruction with this method using 3 kinects was better than Visual Hull reconstruction using more than twenty cameras. Here is a reconstruction exemple of a subject walking on a treadmill :
Gamification of rehabilitation or learning exercice
The use of low cost equipments is promising in rehabilitation or training environement. Embedded with gaming platform, they allow the user to interact with his motion. With an adapted gameplay focused on the rehabilitation exercice or task to learning, the subject focus on the game rather than the exercice. The aim is to increase the adherence of the subject and improve the efficience or the exercice. We applied this concept for CP patient rehabilitation with a platform fusing data from a WiiFit and a Kinect. Thanks to these sensors, the rehabilitation motion is analysed in real-time to assess if the motion is done correctly.
The visual feedback of trunk movement in the developed active video game
With Prof Cobb, we develop the use of serious game plaform based on virtual and augmented reality, dedicated for surgery task learning and rehabilitation exercice.