Video Enhancement and Analysis: From Content Analysis to Video Stabilization for YouTube
Speaker: Irfan Essa
Abstract:
The talk will describe a variety of efforts undertaken on analysis of video to enhancement and synthesis of video. An overview of the past work on representing and analyzing videos as a stochastic process and
use of this in a form of Video Textures will be provided. Majority of the talk will then focus on the recent effort which resulted in a widely-used video stabilizer (currently implemented on YouTube) and its extensions. This method generates stabilized videos by employing L1-optimal camera paths to remove undesirable motions. We compute camera paths that are optimally partitioned into constant, linear and parabolic segments mimicking the camera motions employed by professional cinematographers. To this end, we propose a linear programming framework to minimize the first, second, and third derivatives of the resulting camera path. Our method allows for video stabilization beyond the conventional filtering that only suppresses high frequency jitter. An additional challenge in videos shot from mobile phones are rolling shutter distortions. We demonstrate a solution based on a novel mixture model of homographies parametrized by scanline blocks to correct these rolling shutter distortions. Our method does not rely on a-priori knowledge of the readout time nor requires prior camera calibration. This work is in collaboration with Matthias Grundmann and Vivek Kwatra at Google.
Speaker’s bio:
Irfan Essa is a Professor in the School of Interactive Computing (iC) of the College of Computing (CoC), and Adjunct Professor in the School of Electrical and Computer Engineering, Georgia Institute of Technology(GA Tech), in Atlanta, Georgia, USA. He works in the areas of Computer Vision,Computer Graphics, Computational Perception, Robotics and Computer Animation, Machine Learning, and Social Computing, with potential impact on Video Analysis and Production (e.g., Computational Photography & Video,Image-based Modeling and Rendering, etc.) Human Computer Interaction, and Artificial Intelligence research. His specific research interests are in Video Analysis & Synthesis, and Activity & Behavior Recognition. He also works in the new area of Computational Journalism. Specifically, he is interested in the analysis, interpretation, authoring, and synthesis (of video), with the goals of building aware environments & supporting healthy living, recognizing & modeling human behaviors, empowering humans to effectively interact with each other, with media & with technologies, and developing dynamic & generative representations of time-varying streams. He has published over 150 scholarly articles in leading journals and conference venues on these topics. He has been awarded numerous awards, including the NSF CAREER award and is currently an IEEE Fellow.