Hosted at Imperial College London by I-X/Digital Foundry, this talk by Lukas Schäfer, Researcher at Microsoft Research (Cambridge),presents recent research on decision-making in modern video games conducted by the People-Centric AI team at Microsoft Research Cambridge. After motivating why video games are compelling testbeds for studying decision-making, Lukas will present work examining how the choice of visual encoders affects the performance and training efficiency of behaviour cloning (BC) agents. His experiments show that carefully selected pre-trained visual encoders can significantly reduce computational cost and boost performance. Given the substantial data requirements of BC for complex tasks, they then investigate predictive inverse dynamics models (PIDM)—which condition a policy on predicted future states—as an alternative to BC. These models have demonstrated improved performance over BC but remain poorly understood. Lukas will present theoretical insights that show that PIDM’s performance gains can be explained with a bias-variance trade-off: conditioning the policy on future context can reduce uncertainty about action predictions but also introduce bias whenever future predictions are inaccurate. He will further show that these insights translate into significant sample efficiency gains in 2D navigation tasks and complex 3D environments in modern video games. Finally, he will move from decision-making models that model the future to world and human action models (WHAM), which combine an environment model (world model) with an imitation-learning policy representing human gameplay. Inspired by the recipe behind LLMs, he will demonstrate the promise of scale for such models and explore how they can support workflows for video game creatives.
Bio:
Lukas Schäfer is a postdoctoral researcher at Microsoft Research in Cambridge, UK, where he is part of Katja Hofmann’s team working on machine learning for video games. His work focuses on developing autonomous agents that enable novel experiences and tools in video games, with an emphasis on imitation learning approaches. Lukas holds a PhD and MSc in Informatics from the University of Edinburgh and a BSc in Computer Science from Saarland University. His PhD research explored how to make deep reinforcement learning more efficient, with a particular focus on multi-agent systems requiring cooperation among multiple agents. Lukas is co-author of the introductory textbook “Multi-Agent Reinforcement Learning: Foundations and Modern Approaches” (https://marl-book.com/), published by MIT Press. His work has appeared in leading ML conferences, including NeurIPS, and AAMAS, and has been supported by the German Academic Exchange Service (DAAD) and the Stevenson Exchange Scholarship.
