Abstract:
Intelligent systems, whether natural or artificial, must act in a world that is highly unpredictable. To plan actions with uncertainty is a stochastic optimal control problem. However, there are two fundamental problems: the optimal control solution is intractable to compute and intractable to represent due the non-trivial state dependence of the optimal control. This has prevented large scale application of stochastic optimal control theory sofar. The path integral control theory describes a class of control problems whose solution can be computed as an inference computation. In this presentation we formalize the intuitive notion that the efficiency of the inference computation is related to the proximity of the sampling control to the optimal control. Secondly, we show new results that allow approximate computation of state dependent optimal controls using the cross entropy method. These two ingredients together suggest a novel adaptive sampling procedure, called PICE (path integral cross entropy method), that learns a controller based on self-generated data. |The adaptive sampling procedure can be used to efficiently compute optimal controls but can also be used to accelerate other Monte Carlo computations. We illustrate the results on a few examples in robotics and time series.
Bio:
Bert Kappen studied particle physics in Groningen, the Netherlands and completed his PhD in this field in 1987 at the Rockefeller University in New York. From 1987 until 1989 he worked as a scientist at the Philips Research Laboratories in Eindhoven, the Netherlands. Since 1989, he is conducting research on neural networks at the laboratory for biophysics of the University of Nijmegen, the Netherlands. Since 1997 he is associate professor and since 2004 full professor at this university.His group consists of 10 people and is involved in research on Bayesian machine learning, stochastic control theory, computational neuroscience and several applications in collaboration with industry. His research was awarded in 1997 the prestigious national PIONIER research subsidy. He co-founded in 1998 the company Smart Research, that sells forensic software for DNA matching (www.bonaparte-dvi.com) and Big4Data (big4data.nl), which develops custom made machine learning tools. He is director of the Dutch Foundation for Neural Networks (SNN), which coordinates research on neural networks and machine learning in the Netherlands. Since 2009, he is honorary faculty at UCL’s Gatsby Computational Neuroscience Unit in London.