The speaker will hold talks.

Title: Neural Networks for Probabilists (at 2-3pm) 

Abstract: Deep neural networks are a centerpiece of modern machine learning. They are also fascinating probabilistic models, about which much remains unclear. In this pre-talk I will define neural networks, explain how they are used in practice, and give a survey of the big theoretical questions they have raised. If time permits, I will also explain how neural networks are related to a variety of classical areas in probability and mathematical physics, including random matrix theory, optimal transport, and combinatorics of hyperplane arrangements. 

Title: Neural Networks at Finite Width and Large Depth (at 3-4pm) 

Abstract: Deep neural networks are often considered to be complicated “black boxes,” for which a full systematic analysis is not only out of reach but also impossible. In this talk, which is based on ongoing joint work with Sho Yaida and Dan Roberts, I will make the opposite claim. Namely, that deep neural networks with random weights and biases are perturbatively solvable models. Our approach applies to networks at finite width n and large depth L, the regime in which they are used in practice. A key point will be the emergence of a notion of “criticality,” which involves a finetuning of model parameters (weight and bias variances). At criticality, neural networks are particularly well-behaved but still exhibit a tension between large values for n and L, with large values of n tending to make neural networks more like Gaussian processes and large values of L amplifying higher cumulants. Our analysis at initialization has a number of consequences for networks during and after training, which I will discuss if time permits.