BEGIN:VCALENDAR
VERSION:2.0
PRODID:www.imperial.ac.uk
BEGIN:VEVENT
UID:607e302aa32c8
DTSTART:20210209T140000Z
SEQUENCE:0
TRANSP:OPAQUE
DTEND:20210209T160000Z
URL:https://www.imperial.ac.uk/events/128347/boris-hanin-princeton-tba-stoc
hastic-analysis-seminar/
LOCATION:United Kingdom
SUMMARY:Boris Hanin (Princeton): Two talks on Neural Networks (Stochastic A
nalysis Seminar)
CLASS:PUBLIC
DESCRIPTION:The speaker will hold talks.\nTitle: Neural Networks for Proba
bilists (at 2-3pm) \nAbstract: Deep neural networks are a centerpiece o
f modern machine learning. They are also fascinating probabilistic models\
, about which much remains unclear. In this pre-talk I will define neural
networks\, explain how they are used in practice\, and give a survey of th
e big theoretical questions they have raised. If time permits\, I will als
o explain how neural networks are related to a variety of classical areas
in probability and mathematical physics\, including random matrix theory\,
optimal transport\, and combinatorics of hyperplane arrangements. \n\nTi
tle: Neural Networks at Finite Width and Large Depth (at 3-4pm) \nAbstra
ct: Deep neural networks are often considered to be complicated “black
boxes\,” for which a full systematic analysis is not only out of reach b
ut also impossible. In this talk\, which is based on ongoing joint work wi
th Sho Yaida and Dan Roberts\, I will make the opposite claim. Namely\, th
at deep neural networks with random weights and biases are perturbatively
solvable models. Our approach applies to networks at finite width n and la
rge depth L\, the regime in which they are used in practice. A key point w
ill be the emergence of a notion of “criticality\,” which involves a f
inetuning of model parameters (weight and bias variances). At criticality\
, neural networks are particularly well-behaved but still exhibit a tensio
n between large values for n and L\, with large values of n tending to mak
e neural networks more like Gaussian processes and large values of L ampli
fying higher cumulants. Our analysis at initialization has a number of con
sequences for networks during and after training\, which I will discuss if
time permits.
X-ALT-DESC;FMTTYPE=text/html:The speaker will hold talks.

\nTitle:** **Neural Networks for Probab
ilists (at 2-3pm)

\nAbstract: Deep neural networks are a centerpiece of modern
machine learning. They are also fascinating probabilistic models\, about w
hich much remains unclear. In this pre-talk I will define neural networks\
, explain how they are used in practice\, and give a survey of the big the
oretical questions they have raised. If time permits\, I will also explain
how neural networks are related to a variety of classical areas in probab
ility and mathematical physics\, including random matrix theory\, optimal
transport\, and combinatorics of hyperplane arrangements.

\n

\nTitle: Neural
Networks at Finite Width and Large Depth (at 3-4pm)

\nAbstract: Deep neural ne
tworks are often considered to be complicated “black boxes\,” for whic
h a full systematic analysis is not only out of reach but also impossible.
In this talk\, which is based on ongoing joint work with Sho Yaida and Da
n Roberts\, I will make the opposite claim. Namely\, that deep neural netw
orks with random weights and biases are perturbatively solvable models. Ou
r approach applies to networks at finite width n and large depth L\, the r
egime in which they are used in practice. A key point will be the emergenc
e of a notion of “criticality\,” which involves a finetuning of model
parameters (weight and bias variances). At criticality\, neural networks a
re particularly well-behaved but still exhibit a tension between large val
ues for n and L\, with large values of n tending to make neural networks m
ore like Gaussian processes and large values of L amplifying higher cumula
nts. Our analysis at initialization has a number of consequences for netwo
rks during and after training\, which I will discuss if time permits.

DTSTAMP:20210420T013642Z
END:VEVENT
END:VCALENDAR