Sören Christensen: Part I: Is Learning in Biological Neural Networks based on Stochastic Gradient Descent? An analysis using stochastic processes; Part II: William Feller - From Kiel to Stockholm, Shaping Probability in a Stormy Era

Abstract

Part I: The fundamental differences between how biological and artificial neural networks learn have been a subject of intense research. While artificial networks heavily rely on optimization techniques of Stochastic Gradient Descent (SGD)-type, the biological learning process has often been assumed to operate solely on local information, making SGD seemingly inapplicable. This talk challenges this line of argument by studying a stochastic process model for supervised learning in biological neural networks. Our results show that a process approximating a continuous gradient step emerges through the accumulation of numerous local updates in response to each learning opportunity. This suggests that SGD-like optimization may be a fundamental mechanism underlying learning in biological brains.

Part II: William (Willy) Feller (*July 7, 1906 in Zagreb; †January 14, 1970 in New York City) was a towering figure in 20th century probability theory, and his work has had an influence that can hardly be overestimated. He found his way to probability theory in Kiel before coming to the University of Stockholm (via Copenhagen). His early explorations in Kiel laid essential foundations for the measure-theoretic foundation of the frequentist probability paradigm. This talk will shed light on Feller's formative years, tracing the development of his ideas against the background of a rapidly changing world. We will explore the mathematical innovations that emerged from his time in Kiel, while acknowledging the profound impact of the Nazi era on his life and work.