Zach Furman: Introduction to singular learning theory

Abstract.

Despite the practical success of deep neural networks, we have little understanding of how or why they actually work, from why these networks generalize to what algorithms they learn from training data. Recently, research has found a possible opening to address this: Sumio Watanabe's singular learning theory, originally conceived over 20 years ago to solve problems in Bayesian statistics. In this talk, I'll explain the basics of singular learning theory: what makes models like neural networks singular, why this helps them generalize, and what this might tell us about the internal structure they learn.