Filippa Bjare: Threshold-like behavior in Large Language Models

Abstract: This paper presents deep neural networks with a focus on large language models and their rapid increase in performance. Many researchers in the field describe these improvements as demonstrating emergent abilities that exhibit threshold-like behavior. However, an alternative interpretation is offered by Schaeffer et al. (NeurIPS, 2023), where emergent abilities are seen to be induced by certain metrics. This paper provides a closer look at their arguments as well as an analogy to random-graphs, which will be rigorously studied in detail.