Over-the-Air Computation for Machine Learning: Model Aggregation via Retransmissions
Time: Fri 2022-11-04 09.30
Location: D31, Lindstedtsvägen 5, Stockholm
Video link: for online defense
Subject area: Electrical Engineering
Doctoral student: Henrik Hellström , Nätverk och systemteknik
Opponent: Professor Kaibin Huang, The University of Hong Kong
Supervisor: Professor Carlo Fischione, Nätverk och systemteknik, ACCESS Linnaeus Centre; Professor Viktória Fodor, Nätverk och systemteknik
With the emerging Internet of Things (IoT) paradigm, more than a billion sensing devices will be collecting an unprecedented amount of data. Simultaneously, the field of data analytics is being revolutionized by modern machine learning (ML) techniques that enable sophisticated processing of massive datasets. Many researchers are envisioning a combination of these two technologies to support exciting applications such as environmental monitoring, Industry 4.0, and vehicular communications. However, traditional wireless communication protocols are inefficient in supporting distributed ML services, where data and computations are distributed over wireless networks. This motivates the need for new wireless communication methods. One such method, over-the-air computation (AirComp), promises to communicate with massive gains in terms of energy, latency, and spectrum efficiency compared to traditional methods.
The expected efficiency of AirComp is due to the complete spectrum sharing for all participating devices. Unlike in traditional physical-layer communications, where interference is avoided by allocating orthogonal communication channels, AirComp promotes interference to compute a function of the individually transmitted messages. However, AirComp can not reconstruct functions perfectly but introduces errors in the process, which harms the convergence rate and region of optimality of ML algorithms. The main objective of this thesis is to develop methods that reduce these errors and analyze their effects on ML performance.
In the first part of this thesis, we consider the general problem of designing wireless methods for ML applications. In particular, we present an extensive survey which divides the field into two broad categories, digital communications and analog over-the-air-computation. Digital communications refers to orthogonal communication schemes that are optimized for ML metrics, such as classification accuracy, privacy, and data-importance, rather than traditional communication metrics such as fairness, data rate, and reliability. Analog over-the-air-computation refers to the AirComp method and its application to distributed ML, where communication-efficiency, function estimation, and privacy are key concerns.
In the second part of this thesis, we focus on the analog over-the-air computation problem. We consider a network setup with multiple devices and a server that can be reached via a single hop, where the wireless channel is modeled as a multiple-access channel with fading and additive noise. Over such a channel, the AirComp function estimate is associated with two types of error: 1) misalignment errors caused by channel fading and 2) noise-induced errors caused by the additive noise. To mitigate these errors, we propose AirComp with retransmissions and develop the optimal power control scheme for such a system. Furthermore, we use optimization theory to derive bounds on the convergence of an AirComp-supported ML system that reveal a relationship between the number of retransmissions and loss of the ML model. Finally, with numerical results we show that retransmissions can significantly improve ML performance, especially for low-SNR scenarios.