Adaptation and Learning over Networks

Instructor: Prof. Ali H. Sayed (UCLA, USA)

---

News:

• The course begins 18th March, 2014, 13:00-17:00, Room: M38. The course is for one week, comprising four lectures spanning 7.5-8 hours.
• First lecture is about 4 hours. The other three lectures are about 1.5-2 hours.
• Class room address: M38, Brinellvägen 64, M-Building, KTH Main Campus
• Class room address: Wednesday and Friday class at room Q2, Osquldas Väg 10 (where the first class was held). The thursday class will be at Room No B1 (This B building is in front of KTH Hallen). 
• The course material and assignment are online. Using your KTH social account, log in and check the left side menu of this page and click on ''Course Material and Assignment''. The interested people who do not have KTH account (such as students from Uppsala) need to contact with Saikat Chatterjee by an email to get the material.
• The course assignment deadline is extended to Wednesday 26th March evening. Please send as as attachment by email to Saikat (scan it or by latex).

---

Contact person: Saikat Chatterjee (sach@kth.se)

 

About the course

This course is an ACCESS Graduate School course (intensive).

The short course will introduce students to the fundamentals of online adaptation, learning, inference, and distributed optimization over multi-agent adaptive networks. The lectures will highlight how collaboration among agents can lead to superior adaptation and learning performance over graphs. Adaptive networks consist of a collection of agents with learning abilities. The agents interact with each other on a local level and diffuse information across the network to solve inference and optimization tasks in a decentralized manner. Such networks are scalable, robust to node and link failures, and are particularly suitable for learning from big data sets by tapping into the power of collaboration among distributed agents. Still, some surprising phenomena arise when information is processed in a decentralized fashion over networked systems and these deserve closer attention due to the coupling effect among the agents. The course will overview such phenomena in the context of adaptive networks, and consider examples related to distributed sensing, intrusion detection, clustering, and machine learning applications.

• Lecture 1 (Date: 18th March, 2014; time: 13:00-17:00; Room No: M38): Background material. Complex gradients and complex Hessian matrices. Convexity, strict convexity, and strong convexity. Mean-value theorems. Lipschitz conditions. Basic linear algebra results. Single-agent optimization. Steepest-descent algorithms. Stochastic-gradient algorithms. Adaptation and learning. Convergence and stability.
• Lecture 2 (Date: 19th March, 2014; time: 13:00-15:00; Room No: Q2):  Network model and topology. Network objective. Pareto optimality. Non-cooperative strategies. Centralized strategy. Performance comparison.
• Lecture 3 (Date: 20th March, 2014; time: 13:00-15:00; Room No: B1; This B building is in front of KTH Hallen): Distributed strategies: incremental, consensus, and diffusion. Distributed optimization. Online adaptation and learning. Benefits of cooperation. Combination policies. Synchronous versus asynchronous behavior. Sparse solutions.
• Lecture 4 (Date: 21st March, 2014; time: 13:00-15:00; Room No: Q2): Network stability and performance measures. Comparison of consensus and diffusion strategies for adaptation and learning. Mean-square-error metric. Excess-risk metric. Distributed strategies for least-squares and state-space estimation.

General information

• Course credit: 2 points (2 hp)
• Instructor: Prof. Ali H. Sayed
• Instructor's email: sayed@ee.ucla.edu
• Contact: Saikat Chatterjee (sach@kth.se)
• Class Room: M38
• Work load: Total 7.5 hours lecture and assignments. 

 Assessment

The assessment will be based on one assignment given at the beginning of the course and involving several problems covering the material from all five lectures.