EO3274 Pattern Recognition, Machine Learning and Data Analysis

The first part of the course starts on Monday 21st March 10:00 - 12:00 at room no U41. Please be there. We will update you.

Schedule: There are two parts in the course. The course will span over Period 4 and 1 in KTH. The first part of the course overlaps with the master level course EQ2341 Pattern Recognition and Machine Learning. See the website of EQ2341 and be aware of the class schedule. The second part of the course is following EQ2415 Machine Learning and Data Science in period 1.

Course Code (Kurskod):   EO3274

Credits (högskolepoäng): 12 hp

Suggested course category: Class II: Fundamental course (Central läskurs)

Examiner (Examinator): Saikat Chatterjee

Abstract (Kort beskrivning): The course will cover fundamentals of pattern recognition and machine learning from basic to advanced topics, and their applications to classical and state-of-art data analysis. The course is mainly intended for PhD students who deal with recognition and learning from natural and man-made data. 

Kursen behandlar grunderna i mönsterigenkänning och maskininlärning. Från grundläggande till avancerade ämnen samt deras tillämpningar till klassisk och nyare data analys. Kursen är tänkt för doktorander som arbetar med automatisk igenkänning och inlärning från naturliga och konstruerade datakällor. 

Keywords:

Pattern recognition, machine learning, data analysis, regression, Bayesian learning, expectation-maximization, Markov models, approximate inference, convex optimization, graphical models, Gaussian process, sparse representations, deep learning, online learning, learning over networks.

Intended Learning Outcomes (Lärandemål): After the successful completion of course, the students should be able to 

1. Identify and formulate recognition, learning and analysis problems given a dataset.
2. Design systems and algorithms. Critically compare the algorithms in a trade-off between complexity and performance. Finally report the results.
3. Implement algorithms through convex optimization and Bayesian learning approaches.
4. Contribute to the frontier research area.

Efter genomgången kurs ska studenterna för godkänt betyg kunna

1. Identifiera och formulera igenkänning och inlärning samt kunna analysera problem givet en viss datamängd.
2. Designa system och algoritmer. Kritiskt kunna jämföra avvägningen mellan komplexitet och prestanda. Kunna sammanställa resultaten i en rapport.
3. Implementera algoritmer med hjälp av konvex optimering och Bayesianska metoder.
4. Bidra till forskning inom området.

Course main content: 

Theoretical content: Bayes minimum risk criterion, maximum likelihood (ML), maximum-a-posteriori (MAP), recognition for sequence of vectors, hidden Markov model (HMM), graphical models, Gaussian process, expectation-maximization (EM), approximate inference, variational Bayes, artificial neural network (ANN), back propagation, vanishing gradient problem, deep learning, restricted Boltzman machines (RBM), sparse representations, dictionary learning, convex optimization, greedy methods, sparse kernel machines – relevance vector machine (RVM) and support vector machine (SVM), graphical models, message passing, approximate message passing, adaptive learning, online learning, learning over networks, doubly stochastic networks, adaptation over networks.

Project content: Multimedia, gene sequence and financial data pre-processing, feature extraction, and machine learning problems.

Course disposition:

24 lectures span over two study periods, 12 tutorials for exercise, 3-4 master tests (sudden short tests and 2 of them mandatory), mid-term written exam of 5 hours (mandatory), 3 given projects (mandatory), 1 research project, presentation of one research paper.

Requirements of final grade (Krav for slutbetyg):

1. Must need to pass the mid-term exam.
2. Must need to perform 3 given projects.
3. Must need to pass 2 master tests.
4. At-least 75% attendance in teaching classes.
5. Satisfactory performance in research project (preferably in own research area). Expectation is to find innovative publishable research result and proper report.
6. Satisfactory performance in paper presentation.

Eligibility: Only for PhD students. Basic knowledge in linear algebra and probability theory

Course literature:

1. Pattern Recognition, Compendium by Arne Leijon and Gustav Henter.
2. Pattern Recognition and Machine Learning, by C.M. Bishop.
3. Deep learning methods and applications, by L. Deng and D, Yu.
4. Adaptation, learning and optimization over networks, by A.H. Sayed.
5. Sparse and redundant representations: from theory to applications in signal and image processing, by M. Elad.
6. Advanced data analysis from an elementary point of view, by C.R. Shalizi.
7. Research paper handouts

Required equipment: Computer access with Matlab and and any standard programming language such as C.

Schedule for teaching class in 2nd Part of the course: