Course plan

Course literature and content We will follow the book --Objective Along with developing skills and good understanding of the subject, the purpose of the PhD level course will be served if the students are able to come up with new research problems and solutions. The main objective is new research problem formulation, solution and eventual publication. To achieve the goal, all the students are expected to collaborate.¶

¶

Course textbook Title: PEATTERN RECOGNITION AND MACHINE LEARNING by. Author: CHRISTOPHER M. BISHOP. Following the book, wSpringer Publication. ¶

Course content We will coverfollow the following topics. (chapters from the book):

* Introduction to the coursesubject (Chapter 1)
* Necessary probability theory (Chapter 2)
* Linear models for regression (Chapter 3)
* Linear models for classification (Chapter 4)
* Kernel methods (Chapter 6)
* Sparse Kkernel Mmachines (Chapter 7)
* Graphical models (Chapter 8)
* Mixture models and EM (expectation-maximization) (Chapter 9)
* Approximate Inference (Chapter 10)
¶ General information Course credit: 8 points Instructor: Saikat Chatterjee Instructor's office: Osquldas väg 10, floor 3. Instructor's email: sach@kth.se Meeting times: For discussion, mail to Instructor and fix a time Class Room: The list will be given in the 'Schedule' table Work load: 3 hours per lecture and assignment. Prerequisites: Good understanding of probability theory. Teaching and learning methodology: The lectures will be based on blackboard and slides. The style will be more discussion oriented. Students have to present papers and execute projects.¶ Outcome: Good understanding of the topic and eventually some good research problem formulation, solution and publication.¶ Schedule and lecture notes Lecture Time Place Content Slides 1 2011-10-24 10:00 - 12:00 Osquldasväg 6B plan 1 (Q13) Introduction to the Course Slide 1 2 2011-10-31 10:00 - 12:00 Drottning Kristinasväg 30BV (L31) Uncertainty and Uniqueness Slide 2 3 2011-11-07 10:00 - 12:00 Drottning Kristinasväg 30 (L42) Pursuit Algorithms - Design Slide 3 4 2011-11-14 10:00 - 12:00 Osquldasväg 6B plan 1 (Q11) Pursuit Algorithms - Guarantees Slide 4 5 2011-11-21 10:00 - 12:00 Drottning Kristinasväg 30 (L42) From Exact to Approximate Solutions Slide 5 6 2011-11-28 10:00 - 12:00 Osquldasväg 6B plan 2 (Q21) From Exact to Approximate Solutionss Slide 6 7 2011-12-05 10:00 - 12:00 Osquldasväg 6B plan 1 (Q15) Iterative-Shrinkage Algorithm Slide 7 8 2011-12-12 10:00 - 12:00 Osquldasväg 6B plan 1 (Q15) Towards Average Performance Analysis Slide 8 ¶ List of papers for presentation For paper presentation, each group consists of 3-4 persons. Each group has to read two good papers and make a technical note followed by presentation. Please find some papers which you may choose.¶ No Title 1 A Simple Proof of the Restricted Isometry Property for Random Matrices 2 Asymptotic Analysis of MAP Estimation via the Replica Method and Compressed Sensing 3 Decoding by Linear Programming 4 Distributed Basis Pursuit 5 Information Theoretic Bounds for Compressed Sensing 6 Message Passing Algorithms for Compressed Sensing: Part I and II 7 MIMO Radar Using Compressive Sampling 8 Optimally Sparse Representation in General (non-Orthogonal) Dictionaries via L_1 Minimization 9 Performance Analysis for Sparse Support Recovery 10 Regime Change: Bit-Depth versus Measurement-Rate in Compressive Sensing 11 Signal Processing With Compressive Measurements 12 Support Recovery of Sparse Signals 13 Uncertainty Principles and Ideal Atomic Decomposition 14 Beyond Nyquist: Efficient Sampling of Sparse Bandlimited Signals According to interest, you may choose other quality papers. Please inform the paper title to the instructor.¶ Objective: For writing good technical notes, the group members should do intense discussion and exchange of ideas, revolving around the chosen papers. By this way, students are expected to learn from each other.¶ Tasks for technical note writing: (1) Read the paper with most attention. (2) Understand the central issues - what is the research and why and how? (3) If theorems are there, try to relate with reality. (4) You may not need to rework all the hard proofs - but, for your own learning, try to go as far as possible. (5) Then write a good summary of the research work (within 2-3 pages). (6) Present the work so that audience understands the summary. (7) Provide a thought process such that the papers can be useful for your own research.¶ Presentation dates: 15th and 16th December, 2011. If we do not finish within these two days, then we may finish by 17th December (thats a Saturday). I prefer that the presentations should be finished by these dates. However, if somebody wants more time, that can be arranged in the first week of March, 2012. You may present on 5th March and 6th March, 2012.¶ Presentation time limit: Approximately one hour. Ultimately, it depends on the audience interactions.¶ ¶ List of projects For projects, we are interested in hands on learning. The first project will be given by the instructor to the group. The second project is individual where a student has to choose a good paper (mostly dealing with a practical problem) and understand the paper followed by reproducing the results in the paper. The individual paper work has to be presented which will be counted as the individual paper presentation as well as individual project presentation. The overall format is as follows.¶ Type Title Group Instrutor will provide the problem. The problem is to implement several algorithms and compare them. This project will be decided after third lecture. Individual Choose a practical paper. Read it carefully and try to reproduce the experimental / numerical results. Present the work. Project submission and presentation date: 20th December, 2011. I prefer that the presentations should be finished by this date. However, if somebody wants more time, that can be arranged in the first week of March, 2012. You may present on 5th March and 6th March, 2012.¶ Presentation time limit: Approximately 40 minutes. Ultimately, it depends on the audience interactions.¶ The details of the group project assigned by the istructor is here. Download from ProjectAssignedByInstructor ¶ ¶ Ultimate Objective Being a new research area with lots of activities, the purpose of the course will be served if the students are able come up with new research results.inference (Chapter 10)
¶

General information Course credit: 8 points Instructor: Saikat Chatterjee Instructor's office: Osquldas väg 10, floor 3 (Plan 3). Instructor's email: sach@kth.se Meeting times: For discussion, mail to Instructor and fix a time Work load: 3 hours per lecture and assignment. Prerequisites: Good understanding of probability theory. No measure theory knowledge required. Teaching and learning methodology: The lectures will be based on blackboard and slides. The lectures will be more discussion oriented. Students have to present papers and execute projects.¶

¶

Evaluation The evaluation criteria are based on the solving of assignments, presentation of research papers and project assignments.¶

Student feedback: The student feedback will be collected after the course is over and posted here. Student Feedback ¶

Schedule and lecture notes Class room: SIP Conference room; Osquldas Väg 10, Plan 3. Time: 14:00 - 17:00 (Three hours per lecture) A specific request: Please keep the instructor's mobile no: 0738913581. As the class room is inside a locked door, please be in front of the locked door (SIP lab) on time before class starts. We open the door exactly on time at 14:00 and let in all the listeners. In case, if you are late and nobody is there to open the door, call the mobile number of the instructor.¶

¶

Lecture Date Content Slides 1 2013-04-08 Chapter 1 Slide 1 2 2013-04-18 Chapter 2 Slide 2 3 2013-04-30 Chapter 3 Slide 3 4 2013-05-09 Chapter 4 Slide 4 5 2013-05-20 Chapter 4 Slide 5 6 2013-05-30 Chapter 6 Slide 6 7 2013-06-10 Chapter 6,7 Slide 7 8 2013-06-19 Chapter 7 Slide 8 9 2013-07-01 Chapter 8 Slide 9 10 2013-07-10 Chapter 8 Slide 10 11 2013-07-19 Chapter 9 Slide 11 12 2013-07-29 Chapter 9,10 Slide 12 13 2013-08-08 Chapter 10 Slide 13 14 2013-08-19 Paper presentation 15 2013-08-29 Discussion: New research problems Slide 15 ¶

Paper presentation For paper presentation, each group consists of two persons. Each group has to read two good papers and make a technical note followed by presentation. According to interest, students may choose quality papers. Please inform the paper title to the instructor.¶

Objective of technical note: For writing good technical notes, the group members should do intense discussion and exchange of ideas, revolving around the chosen papers. By this way, students are expected to learn from each other.¶

Tasks for technical note writing: (1) Read the paper with most attention. (2) Understand the central issues - what is the research and why and how? (3) You need to rework all the theory. (4) Then write a good summary of the research work (within 2-3 pages). (5) Present the work so that audience understands the summary. (7) Provide a thought process such that the papers can be useful for your own research.¶

Presentation date: 2013-08-19. If we do not finish within that day, then the next day has to be used (that is 2013-08-20). I prefer that the presentations should be finished by these dates. However, if somebody wants more time, that can be arranged in the third week of January, 2014. You may present on 2014-01-13.¶

Presentation time limit: Approximately one hour. Ultimately, it depends on the audience interactions.¶

¶

Projects Objective: For projects, we are interested in hands on learning to implement existing techniques. For example, support vector machine (SVM). We will fix the project topics on the first lecture.¶

Group work: The projects will be excuted by forming groups. A group consists of two persons.¶

Project submission and presentation date: 2013-09-30. I prefer that the all the project presentations should be finished on this date. However, if somebody wants more time, that can be arranged after the third week of January, 2014, on case by case basis.¶

Presentation time limit: Approximately 40 minutes. Ultimately, it depends on the audience interactions.¶

¶

On new research problems Objective: The students are encouraged to formulate new research problems and solutions. The problems may or may not be associated with their main research (thesis) topics.¶

Mode of operation: (1) Throughout the course lectures and learning, the students should try to identify a concrete research problem where the gained knowledge and skill can be used. (2) On the last lecture class (2013-08-29), the students will go to the board or use projector to make the class undertsand about the problem formulation: motivation, the exact setup and a possible solution approach. (3) Then the student will handover a latex code and pdf document with the details. (4) The instructor compiles all the latex codes and form the bigger set of all new research problems. (5) The instructor sends the bigger note to everybody and the students are encouraged to collaborate for solutions. By forming the bigger note, everything is in one place and everybody knows the problems of others.¶

Hope: We will be able to solve some new interesting problems and be awarded with publications.¶