EQ2845 Information Theory and Source Coding 7.5 credits
Informationsteori och källkodning
The course treats the principles underlying the encoding of speech, audio, video, and images at low bit rates. Source coding techniques such as scalar and vector quantization, orthogonal transforms, and linear prediction are introduced and their performance is analyzed theoretically. The theoretical bounds on the performance of source coders are discussed.
Education cycle
Second cycleMain field of study
Electrical Engineering
Grading scale
A, B, C, D, E, FX, F
Course offerings
Spring 19 for programme students
-
Periods
Spring 19 P3 (7.5 credits)
-
Application code
60811
Start date
15/01/2019
End date
15/03/2019
Language of instruction
English
Campus
KTH Campus
Tutoring time
Daytime
Form of study
Normal
-
Number of places
No limitation
Schedule
Planned timeslots
P3: B1, D1, F1, J1, D2, E2. more info
Course responsible
Markus Flierl <mflierl@kth.se>
Teacher
Markus Flierl <mflierl@kth.se>
Part of programme
- Master's Programme, ICT Innovation, 120 credits, year 1, DMTE, Optional
- Master's Programme, ICT Innovation, 120 credits, year 1, DMTK, Optional
- Master's Programme, Information and Network Engineering, 120 credits, year 1, Recommended
- Master's Programme, Information and Network Engineering, 120 credits, year 1, COE, Recommended
- Master's Programme, Information and Network Engineering, 120 credits, year 1, INF, Recommended
- Master's Programme, Information and Network Engineering, 120 credits, year 1, MMB, Recommended
Spring 20 for programme students
-
Periods
Spring 20 P3 (7.5 credits)
-
Application code
60683
Start date
15/01/2020
End date
14/03/2020
Language of instruction
English
Campus
KTH Campus
Tutoring time
Daytime
Form of study
Normal
-
Number of places
No limitation
Course responsible
Markus Flierl <mflierl@kth.se>
Teacher
Markus Flierl <mflierl@kth.se>
Part of programme
- Master's Programme, ICT Innovation, 120 credits, year 1, VCCN, Optional
- Master's Programme, ICT Innovation, 120 credits, year 1, VCCO, Optional
- Master's Programme, Information and Network Engineering, 120 credits, year 1, Recommended
- Master's Programme, Information and Network Engineering, 120 credits, year 1, COE, Recommended
- Master's Programme, Information and Network Engineering, 120 credits, year 1, INF, Recommended
- Master's Programme, Information and Network Engineering, 120 credits, year 1, MMB, Recommended
Intended learning outcomes
After passing this course, participants should be able to
- describe and use the principles of information theory, like entropy, mutual information, asymptotic equipartition, data processing, prefix codes, Kraft inequality, noiseless source coding, maximum entropy, rate distortion, noisy source coding, Shannon lower bound, backward channel, reverse waterfilling, energy concentration, etc. to develop source coding algorithms,
- develop source coding schemes for lossless coding, like Huffman coding, arithmetic coding, Lempel-Ziv coding, universal source coding,
- develop source coding schemes for lossy coding, like scalar and vector quantization, Lloyd-Max quantization, entropy-constrained quantization, high-rate quantization, transform coding, predictive coding,
- implement (for example with MatLab) and assess the developed source coding schemes / algorithms,
- explain coding design choices using the principles of information theory,
- develop source coding schemes for a given source coding problem,
- model and assess source coding schemes using the principles of information theory,
- analyze given source coding problems, identify and explain the challenges, propose possible solutions, and explain the chosen design.
To achive higher grades, participants should also be able to
- solve more advanced problems in all areas mentioned above.
Course main content
This course introduces the principles of information theory and source coding, discusses fundamental source coding concepts, and provides hands-on experience for selected popular source coding algorithms. The course includes topics on information and entropy, lossless coding, Shannon's noiseless source coding theorem, lossy coding, rate distortion, Shannon's noisy source coding theorem, scalar and vector quantization, transform and predictive coding.
Eligibility
For single course students: 120 credits and documented proficiency in English B or equivalent.
Recommended: EQ1220 Signal Theory or equivalent
Recommended prerequisites
EQ1220 Signal Theory or equivalent.
Literature
T.M. Cover and J.A. Thomas, “Elements of Information Theory,” John Wiley & Sons, Inc., New York.
Examination
- INL1 - Assignment, 1.5, grading scale: P, F
- TEN1 - Exam, 6.0, grading scale: A, B, C, D, E, FX, F
Homework assignments 1.5 ECTS (P/F). Written exam 6 ECTS (A-F).
Offered by
EECS/Intelligent Systems
Contact
Markus Flierl (mflierl@kth.se)
Examiner
Markus Flierl <mflierl@kth.se>
Version
Course syllabus valid from: Spring 2019.
Examination information valid from: Spring 2019.