Skip to main content
About course

EN2500 Information Theory and Source Coding 7.5 credits

Administer About course

PLease see course EQ2845.

The course treats the principles underlying the encoding of speech, audio, video, and images at low bit rates. Source coding techniques such as scalar and vector quantization, orthogonal transforms, and linear prediction are introduced and their performance is analyzed theoretically. The theoretical bounds on the performance of source coders are discussed.

Course offering missing for current semester as well as for previous and coming semesters
Headings with content from the Course syllabus EN2500 (Spring 2009–) are denoted with an asterisk ( )

Content and learning outcomes

Course contents

Information theory of discrete and continuous variables: entropy, Kraft inequality, relative entropy, entropy rate, redundancy rate, mutual information, asymptotic equipartition. Estimation of probability density and probability mass functions. Expectation-Maximization algorithm. Maximum entropy principle.

Lossless coding: nonadaptive codes: Shannon, Huffmann, arithmetic codes. Universal and adaptive codes. Ziv-Lempel codes.

Rate-distortion theory: the rate-distortion function, Shannon lower bound, rate distribution over independent variables, reverse waterfilling, Blahut algorithm.

High-rate quantization: constrained-resolution and constrained-entropy quantization. Vector versus scalar quantization. Practical high-rate-theory-based quantizers: mixture and lattice quantizers, companding.

Low-rate quantization: Lloyd training algorithm for constrained-resolution and constrained-entropy cases. Structured vector quantization (tree-structured, multi-stage, gain-shape, lattice). Fast search methods.

Transforms and filter banks: bases and frames. Transforms and filter banks. Fixed transforms: DFT, DCT, MLT, Gabor frames, Balian-Low theorem. A-priori adaptation: Karhunen-Loeve, a-priori energy concentration. A-posteriori adaptation: a-posteriori energy concentration, best-basis search, matching pursuit.

Linear prediction: closed-loop prediction, noise-shaping, analysis-by-synthesis, spectral flatness, Kolmogorov's Formula, spectral flatness, redundancy, forward and backward prediction.

Intended learning outcomes

To obtain an understanding of the theoretical principles of source coding.

Course disposition

No information inserted

Literature and preparations

Specific prerequisites

For single course students: 120 credits and documented proficiency in English B or equivalent

Recommended prerequisites

EQ1220 Signal Theory or equivalent.

Equipment

No information inserted

Literature

W.B. Kleijn, A basis for source coding, KTH-S3 (2004).

Examination and completion

If the course is discontinued, students may request to be examined during the following two academic years.

Grading scale

A, B, C, D, E, FX, F

Examination

  • HEM1 - Assignment, 1.5 credits, grading scale: P, F
  • TEN1 - Examination, 6.0 credits, grading scale: A, B, C, D, E, FX, F

Based on recommendation from KTH’s coordinator for disabilities, the examiner will decide how to adapt an examination for students with documented disability.

The examiner may apply another examination format when re-examining individual students.

Other requirements for final grade

Written examination.
Homework.

Opportunity to complete the requirements via supplementary examination

No information inserted

Opportunity to raise an approved grade via renewed examination

No information inserted

Examiner

Profile picture Markus Flierl

Ethical approach

  • All members of a group are responsible for the group's work.
  • In any assessment, every student shall honestly disclose any help received and sources used.
  • In an oral assessment, every student shall be able to present and answer questions about the entire assignment and solution.

Further information

Course web

Further information about the course can be found on the Course web at the link below. Information on the Course web will later be moved to this site.

Course web EN2500

Offered by

EES/Electric Power and Energy Systems

Main field of study

This course does not belong to any Main field of study.

Education cycle

Second cycle

Add-on studies

No information inserted

Contact

Markus Flierl