Information theory of discrete and continuous variables: Entropy, Kraft inequality, relative entropy, entropy rate, redundancy rate, mutual information, asymptotic equipartition.
Lossless coding: nonadaptive codes: Shannon, Huffmann, arithmetic codes. Universal and adaptive codes. Ziv-Lempel codes.
Rate-distortion theory: the rate-distortion function, Shannon lower bound, rate distribution over independent variables, reverse waterfilling, Blahut algorithm.
High-rate quantization: resolution-constrained and entropy-constrained quantization. Scalar and vector quantization.
Low-rate quantization: Lloyd training algorithm for resolution-constrained and entropy-constrained cases.
Transform coding: Orthonormal transforms, transform coding gain, Karhunen-Loeve transform, Energy concentration.
Predictive coding: MSE-optimal linear prediction, linear prediction gain, coding of prediction error, closed-loop prediction.