Kalender

Introduction

• Basic ideas and concepts.
• Models of dynamical systems
• Noise models
• Stochastic processes

Reading material

Ljung: Chapters 1, 2

Optional material

 L. Ljung. “Perspectives on System Identification”. Annual  Reviews in Control, Vol. 34(1), pp. 1-12, 2010.

 Lecturer

Håkan Hjalmarsson

Signal Estimation Methods

• Minimum mean-square error estimation
• Prediction, filtering and smoothing.
• Optimal  filtering.
• Wiener/Kalman/particle filters

Reading material

Ljung: Chapter 3

Optional material

Lecturer

Håkan Hjalmarsson

Signal Estimation - Selected Topics

• Some concepts from probability theory
• Geometrical interpretation of estimation
• Properties of the Kalman filter

Reading material

• C.E. De Souza, M.R. Gevers and G.C. Goodwin: Riccati equations in optimal filtering of nonstabilizable systems having singular state transition matrices, IEEE Transactions on Automatic Control, 31(9), pp. 831-838
• probability_theory_1.pdf

Optional material

-

Lecturer

Håkan Hjalmarsson

Problems (should have been handed in at previous lecture)

FEL3201 & FEL 3202: 2E1, 2E2, 2E5

FEL3202: 2E3,  2T4, 2D3, 2D4,

Lecturer

Håkan Hjalmarsson

• System Estimation Methods I: Basic principles and unstructured estimation

• Non-parametric estimation
• Least-squares estimation
• The maximum likelihood method

Reading material

Ljung: Chapter 2.4, 6, 7.1-7.3 (not pp. 208-211), 7.4

Optional material

• L. Ljung.     “On the Estimation of Transfer Functions”. Automatica, Vol. 21(6), pp. 677-696, 1985.
• Slides 1  
• Lemma 2.1

Lecturer

Håkan Hjalmarsson

System Estimation Methods II: Basic Principles cont'd, Structured Estimation

• Model structures
• The prediction error method
• The instrumental variables method
• Frequency domain identification

Reading Material

Ljung: Chapters 4.1-4.3, 7.5-7.7

Lecture Slides

• Lecture 4

Optional Material

J. C. Agüero, J. I. Yuz, G. C. Goodwin, R. A. Delgado, On the equivalence of time and frequency domain maximum likelihood estimation, Automatica, 46(2), 2010, 260-270.

Lecturer

Cristian Rojas

Information contents in data

• Sufficient statistics
• Rao-Blackwells theorem
• Exact and conditional ML

Reading material

E.L Lehmann: Theory of Point Estimation

Optional material

• Slides 3 (old)

 Lecturer

Håkan Hjalmarsson

Problems (should have been handed in at previous lecture)

FEL 3201 (8 credits) + FEL 3202(12 credits):   3E1, 3E2,C2a--d, 6G3   

FEL 3202: 3T2, C2e-g, 6T2

Lecturer

Håkan Hjalmarsson

System Estimation Methods III: Structured Estimation using Subspace identification

Reading Material

Ljung: Chapters  4.3, 7.3, 10.6

Lectur Slides

•  Lecture 5

Lecturer

Cristian Rojas

Problems (should have been handed in at previous lecture)

FEL 3201 (8 credits) + FEL 3202(12 credits):   4G4, 4E1, 4E5a, C3       

FEL 3202:  4E5b, C4

Lecturer

Cristian Rojas

Student Presentations

• Bayesian inference
• Inconsistency of Maximum Likelihood Estimation
• Non-parametric identification: Kernel methods

Lecturer

Håkan Hjalmarsson

See Selected Topics 2

Lecturer: Håkan Hjalmarsson

Fundamental properties of estimation algorithms

• Identifiability
• Persistence of excitation
• Consistency
• Asymptotic distribution of PEM

Reading Material

Ljung: Chapters 4.5-4.6, 8

Optional References

• Lecture 6

Lecturer

Cristian Rojas

Fundamental properties

• Asymptotic covariance expressions (for high model orders)
• Proof of a.s. convergence results

Lecture Slides

• Special topics 3

Lecturer

Cristian Rojas

Model Quality Evaluation

• Bias / variance trade-off
• Stein's paradox and biased estimators
• Confidence intervals / regions
• Variance error quantification
• Geometric approach to variance analysis

Reading Material

• Ljung: Chapter 9
• A Geometric Approach to Variance Analysis in System Identification, H. Hjalmarsson and J. Mårtensson, IEEE Transactions on Automatic Control, 56(5), pp. 983-997, 2011.

Lecture Slides

• Lecture 7
  

Lecturer

Cristian Rojas

Model Quality Evaluation

• Decay rate of the Cramér-Rao bound
• Barankin bound and SNR-threshold effect
• Superefficiency
• Nuisance parameters

Reading material

Lecture Slides

• Special topics 4

Lecturer

Cristian Rojas

Student presentations

•  Structured estimation 1: Low rank approximation using local search
•  Structured estimation 2: Nuclear norm minimization
•  Errors-in-variables identification

Lecturer

Cristian Rojas

Model Selection and Model Validation

• Classic model order selection  criteria (AIC, BIC, MDL),
•  Hypothesis tests

Reading Material

Ljung: Chapter 16

Optional References

• Slides 10 (old)
• Slides 11 (old)

Lecturer

Håkan Hjalmarsson

Identification of non-linear systems

• Linear regressions
• Function expansions
• Radial basis functions
• Wavelet expansions
• Kernel expansions
• Splines
• Non-linear filtering and prediction

Reading Material

Ljung: Chapter 5

Optional References

Lecturer

Håkan Hjalmarsson

A Minimum-Mean-Square Error Perspective on Model Selection

• Model selection as the minimization of the mean-square estimation error
• SURE - Stein's Unbiased Risk Estimate
• ML-estimation

Reading material

Optional material

 Lecturer

Håkan Hjalmarsson

Problems (should have been handed in at previous lecture)

FEL 3201 (8 credits) + FEL 3202(12 credits):   7E2, 7G1, 8E2, 9E3

FEL 3202:  7E8, 8E4, 8G5

Lecturer

Cristian Rojas

Estimation Algorithms

• Solving normal equations using QR-factorization
•  Non-linear optimization
• Two- and multi-stage methods
• The EM-algorithm

Reading Material

Ljung: Chapter 10

Lecture Slides

• Lecture 10

Lecturer

Cristian Rojas