FSF3953 Markov Chains and Processes 7.5 credits

The lectures will cover the following topics.

1. Markov chains: basic definitions
2. Stopping times and the strong Markov property
3. Atomic chains
4. General irreducible chains
5. Feller kernels
6. Ergodic theory and the law of large numbers
7. Central limit theorems and the Poisson equation
8. Geometric ergodicity and Foster-Lyapunov conditions

After having passed the course, the participant is supposed to be able to

• classify Markov chains as irreducible, recurrent or transient, positive or null.
• explain the classical recurrence-transience dichotomy for Markov chains.
• establish that a given Markov chain has a unique invariant distribution.
• explain the central limit theorem for ergodic Markov chains.
• judge whether a given Markov chain is geometrically ergodic using coupling sets and Foster­Lyapunov drift conditions.
• illustrate the theory by examples from time series analysis and Markov chain Monte Carlo methods.

An advanced level course in stochastic processes and knowledge of basic measure theory.

The course is based on lecture notes. Relevant references are, e.g.

Meyn, S. P. and Tweedie, R. L. (2009). Markov Chains and Stochastic Stability. Cambridge University Press, London.

Assmussen, S. (2003). Applied Probability and Queues. Springer, New York.

If the course is discontinued, students may request to be examined during the following two academic years.

• HEM1 - Home assignments, 3.5 credits, grading scale: P, F
• TENM - Oral exam, 4.0 credits, grading scale: P, F

Based on recommendation from KTH’s coordinator for disabilities, the examiner will decide how to adapt an examination for students with documented disability.

The examiner may apply another examination format when re-examining individual students.

The examination consists in a combination of home assignments and an oral exam.

Approved assignments and exam.

Jimmy Olsson

• All members of a group are responsible for the group's work.
• In any assessment, every student shall honestly disclose any help received and sources used.
• In an oral assessment, every student shall be able to present and answer questions about the entire assignment and solution.