Asymptotic Properties of Markov Decision Processes


Speaker: Professor Roger Brockett, Harvard University, USA.

Time: Mon 2009-05-04 13.15 - 14.00

Location: M3, Brinellvägen 64 Entreplan, KTH

Abstract: A large number of applications of the optimal control of Markov
processes are known, ranging from problems motivated by situations arising
in service industries to problems involving network control. These problems
are often classified as being finite time problems or infinite horizon
problems. Recently we derived a differential equation for the expected
value of the minimal cost associated for a wide class of such problems
involving the optimal adjustment of the transition rates subject to an
appropriate performance measure. Although the optimal policy is given
explicitly in feedback form, depending only on the solution of a certain
differential equation, it has remained unclear as to if the feedback control
policy approaches a constant policy in
steady state or if, perhaps, it approaches a periodic policy. In this talk
we will review recent results in this area with emphasis on the asymptotic
situation.

Biography of Roger Brocket: Roger Brocket joined the Department of
Electrical Engineering at the Massachusetts Institute of Technology in 1963
as an assistant professor and Ford Foundation Fellow, working in automatic
control. In 1969 he was appointed Gordon McKay Professor of Applied
Mathematics in the Division of Applied Sciences at Harvard, where he became
An Wang Professor of Electrical Engineering and Computer Sciences in 1989.
Experimental and theoretical aspects of robotics, including aspects of
manipulation, computer control and sensor data fusion, are the focus of his
present work. Dr. Brockett has held visiting positions at a number of
universities, including the Tokyo Institute of Technology. Over the past 30
years, Dr. Brockett has been involved in the professional activities of
IEEE, SIAM and AMS, having served on the advisory committees and editorial
boards for several groups in these societies, including IEEE's Transactions
on Automatic Control and the IEEE Transactions on Robotics and Automation.
He was a co-editor for Systems and Control Letters from its founding until
1980. In 1989, he received the American Automatic Control Council's Richard
E. Bellman Award, and in 1991, he received the IEEE Control Systems and
Engineering Field Award. In 1996 he received SIAM's Reid Prize for his work
in differential equations and control. He was elected to the National
Academy of Engineering in 1991.

Subject area: ACCESS

2009-05-04T13:15 2009-05-04T14:00 Asymptotic Properties of Markov Decision Processes (ACCESS ) Asymptotic Properties of Markov Decision Processes (ACCESS )
Top page top