Reinforcement Learning Books

Reading recent papers is useful, but sometimes there is a need to get a high-level overview of the field, or the need to dive deeper into mathematical background. Hence, here is a short list of reasonably recent books that would be useful for such purposes:

"Reinforcement Learning: State of the Art", 2012
An overview of key recent research results in RL theory.
http://link.springer.com/book/10.1007/978-3-642-27645-3

"Algorithms for Reinforcement Learning", Szepesvari, 2009
Key basics for theory of RLs for MDPs. Solid mathematical details, yet quite accessible.
https://sites.ualberta.ca/~szepesva/papers/RLAlgsInMDPs.pdf

"Reinforcement Learning", Sutton and Barto, 2014-2016
The classic. Provides a nice overview (though not mathematically detailed).
https://webdocs.cs.ualberta.ca/~sutton/book/bookdraft2016sep.pdf

"Markov Decision Processes", Puterman 2005
Mathematical basics for MDPs.
http://onlinelibrary.wiley.com.focus.lib.kth.se/book/10.1002/9780470316887

"Dynamic Programming and Optimal Control", Bertsekas, 2005/2014
Mathematical background for MDPs, multi-armed bandits; analysis of infinite horizon discounted problems (most frequent formulation), as well as chapters on undiscounted, average-cost and continuous-time problems.
http://www.athenasc.com/dpbook.html

"Artificial Intelligence: A Modern Approach", Russell and Norvig, 2010
Chapters 16-17 contain a brief, but very accessible introduction into decision making and RL.
http://aima.cs.berkeley.edu

[If there are other sources you found useful - please contribute!]