Committee login






Small thumbnail

Baidu SEO

Challenges and Intricacies of Marketing in China

Small thumbnail

Asymmetric Alliances and Information Systems

Issues and Prospects

Small thumbnail

Technicity vs Scientificity

Complementarities and Rivalries

Small thumbnail

Freshwater Fishes

250 Million Years of Evolutionary History

Small thumbnail

Biostatistics and Computer-based Analysis of Health Data using SAS

Biostatistics and Health Science Set

Small thumbnail

Predictive Control

Small thumbnail

Fundamentals of Advanced Mathematics 1

Categories, Algebraic Structures, Linear and Homological Algebra

Small thumbnail

Swelling Concrete in Dams and Hydraulic Structures

DSC 2017

Small thumbnail

The Chemostat

Mathematical Theory of Microorganims Cultures

Small thumbnail

Earthquake Occurrence

Short- and Long-term Models and their Validation

Small thumbnail

Markov Decision Processes in Artificial Intelligence

Edited by Olivier Buffet, LORIA, Vandoeuvre-lès-Nancy, France Olivier Sigaud, University Pierre and Marie Curie

ISBN: 9781848211674

Publication Date: February 2010   Hardback   480 pp.

160.00 USD

Add to cart


Ebook Ebook


Markov Decision Processes (MDPs) are a mathematical framework for modeling sequential decision problems under uncertainty as well as Reinforcement Learning problems.
Written by experts in the field, this book provides a global view of current research using MDPs in Artificial Intelligence. It starts with an introductory presentation of the fundamental aspects of MDPs (planning in MDPs, Reinforcement Learning, Partially Observable MDPs, Markov games and the use of non-classical criteria). Then it presents more advanced research trends in the domain and gives some concrete examples using illustrative applications.


Part 1: MDPs: Models and Methods
1. Markov Decision Processes, F. Garcia, E. Rachelson.
2. Reinforcement Learning, O. Sigaud, F. Garcia.
3. Approximate Dynamic Programming, R. Munos.
4. Factored Markov Decision Processes, T. Degris, O. Sigaud.
5. Policy-gradient Algorithms, O. Buffet.
6. Online Resolution Techniques, L. Péret, F. Garcia.
Part 2: Beyond MDPs
7. Partially Observable Markov Decision Processes, A. Dutech, B. Scherrer.
8. Stochastic Games, A. Burkov, L. Matignon, B. Chaib-Draa.
9. DEC-MDP/ POMDP, A. Beynier et al.
10. Non-standard Criteria, M. Boussard, M. Bouzid, A.-I. Mouaddib, R. Sabbadin, P. Weng.
Part 3: Applications
11. Online Learning for Micro-object Manipulation, G. Laurent.
12. Conservation of Biodiversity Conservation, I. Chadès.
13. Autonomous Helicopter Searching for a Landing Area in an Uncertain Environment, P. Fabiani,
F. Teichteil-Königsbuch.
14. Resource Consumption Control for an Autonomous Robot, S. Le Gloannec, A.-I. Mouaddib.
15. Operations Planning, S. Thiébaux, O. Buffet.

About the Authors

Olivier Sigaud is a Professor of Computer Science at the University of Paris 6 (UPMC). He is the Head of the "Motion" Group in the Institute of Intelligent Systems and Robotics (ISIR).
Olivier Buffet has been an INRIA researcher in the Autonomous Intelligent Machines (MAIA) team of the LORIA laboratory since November 2007.


DownloadTable of Contents - PDF File - 138 Kb

DownloadPreface - PDF File - 42 Kb

Related Titles

0.05621 s.