Markov Decision Processes (MDP) Toolbox

Functions related to the resolution of discrete-time Markov Decision Processes.

Description

The MDP toolbox proposes functions related to the resolution of discrete-time Markov Decision Processes: backwards induction, value iteration, policy iteration, linear programming algorithms with some variants.
The functions were developped with MATLAB (note that one of the functions requires the Mathworks Optimization Toolbox) by Iadine Chadès, Marie-Josée Cros, Frédérick Garcia, Régis Sabbadin of the Biometry and Artificial Intelligence Unit of INRA Toulouse (France).

The version 3.0 (September 2009) adds several functions related to Reinforcement Learning and improves the handling of sparse matrices. For more detail see the README file.

Toolbox page: http://www.inra.fr/mia/T/MDPtoolbox

• MATLAB release: MATLAB 7.9 (2009b)

[Patrick Anderson]

This is a very nicely done MDP toolbox. The code is simple and elegant, and there is nice help documentation. The authors at INRA also made it accessible to users from different backgrounds.

The fact that it is updated periodically, and that there are identified authors, is also a nice benefit.

[Sandra]

 

[Tengjiao Wang]

Nice for the icebreaker in stochastic processes to make sense of markov chain and markov process