Markov Decision Processes (MDP) Toolbox

Functions related to the resolution of discrete-time Markov Decision Processes.

Description

The MDP toolbox proposes functions related to the resolution of discrete-time Markov Decision Processes: backwards induction, value iteration, policy iteration, linear programming algorithms with some variants.
The functions were developped with MATLAB (note that one of the functions requires the Mathworks Optimization Toolbox) by Iadine Chadès, Marie-Josée Cros, Frédérick Garcia, Régis Sabbadin of the Biometry and Artificial Intelligence Unit of INRA Toulouse (France).

The version 3.0 (September 2009) adds several functions related to Reinforcement Learning and improves the handling of sparse matrices. For more detail see the README file.

Toolbox page: http://www.inra.fr/mia/T/MDPtoolbox

• MATLAB release: MATLAB 7.9 (R2009b)

[Kristin]

Neat stuff! I have not looked at all the files, just the ones I needed, but I noticed something in mdp_value_iteration.m.

you have:
global mdp_VERBOSE
and then you check for existence, even though by calling GLOBAL, it has now been put in existence. Should you be checking ISEMPTY instead or am I missing something?

[Tengjiao Wang]

Nice for the icebreaker in stochastic processes to make sense of markov chain and markov process

[Sandra]

 

[Patrick Anderson]

This is a very nicely done MDP toolbox. The code is simple and elegant, and there is nice help documentation. The authors at INRA also made it accessible to users from different backgrounds.

The fact that it is updated periodically, and that there are identified authors, is also a nice benefit.