Value iteration

I need help with value iteration. The problem is this: (I used underscores to denote subscripts to try to improve the clarity-> subscript)

J_t_(w_t_) is a function that calculates an expected cost from time t to time t+1 added to the expected cost for all future time periods given all future decisions. The cost function depends on the current state (w_t_), the action taken, (u_t_) and the state uncertainty (n_t_).

J_t_(w_t_)=E(c(w_t_,u_t_,n_t_)+J_t+1_(w_t+1_))

The optimal action will be to choose u_t_ such that J_t_ is minimized.

J*_t_(w_t_)=min(E(c(w_t_,u_t_,n_t_)+J_t+1_(w_t+1_)))

This will converge to a fixed point such that J*_t_=J_t+1_. To find the values at J*, value iteration is needed. So the idea is to choose an arbitrary value for J* at a time very far in the future and use the above equation to work backwards from there to find J*. This is what I need help with. Does anyone have any insight as to how to do this in matlab?

Please ask if you think you can help but this isn't clear, and thanks in advance for any help given.

-Evan

[Rick Rosson]

The subscripts do not add clarity for me. They make it almost impossible for me to understand your question. Could you please format your equations as MATLAB code with simple and clean variable names? Thanks.

Answer 1

[Sofie]

Dear Evan,

I am currently working on the same problem, as I recognize that this is the paper of sun et al(2006) right? Did you manage to solve this issue, and if so how did you do that?

Regards