**Introduction:**

These are results and code for the
problems and examples found in Chapter 4 of this famous book.

- Policy Iteration for the Grid World Example:
- Reproduces this example using in-place and non in-place policy iteration:
- iter_poly_gw_not_inplace.m

- iter_poly_gw_inplace.m (converges to the same final state as the result in the book)

- Exercise 4.2 (Adding a state to grid world):
- ex_4_2_sys_solv.m (This script solve the simple linear system)

- ex_4_2_sys_solv.m (This script solve the simple linear system)
- Policy Improvement for the Jacks Car Rental (JCR) Example:
- jcr_example.m (This script reproduces the results from the book)

- jcr_policy_evaluation.m (the policy evalutation step)

- jcr_policy_improvement.m (policy inprovement step)

- jcr_rhs_state_value_bellman.m (computes the expectation over new states)

- cmpt_P_and_R.m (a utility function for the JCR example)

- sample output using the above code (results obtained when running jcr_example.m)

- jcr_example.m (This script reproduces the results from the book)
- Exercise 4.5 (Modifications on the Jacks Car Rental Example):
- ex_4_5_Script.m (reproduces the results from the book)

- ex_4_5_policy_evaluation.m (the policy evalutation step)

- ex_4_5_policy_improvement.m (policy inprovement step)

- ex_4_5_rhs_state_value_bellman.m (computes the expectation over new states)

- sample output from the above code (results obtained when running gam_Script.m)

- ex_4_5_Script.m (reproduces the results from the book)
- The Gambler's Problem:
- gam_Script.m (runs value iteration for the gambler problem)

- gam_rhs_state_bellman.m (compute the backup computations for gambler problem)

- some sample output that can be obtained from the above code

- gam_Script.m (runs value iteration for the gambler problem)

John Weatherwax