Code and Results for Chapter 6:

Introduction:
These are results and code for the problems and examples found in Chapter 6 of this famous book.

1. Comparing TD(0) and constant alpha MC learning on the Random Walk Example:
   • mk_fig_6_6.m (generates the plot in Figure 6.6)
     
   • mk_arms_error_plt.m (generates the plots in Figure 6.7)
     
   • sample output using the above codes
     
   • mk_batch_arms_error_plt.m (generates the plot in Figure 6.8)
     
   • sample output using the above code (when running mk_batch_arms_error_plot.m)
     
   • cmpt_arms_err.m (auxillary function to produces the plots for Figure 6.7 of the book)
     
   • eg_6_2_learn.m (learns state value function policy with TD(0) and constant alpha MC method)
     
   • eg_rw_batch_learn.m (batch learning for this example)
     
2. The Windy Gridworld Example:
   • run_all_gw_Script.m (driver to run all grid world examples)
     
   • windy_gw_Script.m (driver to solve the windy grid world example)
     
   • windy_gw.m (core code to solve the windy grid world example)
     
   • wgw_w_kings_Script.m (include kings moves)
     
   • wgw_w_kings.m (the core code where we allow kings moves)
     
   • wgw_w_kings_n_wind_Script.m (include kings moves and a wind action)
     
   • wgw_w_kings_n_wind.m (the case where we allow kings moves and wind action)
     
   • wgw_w_stoch_wind_Script.m (include stochastic wind)
     
   • wgw_w_stoch_wind.m (the case when there is stochasitc wind)
     
   • plot_gw_policy.m (plots the grid world policy)
     
   • sample output using the above codes
     
3. The Cliff Walk example (Thanks to J. Rosado for spotting some bugs in these programs)
   • learn_cw_Script.m (driver to solve the cliff walk example)
     
   • learn_cw.m (core code to solve the cliff walk example)
     
   • plot_cw_policy.m (plots the cliff walk policy)
     
   • sample output using the above codes
     
4. The R learning example:
   • R_learn_acq_Script.m (driver for R learning on the access control problem)
     
   • R_learn_acq.m (implements R learning on the access control problem)
     
   • sample output using the above codes
     

   ---

   John Weatherwax