Code and Results for Chapter 7:

Introduction:
These are results and code for the problems and examples found in Chapter 7 of this famous book.

1. Using TD(n) learning to learn random walks:
   • rw_online_ntd_learn_Script.m (driver to run online n-TD learning)
     
   • rw_online_ntd_learn.m (implements online n-TD learning)
     
   • rw_offline_ntd_learn_Script.m (driver to run offline n-TD learning)
     
   • rw_online_ntd_learn.m (implements offline n-TD learning)
     
   • rw_episode.m (generates an episode from the random walk problem)
     
   • sample output using the above codes
     
2. Using TD(lambda) learning to learn random walks:
   • rw_online_tdl_learn_Script.m (driver to run online TD(lambda) learning)
     
   • rw_online_tdl_learn.m (implements online TD(lambda) learning)
     
   • rw_offline_tdl_learn_Script.m (driver to run offline TD(lambda) learning)
     
   • rw_online_tdl_learn.m (implements offline TD(lambda) learning)
     
   • sample output using the above codes
     
3. Online TD(lambda) with eligability traces:
   • rw_online_w_et_Script.m (driver to run online TD(lambda) learning)
     
   • rw_online_w_et.m (implements online TD(lambda) learning with eligability traces)
     
   • sample output using the above codes
     
4. Example 7.4 Implementing the Grid World example with eligability traces:
   • gw_w_et.m (implements the windy grid world learning example with elagability traces)
     
   • gw_w_et_Script.m (a driver for the above function)
     
5. Accumulating traces v.s. Replacing traces
   • rw_accumulating_vs_replacing_Script.m (compares accumulating v.s. replacing traces)
     
   • rw_online_w_replacing_traces.m (implements learning with online replacing traces)
     
   • sample output using the above codes
     
6. Example 7.5 Learning from a one directional Markov chain
   • eg_7_5_episode.m (produces an episode of the example 7.5 Markov chain)
     
   • eg_7_5_learn_at.m (uses accumulating traces to learn the state value function for this task)
     
   • eg_7_5_learn_rt.m (uses replicating traces to learn the state value function for this task)
     
   • eg_7_5_Script.m (drives the two above scripts)
     
   • sample output using the above codes
     

   ---

   John Weatherwax