Code and Results for Chapter 8:

Introduction:
These are results and code for the problems and examples found in Chapter 8 of this famous book.

1. Using Coarse Coding to Approximate a Step Function
   • linAppFn.m (approximate a function with linear patches)
     
   • targetF.m (the target step function)
     
   • stp_fn_approx_Script.m (the driver function that will run everything)
     
   • sample output using the above codes
     
2. The Mountain Car Example:
   • Driver scripts to run the examples (run GetTiles_Mex_Script.m to create the needed MEX interface to GetTiles):
     • GetTiles_Mex_Script.m (compiles and then provides an example Matlab call to Dr. Sutton's tiling software GetTiles)
       
     • mnt_car_learn_Script.m (runs mnt_car_learn.m with appropriate inputs)
       
   • Sample output produced from the above codes (plot of -max_a Q_t(s,a))
     
   • Component code called from the above driver functions:
     • GetTiles_Mex.C (a Matlab MEX wrapper to Dr. Sutton's tiling software GetTiles)
       
     • tiles.C (Dr. Sutton's tiling software in double precision)
       
     • tiles.h (Dr. Sutton's tiling software in double precision)
       
     • mnt_car_learn.m: Learns the mountaintop example. Special thanks to Paul Fackler for finding a bug that was vastly slowing the code down.
       
     • get_ctg.m (extracts the cost to go function)
       
     • ret_q_in_st.m (returns the action-function for the given state)
       
     • next_state.m (returns the action-function for the given state)
       
3. Learning Rates for the Mountain Car Example (Replacement vs. Accumulate Traces):
   • do_mnt_car_Exps.m (does the alpha lambda experiments performed in the book)
     
   • sample output using the above codes