**Introduction:**

These are results and code for the
problems and examples found in Chapter 8 of this famous book.

- Using Coarse Coding to Approximate a Step Function
- linAppFn.m (approximate a function with linear patches)

- targetF.m (the target step function)

- stp_fn_approx_Script.m (the driver function that will run everything)

- sample output using the above codes

- linAppFn.m (approximate a function with linear patches)
- The Mountain Car Example:
- Driver scripts to run the examples (run GetTiles_Mex_Script.m to create the needed MEX interface to GetTiles):
- GetTiles_Mex_Script.m (compiles and then provides an example Matlab call to Dr. Sutton's tiling software GetTiles)

- mnt_car_learn_Script.m (runs mnt_car_learn.m with appropriate inputs)

- GetTiles_Mex_Script.m (compiles and then provides an example Matlab call to Dr. Sutton's tiling software GetTiles)
- Sample output produced from the above codes (plot of -max_a Q_t(s,a))

- Component code called from the above driver functions:
- GetTiles_Mex.C (a Matlab MEX wrapper to Dr. Sutton's tiling software GetTiles)

- tiles.C (Dr. Sutton's tiling software in double precision)

- tiles.h (Dr. Sutton's tiling software in double precision)

- mnt_car_learn.m: Learns the mountaintop example. Special thanks to Paul Fackler for finding a bug that was vastly slowing the code down.

- get_ctg.m (extracts the cost to go function)

- ret_q_in_st.m (returns the action-function for the given state)

- next_state.m (returns the action-function for the given state)

- GetTiles_Mex.C (a Matlab MEX wrapper to Dr. Sutton's tiling software GetTiles)

- Driver scripts to run the examples (run GetTiles_Mex_Script.m to create the needed MEX interface to GetTiles):
- Learning Rates for the Mountain Car Example (Replacement vs. Accumulate Traces):
- do_mnt_car_Exps.m (does the alpha lambda experiments performed in the book)

- sample output using the above codes

- do_mnt_car_Exps.m (does the alpha lambda experiments performed in the book)

John Weatherwax