Feature Transforamtions for Classification by Maximization of Mutual Information

This page presents an implementation (code and results) of the maximization of mutual information algorithm initially developed by Kari Torkkola in a serial of papers. A version of this algorithm was implemented by myself and the results of running it on one standard data set are presented. For much more information including links to some of the original papers please visit Kari Torkkola's original maximimization of mutual information page.

Below you will find some pictures showing the initial distribution of data in each class. As the iterates of the maximization of mutual information algorithm proceeds one can see the classes begin to separate into much more distinct clusters. Classification in this transformed space will be easier than in the original. Please email with any questions/comments you might have.

The initial two dimensional projection (using multiple discriminant analysis) of the 36 feature data set LANDSAT onto two dimenions.

The result from the first step of the optimization algorithm. Notice how the algorithm is "pulling" the data points apart.

The result from the second step of the optimization algorithm, even more sparation of the classes can be seen.

The final iterate of our optimization routine.

Maximization of Mutual Information Matlab Code

John Weatherwax
Last modified: Wed Apr 4 21:10:02 EDT 2007