Feature Transforamtions for Classification by Maximization of Mutual Information
This page presents an implementation (code and results) of the
maximization of mutual information algorithm initially developed by Kari
Torkkola in a serial of papers. A version of this algorithm was
implemented by myself and the results of running it on one standard data
set are presented. For much more information including links to some of
the original papers please visit Kari Torkkola's original maximimization
of mutual information
page.
Below you will find some pictures showing the initial distribution of data
in each class. As the iterates of the maximization of mutual information
algorithm proceeds one can see the classes begin to separate into much
more distinct clusters. Classification in this transformed space will
be easier than in the original. Please email with any questions/comments
you might have.
The initial two dimensional projection (using multiple
discriminant analysis) of the 36 feature data set LANDSAT onto
two dimenions.
The result from the first step of the optimization algorithm. Notice how the algorithm is
"pulling" the data points apart.
The result from the second step of the optimization algorithm, even more sparation of the
classes can be seen.
The final iterate of our optimization routine.
Maximization of Mutual Information Matlab Code
John Weatherwax
Last modified: Wed Apr 4 21:10:02 EDT 2007