Friday, August 27, 2010

Karhunen-Loeve transform in java

Background knowledge and matlab code here.
The matlab code below is taken from the previous paper.
%number of examples
%dimension of each example
%centering the data
%covariance matrix
%computing the eigenspace:
[U D]=eig(C);
%projecting the centered data
over the eigenspace

Helpful lecture video
Download JAMA package
Java code:
import Jama.*;

public class KLTransform {

//Matrix x[i,j]= jth feature in ith example
public Matrix k_lTransform(Matrix x)
   x = x.transpose();
   int nExample = x.getColumnDimension();

  //calculate mean
   Matrix mean = getMean(x);
   double[][] oneD = new double[1][nExample];
   for(int i = 0; i < nExample; i++)
       oneD[0][i] = 1;
   Matrix ones = new Matrix(oneD);
   //center the data
   Matrix xm = x.minus(mean.times(ones)); 

   //Calculate covariance matrix
   Matrix cov  = xm.times(xm.transpose());
   In the matlab code, the covariance matrix is divided with N (nExample). 
   Now cov and cov/nExample have the same eigenvectors but different eigenvalues. 
   In this code, the division doesn't make any difference as we are only 
   considering the eigenvectors. 
   But there are some cases, like in Kaiser-Guttman stopping rule 
   where only the eigenvectors with eigenvalue > 1 are chosen, 
   division might make a difference.
   cov = cov.times(1.0/nExample);

   //compute eigen vectors
   Matrix eigenVectors = cov.eig().getV();

  //compute pca
   Matrix pca = eigenVectors.transpose().times(xm);
   return pca;
public Matrix getMean(Matrix x) {
 int nExample = x.getColumnDimension();
 int nFeature = x.getRowDimension();
 double[][] meanD = new double[nFeature][1];
 Matrix mean = new Matrix(meanD);
 for(int i = 0; i < nFeature; i++)
   double avg = 0.0;
   for(int j = 0; j < nExample; j++)
     avg+=x.get( i,j);
   mean.set(i, 0, avg/nExample);
 return mean;
public static void main(String[] args)
  KLTransform kl = new KLTransform();

  double [][] d = new double[][]{{1, 2, 3},{4,5,6}};
  Matrix x = new Matrix(d);
  Matrix pc = kl.k_lTransform(x);

  pc.print(pc.getRowDimension(), 2);


Note that in the matlab code, number of examples is the number of columns that is each column is an example. But the java code assumes each row is an example. So the matrix X in matlab is the transpose of matrix X in java.
To test the java and matlab code:
run java with :
[1 2 3]
[4 5 6]

run matlab with: X = [1, 4; 2, 5; 3, 6]

syntax highlighting tutorial

1 comment:

Sajib said...

Wondering how come I haven't visited your blog earlier despite being in the blogging world for about two years. :O

A little suggestion: could you please embed the comment form at the end of your posts? It's really bothering to open up a new window to post the comment. Hope you will think about it.

Don't forget to pay a visit to my blog (linked with my name) and

Eid Mubarak. :)