Problem with reading MAT file (big dataset, 3.65GB ) in c++, r2011a

I'm exploring with a public image dataset (NYU), which is provided as a very large mat file. The dataset is provided as a Matlab .mat file with the following variables:

• images – H x W x 3 x N matrix of RGB images where H and W are the height and width, respectively, and N is the number of images.
• labels – H x W x N matrix of label masks where H and W are the height and width, respectively and N is the number of images. The labels range from 1..C where C is the total number of classes.
• names – C x 1 cell array of the english names of each class.

so it's unpratical to read the whole array because of its size.(for example images matrix size is 480*640*3*2284= 2104934400~!)

once I tried the method according to this issue: http://www.mathworks.es/matlabcentral/answers/49529-problem-with-large-arrays-c-variable-editor-in-r2012a-win64

  //const char* filename = "nyu_data.mat";
  MATFile *pmat;
  mxArray *pa1;

  // open mat file
  pmat = matOpen(filename,"r");
  if(pmat == NULL)
  {
    printf("Error creating file %s\n", filename);
    printf("(Do you have write permission in this directory?)\n");
    return(EXIT_FAILURE);
  }

  // read images info
  pa1 = matGetVariableInfo(pmat, "images");    
  //pa1 = matGetVariable(pmat, "images");    // error! -- matGetVariable will run out of memory
  if (pa1 == NULL) {
    printf("Error reading existing matrix\n");
    return(EXIT_FAILURE);
  }

  int image_dims = mxGetNumberOfDimensions(pa1);
  printf("dims: %d\n", image_dims);

  const mwSize *dims = mxGetDimensions(pa1);
  cout << "pa1 is " << mxGetClassName(pa1) << " with dimensions ";
  for (mwSize c = 0; c < mxGetNumberOfDimensions(pa1); c++) cout << ((c == 0)? "":" x ") << dims[c];
  cout << endl << "total number of elements : " << mxGetNumberOfElements(pa1) << endl;

  // clean up
  mxDestroyArray(pa1);
  if (matClose(pmat) != 0) {
    printf("Error closing file %s\n",filename);
    return(EXIT_FAILURE);
  }
  printf("Done\n");
  return(EXIT_SUCCESS);

since here I can just read the matrix head info. matGetVariable() causes an error,so mxGetData() return NULL. And matGetFp() also return NULL..

How can I get the very i th matrix such as images[i]=image(:,:,:,i) instead of the total matrix?

Answer 1

[gzricky]

now I can read it into matlab workspace (r2009b 64bit). BY the way, one can use command "memory" to see how large sequent space can be used. :P