Import 450MB csv file

Hi,

I am trying to import a huge csv file (515000 rows and 93 columns) onto matlab. So far I have tried the Import Data wizard, xlsread command. Both cases I've got 'not enough memory available' error.

I am running v 7.10.0(R2010a) on a 32-bit Windows XP system powered by 2.93 GHz processor and 2 GB of RAM.

Increasing Java heap memory in MATLAB preferences didn't help (not sure if that's what I am supposed to do though).

Is there a way load the file apart from splitting it up into parts ?

thanks, Chetan

[Fangjun Jiang]

What is in the .csv file? Numerical data or strings? If all are numerical data, 515000*93*8=383M bytes. It could be handled.

[Chetan Kumar K]

its numeric data only. no strings.

[Chetan Kumar K]

@Jiang: Is there anyway to specify matlab memory usage ?? (similar to java vm arguments)

+ Chetan

[Image Analyst]

Are you sure it all needs to be in memory at the same time? What's wrong with reading in part of it and processing parts of it at a time?

[Walter Roberson]

There is no way to specify MATLAB memory usage on Windows. The most you can do is add swap space and use the /3G boot switch.

Unless perhaps you are running a fully-occupied laptop, probably the most cost effective way by far to resolve the problem would be to increase your physical RAM.

Answer 1

[Fangjun Jiang]

See if you can find anything useful here.

Avoiding 'Out of Memory' Errors