Thread Subject: Write/Read large datafile

Subject: Write/Read large datafile

From: Sebastien Paris

Date: 22 Dec, 2011 20:09:08

Message: 1 of 1

Hello,

Just a question for experts of Matlab. Here is my wish's list

i) I want to fast read/write partially from/to huge amount of data (not necesserly with compression). I try the new function matfile of 2011b, but it's extremly slow due to on the fly compression.
matobj = matfile('test.mat' , 'writable' , true);
matobj.X = zeros(300*133 , 5000 , 'single');
tic,matobj.X(: , 1) = randn(39900 , 1 , 'single');,toc

Elapsed time is 4.199436 seconds.

Very slow ...

ii) I want to read few columns given an columns index vector.
iii) the file can be > 2gb


Maybe with HDF5 without compression ?

Any advices ?

Thank you all

Tags for this Thread

Everyone's Tags:

Add a New Tag:

Separated by commas
Ex.: root locus, bode

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Tag Activity for This Thread
Tag Applied By Date/Time
io files Sebastien PARIS 22 Dec, 2011 15:14:10
big data files Sebastien PARIS 22 Dec, 2011 15:14:10
rssFeed for this Thread

Contact us at files@mathworks.com