MATLAB Central > MATLAB Newsreader > Simultaneous disk read and computations

Thread Subject: Simultaneous disk read and computations

Subject: Simultaneous disk read and computations

From: NIcholas

Date: 31 May, 2009 20:17:01

Message: 1 of 1

Reply to this message

Add author to My Watch List

View original format

Flag as spam

Hello,
I have amount of data stored on disk - now about 10GB stored as 20 different files in 500MB blocks. I need to loop through each block, do some computations, take in the next block etc. Right now, reading the file (binary format) takes about as much time as my computations. I was wondering if it were possible to read in the next block while the CPU is processing the first block.
Code looks something like this:

for i=1:nBlock
A=fread(fid{i},inf,'*uint8');
result{i} = doSomething(A);
end

I would like to do something like this:

A{1} = fread(fid{1},inf,'*uint8');
for i=1:nBlock-1
   indCurr = mod(i-1,2)+1; %1 if i if odd, 2 if i is even
   indNext = mod(i,2)+1; %1 if i is even, 2 if i is odd
   b{i} = doSomething(A{indCurr}); % CPU intensive
   A{indNext} = fread(fid{1+1},inf,'*uint8'); % disk intensive
end
b{nBlock} = doSomthing(A{indNext});

where the lines labeled "CPU intensive" and "disk intensive" are done in parralel.
Thanks,
Nick

Tags for this Thread

Add a New Tag:

Separated by commas
Ex.: root locus, bode

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Feed for this Thread

MATLAB Central Terms of Use

NOTICE: Any content you submit to MATLAB Central, including personal information, is not subject to the protections which may be afforded information collected under other sections of The MathWorks, Inc. Web site. You are entirely responsible for all content that you upload, post, e-mail, transmit or otherwise make available via MATLAB Central. The MathWorks does not control the content posted by visitors to MATLAB Central and, does not guarantee the accuracy, integrity, or quality of such content. Under no circumstances will The MathWorks be liable in any way for any content not authored by The MathWorks, or any loss or damage of any kind incurred as a result of the use of any content posted, e-mailed, transmitted or otherwise made available via MATLAB Central. Read the complete Terms prior to use.