Thread Subject: Simultaneous disk read and computations

Subject: Simultaneous disk read and computations

From: NIcholas

Date: 31 May, 2009 20:17:01

Message: 1 of 1

Hello,
I have amount of data stored on disk - now about 10GB stored as 20 different files in 500MB blocks. I need to loop through each block, do some computations, take in the next block etc. Right now, reading the file (binary format) takes about as much time as my computations. I was wondering if it were possible to read in the next block while the CPU is processing the first block.
Code looks something like this:

for i=1:nBlock
A=fread(fid{i},inf,'*uint8');
result{i} = doSomething(A);
end

I would like to do something like this:

A{1} = fread(fid{1},inf,'*uint8');
for i=1:nBlock-1
   indCurr = mod(i-1,2)+1; %1 if i if odd, 2 if i is even
   indNext = mod(i,2)+1; %1 if i is even, 2 if i is odd
   b{i} = doSomething(A{indCurr}); % CPU intensive
   A{indNext} = fread(fid{1+1},inf,'*uint8'); % disk intensive
end
b{nBlock} = doSomthing(A{indNext});

where the lines labeled "CPU intensive" and "disk intensive" are done in parralel.
Thanks,
Nick

Tags for this Thread

Add a New Tag:

Separated by commas
Ex.: root locus, bode

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

rssFeed for this Thread

Contact us at files@mathworks.com