|
Hello,
I have amount of data stored on disk - now about 10GB stored as 20 different files in 500MB blocks. I need to loop through each block, do some computations, take in the next block etc. Right now, reading the file (binary format) takes about as much time as my computations. I was wondering if it were possible to read in the next block while the CPU is processing the first block.
Code looks something like this:
for i=1:nBlock
A=fread(fid{i},inf,'*uint8');
result{i} = doSomething(A);
end
I would like to do something like this:
A{1} = fread(fid{1},inf,'*uint8');
for i=1:nBlock-1
indCurr = mod(i-1,2)+1; %1 if i if odd, 2 if i is even
indNext = mod(i,2)+1; %1 if i is even, 2 if i is odd
b{i} = doSomething(A{indCurr}); % CPU intensive
A{indNext} = fread(fid{1+1},inf,'*uint8'); % disk intensive
end
b{nBlock} = doSomthing(A{indNext});
where the lines labeled "CPU intensive" and "disk intensive" are done in parralel.
Thanks,
Nick
|