fread() optimization binary file
6 views (last 30 days)
Show older comments
I am reading a 40 MB file with three different ways. But the first one is way faster than the 2 others. Do you guys have an idea why ? I would rather implement condition in loops or whiles to separate data than load everything with the first quick method and separate them then - memory saving -
LL=10000000;
fseek(fid,startbytes, 'bof');
%%Read all at once %%%%%%%%%%%%%%%%%%%%%%
tic
AA(:,1)=int32(fread(fid,LL,'int32'));
toc
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
fseek(fid,startbytes,'bof');
%%Read all using WHILE loop %%%%%%%%%%%%%
tic
i=0;
AA2=int32(zeros(LL,1));
while i<LL
i=i+1;
AA2(i,1)=fread(fid,1,'int32');
end
toc
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
fseek(fid,startbytes,'bof');
%%Read all using FOR loop %%%%%%%%%%%%%%%
tic
AA3=int32(zeros(LL,1));
for i=1:LL
AA3(i,1)=fread(fid,1,'int32');
end
toc
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Elapsed time is 0.312916 seconds.
Elapsed time is 138.811520 seconds.
Elapsed time is 116.799286 seconds.
0 Comments
Answers (1)
Walter Roberson
on 26 Jan 2013
Consider reading data in chunks (e.g., 4 Kb): that should give you better speed than reading single int32 at a time, but would also reduce the amount of memory needed.
0 Comments
See Also
Categories
Find more on Scope Variables and Generate Names in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!