Increase the speed of the program
4 views (last 30 days)
I'm working on a big data file and I have to extract some lines to create a matrix. I have to do this alot of times. In the following program, I do it 6 times and it takes 10s. The problem is that I have to do it 3000 times and maybe more!
Here is the code:
filename = 'blg2zyx.htr';
NumSegments = dlmread(filename,'','B7..B7');
NumFrames = dlmread(filename,'','B8..B8');
Data_Base_position = dlmread(filename,'',strcat('B',int2str(NumSegments + 20),'..H',int2str(NumSegments*2-1 + 20)));
data = zeros(6*NumSegments,7)
for b = 0:3002:3002*NumSegments-1
data(t,:) = dlmread(filename,'',strcat('B',int2str(NumSegments*2 + 20+4+b+j),'..H',int2str(NumSegments*2 + 20+4+b+j)));
data = vertcat(Data_Base_position , data)
Can someone tell me why is it slow, and how to increase the speed?
Knut on 6 May 2011
Whenever I see a "for" statement in MATLAB, I question myself if that operation can be vectorized. Is it possible to use dlmread in such a way that it will generate your matrix directly, instead of writing elements one at a time?
Also, if you do:
You will get some profiling info that can help tell you what parts of the code is consuming many cycles
Jarrod Rivituso on 6 May 2011
I may be missing something, but a general thought...
Try replacing all those calls to dlmread with the textscan function.
I think dlmread uses fopen + textscan + fclose each time you call it, and of course does some error checking in there.
You could call fopen once, and then use textscan and frewind in your for loop as necessary, and then call fclose once afterwards.
Also, depending on your input file format, you may be able to have textscan go through the file piece by piece without rewinding at all.
Just a thought. Hope this helps!
Oleg Komarov on 6 May 2011
You're importing the following positions:
You can do the same by importing everything at once and selecting with logical indexing, or can't you (memory problems, other...)?
It will be much fuster and don't underestimate the power of textscan.
You can check memory consumption with:
on win platforms, or use an external memory performance tracker.