Can I speed up this text file parsing code?
6 views (last 30 days)
Show older comments
I am reading in a large text files (~4MB) containing one column of of different readings. The part of my code that sorts the column of 5 reading types in to 5 different vectors is taking ~2 minutes to run. Is there a code change I can make to speed this up?
fid=fopen(fileName);
C = textscan(fid, '%f');
rawData=cell2mat(C);
toc
j=1;
k=1;
display('parsing data');tic
speed=[];voltage=[];current=[];force=[];displacement=[];
while j<=length(rawData)
displacement(k)=rawData(j);
force(k)=rawData(j+1);
current(k)=rawData(j+2) ;
voltage(k)=rawData(j+3);
speed(k)=rawData(j+4);
j=j+5;
k=k+1;
end
fclose(fid);
0 Comments
Accepted Answer
Jan
on 20 Jul 2011
Or try FSCANF instead:
Data = fscanf(fid, '%f', Inf);
fclose(fid);
len = numel(Data);
displacement = Data(1:5:len);
force = Data(2:5:len);
current = Data(3:5:len);
voltage = Data(4:5:len);
speed = Data(5:5:len);
More Answers (1)
Sean de Wolski
on 20 Jul 2011
Preallocate, preallocate, preallocate!!!!!!! (just like m-lint (the orange squiggly line) is telling you)
v = 1:5:length(rawdata);
n = length(v);
force = zeros(n,1);
displacement = zeros(n,1);
current = zeros(n,1);
voltage = zeros(n,1);
speed = zeros(n,1);
for jj = v
displacement(k)=rawData(jj);
force(k)=rawData(jj+1);
current(k)=rawData(jj+2) ;
voltage(k)=rawData(jj+3);
speed(k)=rawData(jj+4);
k=k+1;
end
or even better just use reshape:
rawData = reshape(rawData,[],5)';
displacement = rawData(:,1);
force = rawData(:,2);
current = rawData(:,3);
voltage = rawData(:,4);
speed = rawData(:,5);
3 Comments
Sean de Wolski
on 20 Jul 2011
There's no reason for cells since all data is the same size. The reshape command will give you an nx5 matrix where each column is what you want...
See Also
Categories
Find more on Data Import and Export in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!