Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

To resolve issues starting MATLAB on Mac OS X 10.10 (Yosemite) visit: http://www.mathworks.com/matlabcentral/answers/159016

how to initialize an M×N array whose size is unknown in for-loop

Asked by AP on 10 Nov 2011

Hi All,

I am trying to read 150 text files. Each contains the value of x and y. x is the input of an experiment and y is the measured value. Each experiment has different number of data. The number of data cannot be determined until I have read each file.

How should I use the arrays which store x and y in a for-loop efficiently to speed up the process of reading? How should I initialize them while each experiment has different number of elements? I tried to use a code similar to the following code.

x=zeros(??,150);
y=zeros(??,150);
for i=1:150
   fid=fopen(fname);
   data=textscan(fid,'%f %f');
   x(??,i)=data{1}
   y(??,i)=data{2}
end

By ?? I mean I don't exactly know what to use. Could someone please help me?

Thanks.

0 Comments

AP

Tags

Products

No products are associated with this question.

4 Answers

Answer by Walter Roberson on 10 Nov 2011
Accepted answer
x = cell(150,1);
y = cell(150,1);
for K = 1 : 150
  fid=fopen(fname);
  data = textscan(fid, '%f %f');
  x{K} = data{1};
  y{K} = data{2};
end

You can always put them in to a matrix after you have collected them all. If you do that, you will need to decide how you want to pad the shorter columns.

0 Comments

Walter Roberson
Answer by Bård Skaflestad on 10 Nov 2011

As your data is all numeric, you can do something akin to the following

x = []; y = [];
for i = 1:150,
  % ... like before ...
  x(1:numel(data{1}), i) = data{1};
  y(1:numel(data{2}), i) = data{2};
end

This will reallocate x and y to appropriate sizes whilst reading the input. I'm not, however, advocating this solution due to the reallocation cost. Personally, I'd just use something like

x = cell([1, 150]); y = cell([1, 150]);
for i = 1:150,
  % ... like before ...
  x{i} = data{1};
  y{i} = data{2};
end

although the effectiveness depends greatly on what you intend to do to 'x' and 'y' once the data is in MATLAB.

Sincerely, Bård Skaflestad

0 Comments

Bård Skaflestad
Answer by Jonathan on 10 Nov 2011

A. Hill,

You can probably get a good estimate of the number of elements by examining the file sizes. Suppose the files are in the directory 'C:\files' and the extensions are '.dat'. You can use something like the following.

dirName = 'C:\files';
listing = dir(fullfile(dirName, '*.dat'));
sizeAllFiles = sum([listing.bytes]);
sizeOneEntry = 8; % or some more appropriate number.
numRowsAll = sizeAllFiles / sizeOneEntry;
x=zeros(numRowsAll,150);
y=zeros(numRowsAll,150);
iRow = 1;
for i=1:150
   fid  = fopen(fname);
   data = textscan(fid,'%f %f');
   numRowsOne = size(data{1},1);
   x(iRow:iRow+numRowsOne,i) = data{1};
   y(iRow:iRow+numRowsOne,i) = data{2};
   iRow = iRow + numRowsOne;
end
x = x(1:iRow-1);
y = y(1:iRow-1);

Let me know if this helps.

~Jonathan

0 Comments

Jonathan
Answer by Naz on 10 Nov 2011

One but not very intelligent way to do it is allocate more memory that you need. You probably have an idea of what what would be a maximum number of values in any of your txt. file. If you know that it should be less than a thousand, allocate a thousand rows (or 2000...). Also, you can fill the matrix with distinct value that you know does not exist in your data:

x=zeros(150,1000); % 1000 columns of x data
y=zeros(150,1000);
x(x==0)=pi;  % change your zeros to pi or (-1) to have it as a mark of the end of data (similar how they use 0 at the end of the string in C)
y(y==0)=pi;

then perform the same loop as you wrote above. When you read data, you know where to stop since there is a mark.

1 Comment

Jonathan on 11 Nov 2011

I use a similar technique often. I would initialize x as "x = nan(150, 1000);". This way, whenever the data is expected to contain no nans, there is no need to pick another value (such as pi or -1) as a default value. If using pi as the default value is desirable, I use "x = pi*ones(150,1000);".

Naz

Contact us