Path: news.mathworks.com!newsfeed-00.mathworks.com!panix!news.maxwell.syr.edu!news-rtr.nyroc.rr.com!news-out.nyroc.rr.com!twister.nyroc.rr.com.POSTED!53ab2750!not-for-mail
From: John D'Errico <woodchips@rochester.rr.com>
Newsgroups: comp.soft-sys.matlab
Mail-Copies-To: nobody
Subject: Re: growing array in Matlab? (new code)
Organization: I'm not really very organized
References: <ddvkd9$7ar$1@ruby.cit.cornell.edu> <woodchips-EBB412.12532017082005@syrcnyrdrs-02-ge0.nyroc.rr.com> <de02f3$k3c$1@ruby.cit.cornell.edu> <woodchips-EDAAE9.21275017082005@syrcnyrdrs-01-ge0.nyroc.rr.com> <MPG.1d6e3a12f3c423c98992c@news.mathworks.com> <woodchips-E55FA6.08284918082005@syrcnyrdrs-02-ge0.nyroc.rr.com>
User-Agent: MT-NewsWatcher/3.4 (PPC Mac OS X)
Message-ID: <woodchips-7262EC.11380018082005@syrcnyrdrs-03-ge0.nyroc.rr.com>
Lines: 221
Date: Thu, 18 Aug 2005 15:38:00 GMT
NNTP-Posting-Host: 66.66.154.144
X-Complaints-To: abuse@rr.com
X-Trace: twister.nyroc.rr.com 1124379480 66.66.154.144 (Thu, 18 Aug 2005 11:38:00 EDT)
NNTP-Posting-Date: Thu, 18 Aug 2005 11:38:00 EDT
Xref: news.mathworks.com comp.soft-sys.matlab:296719


In article 
<woodchips-E55FA6.08284918082005@syrcnyrdrs-02-ge0.nyroc.rr.com>,
 John D'Errico <woodchips@rochester.rr.com> wrote:

> In article <MPG.1d6e3a12f3c423c98992c@news.mathworks.com>,
>  Loren Shure <loren.shure@mathworks.com> wrote:
> 
> > I have not tried anything here and don't know the performance 
> > implications, but instead of persistent, what if you made your grow 
> > array function return a handle to a nested function which stored the 
> > data there?  That way, you could have more than instance.  Just a 
> > thought...
> 
> Hi Loren,
> 
> Cute idea. I don't have enough experience with
> function handles that I'd have thought of it myself.
> Its like those anomalous functions. I'm still learning
> to use them properly.
> 
> Thanks,
> John

It is an interesting idea. The code is below. It also
runs in linear time, as did my last attempt. But it is
also sadly twice as slow as was that last code. The
profile tool suggests that 47% of the time was spent
in function overhead. This explains where the doubled
time came from.

John


function funH=growdata2(appendmode)
% incremental growth of an array by appending rows (or columns)
%
% usage always involves 3 steps
%  (initial call):   fun1=growdata;
%  (growth call):    fun1(newdata)
%  (final call):     A = fun1();
% 
% arguments: (input)
%  appendmode - optional argument, which specifies how to append.
%               appendmode == 'rows' --> append new data as rows
%                  of the final array.
%
%               appendmode == 'columns' --> append new data as
%                  columns of the final array.
%
%               DEFAULT: 'rows'
%
% arguments: (output)
%  funH       - function handle which will be used for the growth
%               and terminal calls to unpack the grown array of
%               data.
%
%
% Example usage 1:
%  
%  fun1 = growdata;
%  for i = 1:100000
%    fun1(i)
%  end
%  A = fun1();
%
% A will have size [100000,1]
%
%
% Example usage 2:
%
%  Note: growdata uses a function handle to a nested function,
%  so multiple arrays may be grown at the same time, simply by
%  assigning a different function name on the first call to
%  growdata. We can also change the appending style for each
%  instance.
%
%  fun1 = growdata;            % append as new rows by default
%  fun2 = growdata('columns'); % append as new columns
%  fun3 = growdata('rows');    % append as new rows
%  for i = 1:100000
%    fun1(rand(2,3))
%    fun2(sin(rand(2,1)))
%    fun3(round(rand(1)*2),2)
%  end
%  A = fun1();
%  B = fun2();
%  C = fun3();
%
% In this case, A will have final size [200000,3], B will
% have size [2, 100000], and C will have a ramdom number of
% rows, with 2 columns.
 
 
% when growdata is called, a function handle to a nested
% function is returned.
% first, set up the variables we will need to remember
 
% check for appendmode
if (nargin<1)|isempty(appendmode)
  appendmode = 'rows';
end
valid = {'rows','columns'};
ind = strmatch(appendmode,valid);
if isempty(ind)
  error 'Appendmode must be ''rows'' or ''columns'' or any contraction'
end
% appendmode == 0 --> append as rows
% appendmode == 1 --> append as columns
appendmode = ind - 1;
 
% default number of rows in each cell
rowcount = 20000;
 
% we don't know how many columns
columncount = NaN;
celldata = {zeros(rowcount,0)};
n = 0;
totalrows = 0;
 
% Return a handle to the function which will actually
% do all the work
funH = @growthfun;
 
% ====================================================
% nested function definition
function A = growthfun(newdata)
 
% is this a growth call, or a final unpacking call?
if (nargout==0) && (nargin>0)
  % its a growth step
  
  % if appendmode says to use columns, then transpose the
  % data, append as rows. we will re-transpose at the very
  % end to undo this.
  if appendmode
    newdata = newdata';
  end
  
  % size of the appending data
  [r,c]=size(newdata);
  
  % was this the first call after initialization?
  if isnan(columncount)
    % first time called, we need to know the number
    % of columns to expect
    columncount = c;
    
    if r < rowcount
      % rowcount is large enough for now
      celldata = {[newdata;zeros(rowcount - r,columncount)]};
      n = r;
      totalrows = r;
    else
      % the first call overwhelmed rowcount, so make it larger
      n = 0
      totalrows = r;
      rowcount = 2*r;
      celldata = {newdata, zeros(rowcount,columncount)}; 
    end
  
  else
    % its an appending call after we have seen some data
    if c ~= columncount
      error '# of columns are incompatible for appending to this data'
    end
    
    % stuff into the last cell
    if (n+r)<rowcount
      celldata{end}(n+(1:r),:) = newdata;
      n = n+r;
      totalrows = totalrows + r;
    elseif (n+r)==rowcount
      % exactly filled that last cell, so add a new
      % (empty) cell on to the end
      celldata{end}(n+(1:r),:) = newdata;
      celldata{end+1} = zeros(rowcount,columncount);
      totalrows = totalrows + r;
      n = 0;
    else
      % we will overfill the last cell
      s = rowcount - n;
      celldata{end}(n+(1:s),:) = newdata;
      celldata{end+1} = newdata((s+1):end,:);
      totalrows = totalrows + r;
      n = size(celldata{end},1);
      
      if n>=rowcount
        % enlarge the cell size
        rowcount = n;
        celldata{end+1} = zeros(rowcount,columncount);
        n = 0;
      end
      
    end
  end
  
elseif (nargin==0)
  % its an unpacking step
  
  % first drop any extraneous rows in the last cell
  celldata{end} = celldata{end}(1:n,:);
  
  % concatenate
  if ~appendmode
    % as rows
    A = cat(1,celldata{:});
  else
    % as columns, but we did it as rows, so transpose
    A = cat(1,celldata{:})';
  end
  
  % and clear the data to return some memory
  clear celldata
  
elseif (nargout>0) && (nargin>0)
  % cannot both append and unpack in the same step.
  error 'Cannot append more data and unpack in the same call'
end
 
end % end nested function
end % end main fun