Matrix Buffer

Buffer Class to efficiently store blocks of data.

Description

This class implements a very simple but efficient way to temporarily store data.

This tool was created to store simulation data of a switching control system that is returned by the ode functions.

This class is NOT intended to store large numbers of small data chunks, for example single vectors. This class is intended to store an unknown number of matrices that match in one dimension. On read-out, the stored object are concatenated in one direction.

To store a large number of small data elements, preallocate memory by creating a matrix to hold several data elements. Store this matrix in the buffer class when filled and create a new one. Adaption of the size of the temporary
variable, for example doubling of the size with an upper bound, is a compromise between memory overhead and execution time.

It was kindly expressed by John D'Errico, that his tool (file-id 8334) solves a similar task.

methods of class buffer:
constructor - creates empty buffer
store - stores a matrix in the buffer
flush - reads out the buffer concatenating the elements in rows (default) or columns.

example:

buf = buffer
buf = store(buf,eye(3))
buf = store(buf,eye(3))
flush(buf)
flush(buf,1)

• MATLAB release: MATLAB 7.2 (R2006a)

[Jan Wolff]

Comment to the review from John D'Errico:

Thx for your constructive remarks. I added the help as requested.

Your growdata function is quite nice, but serves a different purpose. The
advantage of the buffer class is to be able to store matrices which have to
match in only one dimension. Storing a matrix or multiple vectors at a time is
not possible with growdata.

Try doing your benchmark with larger amounts of data. The buffer class will
have the same execution time like with small data chunks. The execution time
of growdata increases.

CODE:
function bench

bench(1,3,25000);
%bench(100,100,25000);
bench(1,300,250);

return
%%%%%%%%%%%%%%%%%%%%
function bench(M,N,iter)
d=reshape([1:M*N],M,N);
disp(['Parameters: iterations=' num2str(iter) ' chunk size=' num2str(N*M)])
disp buffer
buf=buffer; tic,for i=1:iter,buf=store(buf,d);end,toc
clear buf;
disp growdata
funh = growdata2;tic,for i=1:iter,funh(d);end,toc
clear funh;
return

---------
>> test
Parameters: iterations=25000 chunk size=3
buffer
Elapsed time is 7.491122 seconds.
growdata
Elapsed time is 1.268556 seconds.
Parameters: iterations=250 chunk size=300
buffer
Elapsed time is 0.007169 seconds.
growdata
Elapsed time is 0.172727 seconds.

[John D'Errico]

Rule number 1: If you place something on the file exchange that does exactly the same thing as an existing code that is already there, at least do a better job at what it does. This fails rule number 1, and fails it miserably. It uses 4 functions to do the job that growdata or growdata2 do with one single function. It is much slower. Test it out:

% initialization
b = buffer
b: buffer with 0 elements.
{}

% growth
tic,for i=1:25000,b=store(b,rand(1,3));end,toc
Elapsed time is 104.959756 seconds.

% unpacking
tic,data = flush(b,1);toc
Elapsed time is 0.145784 seconds.
size(data)
ans =
25000 3

In comparison, try growdata2.

% initialization step
funh = growdata2;
% growth
tic,for i=1:25000,funh(rand(1,3));end,toc
Elapsed time is 3.101270 seconds.

% the unpacking step
tic,data = funh();toc
Elapsed time is 0.011790 seconds.

size(data)
ans =
25000 3

An interesting feature of these buffer tools is the time required to run them grows quadratically. Store twice as much, and the time grows by a factor of 4. The growdata tools are much closer to linear in the time required. Again, a test will show this nicely.

b = buffer;
tic,for i=1:5000,b=store(b,rand(1,3));end,toc
Elapsed time is 2.621039 seconds.

b = buffer;
tic,for i=1:10000,b=store(b,rand(1,3));end,toc
Elapsed time is 13.128627 seconds.

funh = growdata2;
tic,for i=1:25000,funh(rand(1,3));end,toc
Elapsed time is 3.108313 seconds.

funh = growdata2;
tic,for i=1:50000,funh(rand(1,3));end,toc
Elapsed time is 6.182426 seconds.

Rule number 2: Document your code. This set of tools has absolutely no help. You need to edit the functions just to figure out how to use them. When you use the help facility, you get this very unenlightening piece of information:

help buffer

buffer class
author: jan wolff, 2007

Rule number 3: At the very least, acknowledge the existing code. This submission does not do so.

http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId=8334&objectType=file