Path: news.mathworks.com!not-for-mail
From: "David " <dbecker@ipg.tu-darmstadt.de>
Newsgroups: comp.soft-sys.matlab
Subject: Re: efficiently fwrite a cell-array of matrices
Date: Tue, 22 Jan 2013 10:31:08 +0000 (UTC)
Organization: TU Darmstadt
Lines: 85
Message-ID: <kdlppc$ijj$1@newscl01ah.mathworks.com>
References: <kdjob0$98l$1@newscl01ah.mathworks.com> <kdkbe8$ik3$1@newscl01ah.mathworks.com>
Reply-To: "David " <dbecker@ipg.tu-darmstadt.de>
NNTP-Posting-Host: www-00-blr.mathworks.com
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: newscl01ah.mathworks.com 1358850668 19059 172.30.248.45 (22 Jan 2013 10:31:08 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Tue, 22 Jan 2013 10:31:08 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 3950903
Xref: news.mathworks.com comp.soft-sys.matlab:787065

Hi Jan and Bruno, hi everyone,

thanks for your answers so far!

After thinking for one more day I found some solution to my problem, which is more efficient than the for-looping (see below) as long as the matrices in A are "small" (up to ~12x12).
I will post my solution here:

My idea sounds complicated first, but turns out to be efficient for small matrix-sizes:
1) initialize a zeros-output array "outblock" with the same number of elements, that A has (for my example: 10)
2) find unique sizes of the matrices in A and the corresponding index-mapping (using "unique"), and then for each unique size:

3a) build up an index-set for the output elements for the entries in A with the same size using an efficient runLengthDecoding (you could use "rude" by "us" for that)
3b) copy all data from all matrices of same size from A to the correct output-positions in "outblock"

4) write out the ouput-datablock to the file (so, overall, only fwrite-call is required.


here is the code for that function:
============================================
function fwriteCell( fid, C, dataType )

if nargin<3
    dataType = class(C{1});
end;

[heights,widths] = cellfun(@size,C);
sizeHW = [heights',widths'];

[uniqueSizes,~,map] = unique(sizeHW,'rows');

uniqueNumel = prod(uniqueSizes,2);
totalNumel = sum(uniqueNumel(map));

numberOfDifferentSizes = length(uniqueNumel);
outind = runLengthDecoding(uniqueNumel(map),map); %you may use "rude" for this

outblock = zeros(totalNumel,1);

for i=1:numberOfDifferentSizes
    tmp = C(map==i);
    tmp = horzcat(tmp{:});
    tmp = reshape(tmp,1,numel(tmp));
    outblock(outind==i)=tmp;    % this need most of the runtime for large matrices
end;

fwrite(fid,outblock,dataType);
end
============================================

additionally, here is code for evaluating the efficiency, compared to the naive for-looping:

=================================
%create the array for testing purposes
N=100000;
C = cell(1,N);
for i=1:N
    C{i} = randi(1000,randi(3),randi(3)-1);
end;

fid = fopen('myfile.bin','w');

% variant 1: the complex version using runlength-decoding
tic
fwriteCell(fid,C);
toc

%variant 2: for-loop
tic
for i=1:N
    fwrite(fid,C{i});
end;
toc

fclose(fid);
=================================

for me, the result is:
Elapsed time is 0.202902 seconds.
Elapsed time is 1.352630 seconds.

so, for small matrices, as in this example (1x0 - 3x2), fwriteCell() outperforms the naive implementation by a factor of almost 7, BUT: the larger the matrices in A get, the closer the runtimes will get, and eventually (~12x12-matrices and larger), the naive version will outperform fwriteCell()! For my application, this is sufficient for now, as matrices are small

Maybe somebody has another idea or a hint how I might improve my solution. The row "outblock(outind==i)=tmp;" almost 100% of that functions runtime for large matrices and large N, probably due to the (stupid) copying of data in the memory... I'm not shure if I can improve this in any way...

best regards, David.