From: "David " <>
Newsgroups: comp.soft-sys.matlab
Subject: Re: efficiently fwrite a cell-array of matrices
Date: Tue, 22 Jan 2013 10:31:08 +0000 (UTC)
Organization: TU Darmstadt
Lines: 85
Message-ID: <kdlppc$ijj$>
References: <kdjob0$98l$> <kdkbe8$ik3$>
Reply-To: "David " <>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: 1358850668 19059 (22 Jan 2013 10:31:08 GMT)
NNTP-Posting-Date: Tue, 22 Jan 2013 10:31:08 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 3950903
Xref: comp.soft-sys.matlab:787065

Hi Jan and Bruno, hi everyone,

thanks for your answers so far!

After thinking for one more day I found some solution to my problem, which is more efficient than the for-looping (see below) as long as the matrices in A are "small" (up to ~12x12).
I will post my solution here:

My idea sounds complicated first, but turns out to be efficient for small matrix-sizes:
1) initialize a zeros-output array "outblock" with the same number of elements, that A has (for my example: 10)
2) find unique sizes of the matrices in A and the corresponding index-mapping (using "unique"), and then for each unique size:

3a) build up an index-set for the output elements for the entries in A with the same size using an efficient runLengthDecoding (you could use "rude" by "us" for that)
3b) copy all data from all matrices of same size from A to the correct output-positions in "outblock"

4) write out the ouput-datablock to the file (so, overall, only fwrite-call is required.

here is the code for that function:
function fwriteCell( fid, C, dataType )

if nargin<3
    dataType = class(C{1});

[heights,widths] = cellfun(@size,C);
sizeHW = [heights',widths'];

[uniqueSizes,~,map] = unique(sizeHW,'rows');

uniqueNumel = prod(uniqueSizes,2);
totalNumel = sum(uniqueNumel(map));

numberOfDifferentSizes = length(uniqueNumel);
outind = runLengthDecoding(uniqueNumel(map),map); %you may use "rude" for this

outblock = zeros(totalNumel,1);

for i=1:numberOfDifferentSizes
    tmp = C(map==i);
    tmp = horzcat(tmp{:});
    tmp = reshape(tmp,1,numel(tmp));
    outblock(outind==i)=tmp;    % this need most of the runtime for large matrices


additionally, here is code for evaluating the efficiency, compared to the naive for-looping:

%create the array for testing purposes
C = cell(1,N);
for i=1:N
    C{i} = randi(1000,randi(3),randi(3)-1);

fid = fopen('myfile.bin','w');

% variant 1: the complex version using runlength-decoding

%variant 2: for-loop
for i=1:N


for me, the result is:
Elapsed time is 0.202902 seconds.
Elapsed time is 1.352630 seconds.

so, for small matrices, as in this example (1x0 - 3x2), fwriteCell() outperforms the naive implementation by a factor of almost 7, BUT: the larger the matrices in A get, the closer the runtimes will get, and eventually (~12x12-matrices and larger), the naive version will outperform fwriteCell()! For my application, this is sufficient for now, as matrices are small

Maybe somebody has another idea or a hint how I might improve my solution. The row "outblock(outind==i)=tmp;" almost 100% of that functions runtime for large matrices and large N, probably due to the (stupid) copying of data in the memory... I'm not shure if I can improve this in any way...

best regards, David.