Path: news.mathworks.com!not-for-mail From: "David " <dbecker@ipg.tu-darmstadt.de> Newsgroups: comp.soft-sys.matlab Subject: Re: efficiently fwrite a cell-array of matrices Date: Tue, 22 Jan 2013 10:31:08 +0000 (UTC) Organization: TU Darmstadt Lines: 85 Message-ID: <kdlppc$ijj$1@newscl01ah.mathworks.com> References: <kdjob0$98l$1@newscl01ah.mathworks.com> <kdkbe8$ik3$1@newscl01ah.mathworks.com> Reply-To: "David " <dbecker@ipg.tu-darmstadt.de> NNTP-Posting-Host: www-00-blr.mathworks.com Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Trace: newscl01ah.mathworks.com 1358850668 19059 172.30.248.45 (22 Jan 2013 10:31:08 GMT) X-Complaints-To: news@mathworks.com NNTP-Posting-Date: Tue, 22 Jan 2013 10:31:08 +0000 (UTC) X-Newsreader: MATLAB Central Newsreader 3950903 Xref: news.mathworks.com comp.soft-sys.matlab:787065 Hi Jan and Bruno, hi everyone, thanks for your answers so far! After thinking for one more day I found some solution to my problem, which is more efficient than the for-looping (see below) as long as the matrices in A are "small" (up to ~12x12). I will post my solution here: My idea sounds complicated first, but turns out to be efficient for small matrix-sizes: 1) initialize a zeros-output array "outblock" with the same number of elements, that A has (for my example: 10) 2) find unique sizes of the matrices in A and the corresponding index-mapping (using "unique"), and then for each unique size: 3a) build up an index-set for the output elements for the entries in A with the same size using an efficient runLengthDecoding (you could use "rude" by "us" for that) 3b) copy all data from all matrices of same size from A to the correct output-positions in "outblock" 4) write out the ouput-datablock to the file (so, overall, only fwrite-call is required. here is the code for that function: ============================================ function fwriteCell( fid, C, dataType ) if nargin<3 dataType = class(C{1}); end; [heights,widths] = cellfun(@size,C); sizeHW = [heights',widths']; [uniqueSizes,~,map] = unique(sizeHW,'rows'); uniqueNumel = prod(uniqueSizes,2); totalNumel = sum(uniqueNumel(map)); numberOfDifferentSizes = length(uniqueNumel); outind = runLengthDecoding(uniqueNumel(map),map); %you may use "rude" for this outblock = zeros(totalNumel,1); for i=1:numberOfDifferentSizes tmp = C(map==i); tmp = horzcat(tmp{:}); tmp = reshape(tmp,1,numel(tmp)); outblock(outind==i)=tmp; % this need most of the runtime for large matrices end; fwrite(fid,outblock,dataType); end ============================================ additionally, here is code for evaluating the efficiency, compared to the naive for-looping: ================================= %create the array for testing purposes N=100000; C = cell(1,N); for i=1:N C{i} = randi(1000,randi(3),randi(3)-1); end; fid = fopen('myfile.bin','w'); % variant 1: the complex version using runlength-decoding tic fwriteCell(fid,C); toc %variant 2: for-loop tic for i=1:N fwrite(fid,C{i}); end; toc fclose(fid); ================================= for me, the result is: Elapsed time is 0.202902 seconds. Elapsed time is 1.352630 seconds. so, for small matrices, as in this example (1x0 - 3x2), fwriteCell() outperforms the naive implementation by a factor of almost 7, BUT: the larger the matrices in A get, the closer the runtimes will get, and eventually (~12x12-matrices and larger), the naive version will outperform fwriteCell()! For my application, this is sufficient for now, as matrices are small Maybe somebody has another idea or a hint how I might improve my solution. The row "outblock(outind==i)=tmp;" almost 100% of that functions runtime for large matrices and large N, probably due to the (stupid) copying of data in the memory... I'm not shure if I can improve this in any way... best regards, David.