Path: news.mathworks.com!not-for-mail
From: "Nick Denman" <ngdenmanNOSPAM@gmail.REMOVETHIS.com>
Newsgroups: comp.soft-sys.matlab
Subject: Re: MATLAB Reshape Challenge: N x d matrix to N/K x d x K matrix
Date: Thu, 13 Nov 2008 01:23:02 +0000 (UTC)
Organization: The MathWorks, Inc.
Lines: 105
Message-ID: <gffvhm$185$1@fred.mathworks.com>
References: <f6776e56-5088-4685-85ee-fa28328610e6@v4g2000yqa.googlegroups.com>
Reply-To: "Nick Denman" <ngdenmanNOSPAM@gmail.REMOVETHIS.com>
NNTP-Posting-Host: webapp-02-blr.mathworks.com
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8bit
X-Trace: fred.mathworks.com 1226539382 1285 172.30.248.37 (13 Nov 2008 01:23:02 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Thu, 13 Nov 2008 01:23:02 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 745609
Xref: news.mathworks.com comp.soft-sys.matlab:500533


mprocopio@gmail.com wrote in message <f6776e56-5088-4685-85ee-fa28328610e6@v4g2000yqa.googlegroups.com>...
> Hi folks,
> 
> I'm working on parallelizing some machine learning code in MATLAB. I'm
> using the Parallel Computing Toolbox and the parfor construct;
> therefore, I have certain restrictions on how I must "slice" into
> certain data structures accessed within the parfor (parallel for)
> loop.
> 
> Basically, I need to reshape my training data from an (N x d) two-
> dimensional matrix to a fully "slicable" (N/K x d x K) matrix, where K
> is the number of available parallel threads (which corresponds to the
> number of "slicable" input data partitions). Here, I would slice among
> the third dimension, i.e., partitioned_data
> (:,:,parallel_loop_index_i).
> 
> (For now, assume N is a multiple of K.)
> 
> I'm usually very good with reshape, permute, shiftdim, etc., but I am
> having trouble making this work nicely.
> 
> I could hack in something like slicable_data_partitions = cat(3,
> manual_partition_1, manual_partition_2, ..., manual_partition_K) but
> that's horrible and I was hoping for an efficient one-liner with some
> combination of reshape, permute, etc. Performance is important, as
> this will be fairly large scale, i.e., numel(data) on the order of
> 10^7.
> 
> Note I can actually get reshape to return output int he desired
> dimension, however, the d data elements along the second dimension in
> each of the N rows are no longer in the correct order.
> 
> Detailed example of desired output:
> 
> [N x d, 8 x 3]:
> 
> input_data = reshape(1:24, 3, 8)'   % Note transpose
> 
> input_data =
> 
>      1     2     3
>      4     5     6
>      7     8     9
>     10    11    12
>     13    14    15
>     16    17    18
>     19    20    21
>     22    23    24
> 
> 
> For K = 4, [N/K x d x K, 2 x 3 x 4]
> 
> desired_transformed_data =
> 
> ans(:,:,1) =
> 
>      1     2    3
>      4    5    6
> 
> ans(:,:,2) =
> 
>     7    8     9
>     10     11    12
> 
> ans(:,:,3) =
> 
>     13    14     15
>     16    17     18
> 
> ans(:,:,4) =
> 
>      19    20    21
>     22    23    24
> 
> 
> Any help would be greatly appreciated.
> 
> On a final note, from a memory performance standpoint: any performance
> difference in the dimension along which the data is sliced? I.e.,
> performance difference for [N/K x d x K] and slicing along the third
> dimension, versus [K x d x N/K] and slicing along the first dimension?
> 
> 
> With thanks,
> 
> --Mike

Hi Mike

Try the following:

>> input_data = [1 2 3
       4 5 6
       7 8 9
       10 11 12
       13 14 15
       16 17 18
       19 20 21
       22 23 24];
>> [N d] = size(input_data);
>> K = 4;
>> permuted_data = permute(reshape(permute(input_data,[2 1]),[d N/K K]),[2 1 3])

HTH,
Nick