Thread Subject: Reshaping Dataset

Subject: Reshaping Dataset

From: Jeremy Bing

Date: 25 Jun, 2009 19:31:01

Message: 1 of 17

Hi,

I have data in the form of the following.

X Y Z X Y Z >>
X Y Z X Y Z
X Y Z X Y Z
X Y Z X Y Z
...

But i need the data to be:

X Y Z
X Y Z
X Y Z

Do i used a case statement or a nested forloop to reshape the data? The dataset is quite large.

Subject: Reshaping Dataset

From: Jos

Date: 25 Jun, 2009 20:55:02

Message: 2 of 17

"Jeremy Bing" <jeremy@mytrashmail.com> wrote in message <h20j9k$9ok$1@fred.mathworks.com>...
> Hi,
>
> I have data in the form of the following.
>
> X Y Z X Y Z >>
> X Y Z X Y Z
> X Y Z X Y Z
> X Y Z X Y Z
> ...
>
> But i need the data to be:
>
> X Y Z
> X Y Z
> X Y Z
>
> Do i used a case statement or a nested forloop to reshape the data? The dataset is quite large.

If I understand this correctly, you need column 4-6 to appear below column 1-3?

B = [A(:,1:3) ; A(:,4:6)] ;

hth
Jos

Subject: Reshaping Dataset

From: Jeremy Bing

Date: 25 Jun, 2009 21:09:02

Message: 3 of 17

"Jos " <#10584@fileexchange.com> wrote in message <h20o76$864$1@fred.mathworks.com>...
> "Jeremy Bing" <jeremy@mytrashmail.com> wrote in message <h20j9k$9ok$1@fred.mathworks.com>...
> > Hi,
> >
> > I have data in the form of the following.
> >
> > X Y Z X Y Z >>
> > X Y Z X Y Z
> > X Y Z X Y Z
> > X Y Z X Y Z
> > ...
> >
> > But i need the data to be:
> >
> > X Y Z
> > X Y Z
> > X Y Z
> >
> > Do i used a case statement or a nested forloop to reshape the data? The dataset is quite large.
>
> If I understand this correctly, you need column 4-6 to appear below column 1-3?
>
> B = [A(:,1:3) ; A(:,4:6)] ;
>
> hth
> Jos

Hi,

I need column(4,1:3) put underneath 1:3. i used this method:
 
r2 = numel(r(:,1));

for iii = 1:r2
kdkrm(1:21,1:3) = [r(iii,1:3);r(iii,4:6);r(iii,7:9);r(iii,10:12);r(iii,13:15);r(iii,16:18);r(iii,19:21);r(iii,22:24);r(iii,25:27);r(iii,28:30);r(iii,31:33);r(iii,34:36);r(iii,37:39);r(iii,40:42);r(iii,43:45);r(iii,46:48);r(iii,49:51);r(iii,52:54);r(iii,55:57);r(iii,58:60);r(iii,61:63)];
end

but it is quite long and this only deals with the first row. all other rows under also need to go through this so i think i need a nested forloop.

Thanks

Jeremy

Subject: Reshaping Dataset

From: Nathan

Date: 25 Jun, 2009 21:22:57

Message: 4 of 17

On Jun 25, 2:09 pm, "Jeremy Bing" <jer...@mytrashmail.com> wrote:
> "Jos " <#10...@fileexchange.com> wrote in message <h20o76$86...@fred.mathworks.com>...
> > "Jeremy Bing" <jer...@mytrashmail.com> wrote in message <h20j9k$9o...@fred.mathworks.com>...
> > > Hi,
>
> > > I have data in the form of the following.
>
> > > X Y Z X Y Z  >>
> > > X Y Z X Y Z
> > > X Y Z X Y Z
> > > X Y Z X Y Z
> > > ...
>
> > > But i need the data to be:
>
> > > X Y Z
> > > X Y Z
> > > X Y Z
>
> > > Do i used a case statement or a nested forloop to reshape the data? The dataset is quite large.
>
> > If I understand this correctly, you need column 4-6 to appear below column 1-3?
>
> > B = [A(:,1:3) ; A(:,4:6)] ;
>
> > hth
> > Jos
>
> Hi,
>
> I need column(4,1:3) put underneath 1:3. i used this method:
>
> r2 = numel(r(:,1));
>
> for iii = 1:r2
> kdkrm(1:21,1:3) = [r(iii,1:3);r(iii,4:6);r(iii,7:9);r(iii,10:12);r(iii,13:15);r(iii,16:18);r(iii,19:21);r(iii,22:24);r(iii,25:27);r(iii,28:30);r(iii,31:33);r(iii,34:36);r(iii,37:39);r(iii,40:42);r(iii,43:45);r(iii,46:48);r(iii,49:51);r(iii,52:54);r(iii,55:57);r(iii,58:60);r(iii,61:63)];
> end
>
> but it is quite long and this only deals with the first row. all other rows under also need to go through this so i think i need a nested forloop.
>
> Thanks
>
> Jeremy

How about...
B = [];
for ii = 1:length(A(:,1))
for jj=1:(length(A)/3)
B = [B;A(1,3*(jj-1)+1:3*(jj-1)+3)];
end
end

-Nathan

Subject: Reshaping Dataset

From: James Tursa

Date: 25 Jun, 2009 22:00:18

Message: 5 of 17

"Jeremy Bing" <jeremy@mytrashmail.com> wrote in message <h20j9k$9ok$1@fred.mathworks.com>...
> Hi,
>
> I have data in the form of the following.
>
> X Y Z X Y Z >>
> X Y Z X Y Z
> X Y Z X Y Z
> X Y Z X Y Z
> ...
>
> But i need the data to be:
>
> X Y Z
> X Y Z
> X Y Z
>
> Do i used a case statement or a nested forloop to reshape the data? The dataset is quite large.

reshape(A',3,[])'

But this has a lot of data movement, the two transposes cause the entire dataset to be moved twice. If the time consumption is still too large then we could resort to a c-mex routine and limit the data movement to once, speeding things up.

James Tursa

Subject: Reshaping Dataset

From: TideMan

Date: 25 Jun, 2009 22:55:07

Message: 6 of 17

On Jun 26, 7:31 am, "Jeremy Bing" <jer...@mytrashmail.com> wrote:
> Hi,
>
> I have data in the form of the following.
>
> X Y Z X Y Z  >>
> X Y Z X Y Z
> X Y Z X Y Z
> X Y Z X Y Z
> ...
>
> But i need the data to be:
>
> X Y Z
> X Y Z
> X Y Z
>
> Do i used a case statement or a nested forloop to reshape the data? The dataset is quite large.

How about:
A=rand(6,6);
B=zeros(12,3);
B(1:2:end,:)=A(:,1:3);
B(2:2:end,:)=A(:,4:6);

Subject: Reshaping Dataset

From: James Tursa

Date: 25 Jun, 2009 23:12:01

Message: 7 of 17

"James Tursa" <aclassyguywithaknotac@hotmail.com> wrote in message <h20s1i$evc$1@fred.mathworks.com>...
> "Jeremy Bing" <jeremy@mytrashmail.com> wrote in message <h20j9k$9ok$1@fred.mathworks.com>...
> > Hi,
> >
> > I have data in the form of the following.
> >
> > X Y Z X Y Z >>
> > X Y Z X Y Z
> > X Y Z X Y Z
> > X Y Z X Y Z
> > ...
> >
> > But i need the data to be:
> >
> > X Y Z
> > X Y Z
> > X Y Z
> >
> > Do i used a case statement or a nested forloop to reshape the data? The dataset is quite large.
>
> reshape(A',3,[])'
>
> But this has a lot of data movement, the two transposes cause the entire dataset to be moved twice. If the time consumption is still too large then we could resort to a c-mex routine and limit the data movement to once, speeding things up.
>
> James Tursa

FYI, here is a bare-bones (no argument checking) c-mex file to do the same job. I get about a 20-30% speed improvement with this routine over the reshape method shown above. This may or may not be significant to you.

James Tursa

-----------------------------------

#include "mex.h"
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
    mwSize m, n, i, numel;
    double *target, *source1, *source2;
    
    m = mxGetM(prhs[0]);
    n = mxGetN(prhs[0]);
    numel = m * n / 2;
    plhs[0] = mxCreateDoubleMatrix(2*m, n/2, mxREAL);
    target = mxGetPr(plhs[0]);
    source1 = mxGetPr(prhs[0]);
    source2 = source1 + numel;
    for( i=0; i<numel; i++ ) {
        *target++ = *source1++;
        *target++ = *source2++;
    }
}

Subject: Reshaping Dataset

From: Jeremy Bing

Date: 26 Jun, 2009 16:49:02

Message: 8 of 17

"James Tursa" <aclassyguywithaknotac@hotmail.com> wrote in message <h21081$dg0$1@fred.mathworks.com>...
> "James Tursa" <aclassyguywithaknotac@hotmail.com> wrote in message <h20s1i$evc$1@fred.mathworks.com>...
> > "Jeremy Bing" <jeremy@mytrashmail.com> wrote in message <h20j9k$9ok$1@fred.mathworks.com>...
> > > Hi,
> > >
> > > I have data in the form of the following.
> > >
> > > X Y Z X Y Z >>
> > > X Y Z X Y Z
> > > X Y Z X Y Z
> > > X Y Z X Y Z
> > > ...
> > >
> > > But i need the data to be:
> > >
> > > X Y Z
> > > X Y Z
> > > X Y Z
> > >
> > > Do i used a case statement or a nested forloop to reshape the data? The dataset is quite large.
> >
> > reshape(A',3,[])'
> >
> > But this has a lot of data movement, the two transposes cause the entire dataset to be moved twice. If the time consumption is still too large then we could resort to a c-mex routine and limit the data movement to once, speeding things up.
> >
> > James Tursa
>
> FYI, here is a bare-bones (no argument checking) c-mex file to do the same job. I get about a 20-30% speed improvement with this routine over the reshape method shown above. This may or may not be significant to you.
>
> James Tursa
>
> -----------------------------------
>
> #include "mex.h"
> void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
> {
> mwSize m, n, i, numel;
> double *target, *source1, *source2;
>
> m = mxGetM(prhs[0]);
> n = mxGetN(prhs[0]);
> numel = m * n / 2;
> plhs[0] = mxCreateDoubleMatrix(2*m, n/2, mxREAL);
> target = mxGetPr(plhs[0]);
> source1 = mxGetPr(prhs[0]);
> source2 = source1 + numel;
> for( i=0; i<numel; i++ ) {
> *target++ = *source1++;
> *target++ = *source2++;
> }
> }

Hi,

Thanks for the replies.

I'm not sure I understand the code. I have been trying to do it. My data has 63 columns and 294 rows however this depends on the file being input.
The format for each row is X Y Z X Y Z X Y Z X Y Z >>> 63. I need the all X, Y and Z data to be in one column rather in a row for every Row in the file. I.e I need every 3'd (3,6,9) column to be placed under column 1 (X). Same goes for Y (every second). Then when the first row is complete move to the second row and compile that.

Thanks.

Jeremy

Subject: Reshaping Dataset

From: Nathan

Date: 26 Jun, 2009 16:55:50

Message: 9 of 17

On Jun 26, 9:49 am, "Jeremy Bing" <jer...@mytrashmail.com> wrote:
> "James Tursa" <aclassyguywithakno...@hotmail.com> wrote in message <h21081$dg...@fred.mathworks.com>...
> > "James Tursa" <aclassyguywithakno...@hotmail.com> wrote in message <h20s1i$ev...@fred.mathworks.com>...
> > > "Jeremy Bing" <jer...@mytrashmail.com> wrote in message <h20j9k$9o...@fred.mathworks.com>...
> > > > Hi,
>
> > > > I have data in the form of the following.
>
> > > > X Y Z X Y Z  >>
> > > > X Y Z X Y Z
> > > > X Y Z X Y Z
> > > > X Y Z X Y Z
> > > > ...
>
> > > > But i need the data to be:
>
> > > > X Y Z
> > > > X Y Z
> > > > X Y Z
>
> > > > Do i used a case statement or a nested forloop to reshape the data? The dataset is quite large.
>
> > > reshape(A',3,[])'
>
> > > But this has a lot of data movement, the two transposes cause the entire dataset to be moved twice. If the time consumption is still too large then we could resort to a c-mex routine and limit the data movement to once, speeding things up.
>
> > > James Tursa
>
> > FYI, here is a bare-bones (no argument checking) c-mex file to do the same job. I get about a 20-30% speed improvement with this routine over the reshape method shown above. This may or may not be significant to you.
>
> > James Tursa
>
> > -----------------------------------
>
> > #include "mex.h"
> > void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
> > {
> >     mwSize m, n, i, numel;
> >     double *target, *source1, *source2;
>
> >     m = mxGetM(prhs[0]);
> >     n = mxGetN(prhs[0]);
> >     numel = m * n / 2;
> >     plhs[0] = mxCreateDoubleMatrix(2*m, n/2, mxREAL);
> >     target = mxGetPr(plhs[0]);
> >     source1 = mxGetPr(prhs[0]);
> >     source2 = source1 + numel;
> >     for( i=0; i<numel; i++ ) {
> >         *target++ = *source1++;
> >         *target++ = *source2++;
> >     }
> > }
>
> Hi,
>
> Thanks for the replies.
>
> I'm not sure I understand the code. I have been trying to do it. My data has 63 columns and 294 rows however this depends on the file being input.
> The format for each row is X Y Z X Y Z X Y Z X Y Z >>> 63. I need the all X, Y and Z data to be in one column rather in a row for every Row in the file. I.e I need every 3'd (3,6,9) column to be placed under column 1 (X). Same goes for Y (every second). Then when the first row is complete move to the second row and compile that.
>
> Thanks.
>
> Jeremy

Have you tested any of this code? As far as I can tell, a there are a
lot of ways to do what you are asking here (and it has been shown to
be true).
For example, the code I posted above works for your whole matrix.
Tideman's code can be easily modified to make it work for your whole
matrix.
James Tursa's code (the one liner) will do the reshaping you are
asking with minimal code (but as he said, it has a lot of data
movement which can be a problem with larger matrices).
If you are used to MEX files, you can compile his C code as a MEX file
and run it through that (and I'm assuming it will STILL do exactly
what you asked for).

Why not try these things out and see which one you like best for your
program?
If you don't understand the code, just ask a specific question about
specific code and I'm sure we will be more than willing to let you
know what it does.

-Nathan

Subject: Reshaping Dataset

From: Jeremy Bing

Date: 26 Jun, 2009 17:18:02

Message: 10 of 17

Nathan <ngreco32@gmail.com> wrote in message <54b243f3-71a5-40cd-88b2-0b4b1efca713@l35g2000pra.googlegroups.com>...
> On Jun 26, 9:49?am, "Jeremy Bing" <jer...@mytrashmail.com> wrote:
> > "James Tursa" <aclassyguywithakno...@hotmail.com> wrote in message <h21081$dg...@fred.mathworks.com>...
> > > "James Tursa" <aclassyguywithakno...@hotmail.com> wrote in message <h20s1i$ev...@fred.mathworks.com>...
> > > > "Jeremy Bing" <jer...@mytrashmail.com> wrote in message <h20j9k$9o...@fred.mathworks.com>...
> > > > > Hi,
> >
> > > > > I have data in the form of the following.
> >
> > > > > X Y Z X Y Z ?>>
> > > > > X Y Z X Y Z
> > > > > X Y Z X Y Z
> > > > > X Y Z X Y Z
> > > > > ...
> >
> > > > > But i need the data to be:
> >
> > > > > X Y Z
> > > > > X Y Z
> > > > > X Y Z
> >
> > > > > Do i used a case statement or a nested forloop to reshape the data? The dataset is quite large.
> >
> > > > reshape(A',3,[])'
> >
> > > > But this has a lot of data movement, the two transposes cause the entire dataset to be moved twice. If the time consumption is still too large then we could resort to a c-mex routine and limit the data movement to once, speeding things up.
> >
> > > > James Tursa
> >
> > > FYI, here is a bare-bones (no argument checking) c-mex file to do the same job. I get about a 20-30% speed improvement with this routine over the reshape method shown above. This may or may not be significant to you.
> >
> > > James Tursa
> >
> > > -----------------------------------
> >
> > > #include "mex.h"
> > > void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
> > > {
> > > ? ? mwSize m, n, i, numel;
> > > ? ? double *target, *source1, *source2;
> >
> > > ? ? m = mxGetM(prhs[0]);
> > > ? ? n = mxGetN(prhs[0]);
> > > ? ? numel = m * n / 2;
> > > ? ? plhs[0] = mxCreateDoubleMatrix(2*m, n/2, mxREAL);
> > > ? ? target = mxGetPr(plhs[0]);
> > > ? ? source1 = mxGetPr(prhs[0]);
> > > ? ? source2 = source1 + numel;
> > > ? ? for( i=0; i<numel; i++ ) {
> > > ? ? ? ? *target++ = *source1++;
> > > ? ? ? ? *target++ = *source2++;
> > > ? ? }
> > > }
> >
> > Hi,
> >
> > Thanks for the replies.
> >
> > I'm not sure I understand the code. I have been trying to do it. My data has 63 columns and 294 rows however this depends on the file being input.
> > The format for each row is X Y Z X Y Z X Y Z X Y Z >>> 63. I need the all X, Y and Z data to be in one column rather in a row for every Row in the file. I.e I need every 3'd (3,6,9) column to be placed under column 1 (X). Same goes for Y (every second). Then when the first row is complete move to the second row and compile that.
> >
> > Thanks.
> >
> > Jeremy
>
> Have you tested any of this code? As far as I can tell, a there are a
> lot of ways to do what you are asking here (and it has been shown to
> be true).
> For example, the code I posted above works for your whole matrix.
> Tideman's code can be easily modified to make it work for your whole
> matrix.
> James Tursa's code (the one liner) will do the reshaping you are
> asking with minimal code (but as he said, it has a lot of data
> movement which can be a problem with larger matrices).
> If you are used to MEX files, you can compile his C code as a MEX file
> and run it through that (and I'm assuming it will STILL do exactly
> what you asked for).
>
> Why not try these things out and see which one you like best for your
> program?
> If you don't understand the code, just ask a specific question about
> specific code and I'm sure we will be more than willing to let you
> know what it does.
>
> -Nathan

Ok I will try again.

Subject: Reshaping Dataset

From: Jeremy Bing

Date: 26 Jun, 2009 17:55:04

Message: 11 of 17

"Jeremy Bing" <jeremy@mytrashmail.com> wrote in message <h22vsa$qk2$1@fred.mathworks.com>...
> Nathan <ngreco32@gmail.com> wrote in message <54b243f3-71a5-40cd-88b2-0b4b1efca713@l35g2000pra.googlegroups.com>...
> > On Jun 26, 9:49?am, "Jeremy Bing" <jer...@mytrashmail.com> wrote:
> > > "James Tursa" <aclassyguywithakno...@hotmail.com> wrote in message <h21081$dg...@fred.mathworks.com>...
> > > > "James Tursa" <aclassyguywithakno...@hotmail.com> wrote in message <h20s1i$ev...@fred.mathworks.com>...
> > > > > "Jeremy Bing" <jer...@mytrashmail.com> wrote in message <h20j9k$9o...@fred.mathworks.com>...
> > > > > > Hi,
> > >
> > > > > > I have data in the form of the following.
> > >
> > > > > > X Y Z X Y Z ?>>
> > > > > > X Y Z X Y Z
> > > > > > X Y Z X Y Z
> > > > > > X Y Z X Y Z
> > > > > > ...
> > >
> > > > > > But i need the data to be:
> > >
> > > > > > X Y Z
> > > > > > X Y Z
> > > > > > X Y Z
> > >
> > > > > > Do i used a case statement or a nested forloop to reshape the data? The dataset is quite large.
> > >
> > > > > reshape(A',3,[])'
> > >
> > > > > But this has a lot of data movement, the two transposes cause the entire dataset to be moved twice. If the time consumption is still too large then we could resort to a c-mex routine and limit the data movement to once, speeding things up.
> > >
> > > > > James Tursa
> > >
> > > > FYI, here is a bare-bones (no argument checking) c-mex file to do the same job. I get about a 20-30% speed improvement with this routine over the reshape method shown above. This may or may not be significant to you.
> > >
> > > > James Tursa
> > >
> > > > -----------------------------------
> > >
> > > > #include "mex.h"
> > > > void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
> > > > {
> > > > ? ? mwSize m, n, i, numel;
> > > > ? ? double *target, *source1, *source2;
> > >
> > > > ? ? m = mxGetM(prhs[0]);
> > > > ? ? n = mxGetN(prhs[0]);
> > > > ? ? numel = m * n / 2;
> > > > ? ? plhs[0] = mxCreateDoubleMatrix(2*m, n/2, mxREAL);
> > > > ? ? target = mxGetPr(plhs[0]);
> > > > ? ? source1 = mxGetPr(prhs[0]);
> > > > ? ? source2 = source1 + numel;
> > > > ? ? for( i=0; i<numel; i++ ) {
> > > > ? ? ? ? *target++ = *source1++;
> > > > ? ? ? ? *target++ = *source2++;
> > > > ? ? }
> > > > }
> > >
> > > Hi,
> > >
> > > Thanks for the replies.
> > >
> > > I'm not sure I understand the code. I have been trying to do it. My data has 63 columns and 294 rows however this depends on the file being input.
> > > The format for each row is X Y Z X Y Z X Y Z X Y Z >>> 63. I need the all X, Y and Z data to be in one column rather in a row for every Row in the file. I.e I need every 3'd (3,6,9) column to be placed under column 1 (X). Same goes for Y (every second). Then when the first row is complete move to the second row and compile that.
> > >
> > > Thanks.
> > >
> > > Jeremy
> >
> > Have you tested any of this code? As far as I can tell, a there are a
> > lot of ways to do what you are asking here (and it has been shown to
> > be true).
> > For example, the code I posted above works for your whole matrix.
> > Tideman's code can be easily modified to make it work for your whole
> > matrix.
> > James Tursa's code (the one liner) will do the reshaping you are
> > asking with minimal code (but as he said, it has a lot of data
> > movement which can be a problem with larger matrices).
> > If you are used to MEX files, you can compile his C code as a MEX file
> > and run it through that (and I'm assuming it will STILL do exactly
> > what you asked for).
> >
> > Why not try these things out and see which one you like best for your
> > program?
> > If you don't understand the code, just ask a specific question about
> > specific code and I'm sure we will be more than willing to let you
> > know what it does.
> >
> > -Nathan
>
> Ok I will try again.

This seems to work but how do i concatenate the variable B1 for the loop. (store the data).

    r = cell2mat(r);
    rnum = numel(r(:,1));
    for iii = 1:rnum
         X1=r(iii,1:3:end)';
         Y1=r(iii,2:3:end)';
         Z1=r(iii,3:3:end)';
         B1=[X1,Y1,Z1];
    end

Thank you.

Subject: Reshaping Dataset

From: Nathan

Date: 26 Jun, 2009 18:09:26

Message: 12 of 17

On Jun 26, 10:55 am, "Jeremy Bing" <jer...@mytrashmail.com> wrote:
> "Jeremy Bing" <jer...@mytrashmail.com> wrote in message <h22vsa$qk...@fred.mathworks.com>...
> > Nathan <ngrec...@gmail.com> wrote in message <54b243f3-71a5-40cd-88b2-0b4b1efca...@l35g2000pra.googlegroups.com>...
> > > On Jun 26, 9:49?am, "Jeremy Bing" <jer...@mytrashmail.com> wrote:
> > > > "James Tursa" <aclassyguywithakno...@hotmail.com> wrote in message <h21081$dg...@fred.mathworks.com>...
> > > > > "James Tursa" <aclassyguywithakno...@hotmail.com> wrote in message <h20s1i$ev...@fred.mathworks.com>...
> > > > > > "Jeremy Bing" <jer...@mytrashmail.com> wrote in message <h20j9k$9o...@fred.mathworks.com>...
> > > > > > > Hi,
>
> > > > > > > I have data in the form of the following.
>
> > > > > > > X Y Z X Y Z ?>>
> > > > > > > X Y Z X Y Z
> > > > > > > X Y Z X Y Z
> > > > > > > X Y Z X Y Z
> > > > > > > ...
>
> > > > > > > But i need the data to be:
>
> > > > > > > X Y Z
> > > > > > > X Y Z
> > > > > > > X Y Z
>
> > > > > > > Do i used a case statement or a nested forloop to reshape the data? The dataset is quite large.
>
> > > > > > reshape(A',3,[])'
>
> > > > > > But this has a lot of data movement, the two transposes cause the entire dataset to be moved twice. If the time consumption is still too large then we could resort to a c-mex routine and limit the data movement to once, speeding things up.
>
> > > > > > James Tursa
>
> > > > > FYI, here is a bare-bones (no argument checking) c-mex file to do the same job. I get about a 20-30% speed improvement with this routine over the reshape method shown above. This may or may not be significant to you.
>
> > > > > James Tursa
>
> > > > > -----------------------------------
>
> > > > > #include "mex.h"
> > > > > void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
> > > > > {
> > > > > ? ? mwSize m, n, i, numel;
> > > > > ? ? double *target, *source1, *source2;
>
> > > > > ? ? m = mxGetM(prhs[0]);
> > > > > ? ? n = mxGetN(prhs[0]);
> > > > > ? ? numel = m * n / 2;
> > > > > ? ? plhs[0] = mxCreateDoubleMatrix(2*m, n/2, mxREAL);
> > > > > ? ? target = mxGetPr(plhs[0]);
> > > > > ? ? source1 = mxGetPr(prhs[0]);
> > > > > ? ? source2 = source1 + numel;
> > > > > ? ? for( i=0; i<numel; i++ ) {
> > > > > ? ? ? ? *target++ = *source1++;
> > > > > ? ? ? ? *target++ = *source2++;
> > > > > ? ? }
> > > > > }
>
> > > > Hi,
>
> > > > Thanks for the replies.
>
> > > > I'm not sure I understand the code. I have been trying to do it. My data has 63 columns and 294 rows however this depends on the file being input.
> > > > The format for each row is X Y Z X Y Z X Y Z X Y Z >>> 63. I need the all X, Y and Z data to be in one column rather in a row for every Row in the file. I.e I need every 3'd (3,6,9) column to be placed under column 1 (X). Same goes for Y (every second). Then when the first row is complete move to the second row and compile that.
>
> > > > Thanks.
>
> > > > Jeremy
>
> > > Have you tested any of this code? As far as I can tell, a there are a
> > > lot of ways to do what you are asking here (and it has been shown to
> > > be true).
> > > For example, the code I posted above works for your whole matrix.
> > > Tideman's code can be easily modified to make it work for your whole
> > > matrix.
> > > James Tursa's code (the one liner) will do the reshaping you are
> > > asking with minimal code (but as he said, it has a lot of data
> > > movement which can be a problem with larger matrices).
> > > If you are used to MEX files, you can compile his C code as a MEX file
> > > and run it through that (and I'm assuming it will STILL do exactly
> > > what you asked for).
>
> > > Why not try these things out and see which one you like best for your
> > > program?
> > > If you don't understand the code, just ask a specific question about
> > > specific code and I'm sure we will be more than willing to let you
> > > know what it does.
>
> > > -Nathan
>
> > Ok I will try again.
>
> This seems to work but how do i concatenate the variable B1 for the loop. (store the data).
>
>     r = cell2mat(r);
>     rnum = numel(r(:,1));
>     for iii = 1:rnum
>          X1=r(iii,1:3:end)';
>          Y1=r(iii,2:3:end)';
>          Z1=r(iii,3:3:end)';
>          B1=[X1,Y1,Z1];
>     end
>
> Thank you.

start out with B1 = [] and do B1 = [B1;X1,Y1,Z1];

B1 = [];
r = cell2mat(r);
rnum = numel(r(:,1));
for iii = 1:rnum
     X1=r(iii,1:3:end)';
     Y1=r(iii,2:3:end)';
     Z1=r(iii,3:3:end)';
     B1=[B1;X1,Y1,Z1];
end

Subject: Reshaping Dataset

From: Jeremy Bing

Date: 26 Jun, 2009 18:36:01

Message: 13 of 17

Nathan <ngreco32@gmail.com> wrote in message <10ff833d-fad9-4a7a-8b86-2a46bca5ab85@y10g2000prc.googlegroups.com>...
> On Jun 26, 10:55?am, "Jeremy Bing" <jer...@mytrashmail.com> wrote:
> > "Jeremy Bing" <jer...@mytrashmail.com> wrote in message <h22vsa$qk...@fred.mathworks.com>...
> > > Nathan <ngrec...@gmail.com> wrote in message <54b243f3-71a5-40cd-88b2-0b4b1efca...@l35g2000pra.googlegroups.com>...
> > > > On Jun 26, 9:49?am, "Jeremy Bing" <jer...@mytrashmail.com> wrote:
> > > > > "James Tursa" <aclassyguywithakno...@hotmail.com> wrote in message <h21081$dg...@fred.mathworks.com>...
> > > > > > "James Tursa" <aclassyguywithakno...@hotmail.com> wrote in message <h20s1i$ev...@fred.mathworks.com>...
> > > > > > > "Jeremy Bing" <jer...@mytrashmail.com> wrote in message <h20j9k$9o...@fred.mathworks.com>...
> > > > > > > > Hi,
> >
> > > > > > > > I have data in the form of the following.
> >
> > > > > > > > X Y Z X Y Z ?>>
> > > > > > > > X Y Z X Y Z
> > > > > > > > X Y Z X Y Z
> > > > > > > > X Y Z X Y Z
> > > > > > > > ...
> >
> > > > > > > > But i need the data to be:
> >
> > > > > > > > X Y Z
> > > > > > > > X Y Z
> > > > > > > > X Y Z
> >
> > > > > > > > Do i used a case statement or a nested forloop to reshape the data? The dataset is quite large.
> >
> > > > > > > reshape(A',3,[])'
> >
> > > > > > > But this has a lot of data movement, the two transposes cause the entire dataset to be moved twice. If the time consumption is still too large then we could resort to a c-mex routine and limit the data movement to once, speeding things up.
> >
> > > > > > > James Tursa
> >
> > > > > > FYI, here is a bare-bones (no argument checking) c-mex file to do the same job. I get about a 20-30% speed improvement with this routine over the reshape method shown above. This may or may not be significant to you.
> >
> > > > > > James Tursa
> >
> > > > > > -----------------------------------
> >
> > > > > > #include "mex.h"
> > > > > > void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
> > > > > > {
> > > > > > ? ? mwSize m, n, i, numel;
> > > > > > ? ? double *target, *source1, *source2;
> >
> > > > > > ? ? m = mxGetM(prhs[0]);
> > > > > > ? ? n = mxGetN(prhs[0]);
> > > > > > ? ? numel = m * n / 2;
> > > > > > ? ? plhs[0] = mxCreateDoubleMatrix(2*m, n/2, mxREAL);
> > > > > > ? ? target = mxGetPr(plhs[0]);
> > > > > > ? ? source1 = mxGetPr(prhs[0]);
> > > > > > ? ? source2 = source1 + numel;
> > > > > > ? ? for( i=0; i<numel; i++ ) {
> > > > > > ? ? ? ? *target++ = *source1++;
> > > > > > ? ? ? ? *target++ = *source2++;
> > > > > > ? ? }
> > > > > > }
> >
> > > > > Hi,
> >
> > > > > Thanks for the replies.
> >
> > > > > I'm not sure I understand the code. I have been trying to do it. My data has 63 columns and 294 rows however this depends on the file being input.
> > > > > The format for each row is X Y Z X Y Z X Y Z X Y Z >>> 63. I need the all X, Y and Z data to be in one column rather in a row for every Row in the file. I.e I need every 3'd (3,6,9) column to be placed under column 1 (X). Same goes for Y (every second). Then when the first row is complete move to the second row and compile that.
> >
> > > > > Thanks.
> >
> > > > > Jeremy
> >
> > > > Have you tested any of this code? As far as I can tell, a there are a
> > > > lot of ways to do what you are asking here (and it has been shown to
> > > > be true).
> > > > For example, the code I posted above works for your whole matrix.
> > > > Tideman's code can be easily modified to make it work for your whole
> > > > matrix.
> > > > James Tursa's code (the one liner) will do the reshaping you are
> > > > asking with minimal code (but as he said, it has a lot of data
> > > > movement which can be a problem with larger matrices).
> > > > If you are used to MEX files, you can compile his C code as a MEX file
> > > > and run it through that (and I'm assuming it will STILL do exactly
> > > > what you asked for).
> >
> > > > Why not try these things out and see which one you like best for your
> > > > program?
> > > > If you don't understand the code, just ask a specific question about
> > > > specific code and I'm sure we will be more than willing to let you
> > > > know what it does.
> >
> > > > -Nathan
> >
> > > Ok I will try again.
> >
> > This seems to work but how do i concatenate the variable B1 for the loop. (store the data).
> >
> > ? ? r = cell2mat(r);
> > ? ? rnum = numel(r(:,1));
> > ? ? for iii = 1:rnum
> > ? ? ? ? ?X1=r(iii,1:3:end)';
> > ? ? ? ? ?Y1=r(iii,2:3:end)';
> > ? ? ? ? ?Z1=r(iii,3:3:end)';
> > ? ? ? ? ?B1=[X1,Y1,Z1];
> > ? ? end
> >
> > Thank you.
>
> start out with B1 = [] and do B1 = [B1;X1,Y1,Z1];
>
> B1 = [];
> r = cell2mat(r);
> rnum = numel(r(:,1));
> for iii = 1:rnum
> X1=r(iii,1:3:end)';
> Y1=r(iii,2:3:end)';
> Z1=r(iii,3:3:end)';
> B1=[B1;X1,Y1,Z1];
> end

Hi,

Thanks that worked. Although its telling me to preallocate B1?

Regards,

Jeremy

Subject: Reshaping Dataset

From: Nathan

Date: 26 Jun, 2009 18:55:22

Message: 14 of 17

On Jun 26, 11:36 am, "Jeremy Bing" <jer...@mytrashmail.com> wrote:
> Nathan <ngrec...@gmail.com> wrote in message <10ff833d-fad9-4a7a-8b86-2a46bca5a...@y10g2000prc.googlegroups.com>...
> > On Jun 26, 10:55?am, "Jeremy Bing" <jer...@mytrashmail.com> wrote:
> > > "Jeremy Bing" <jer...@mytrashmail.com> wrote in message <h22vsa$qk...@fred.mathworks.com>...
> > > > Nathan <ngrec...@gmail.com> wrote in message <54b243f3-71a5-40cd-88b2-0b4b1efca...@l35g2000pra.googlegroups.com>...
> > > > > On Jun 26, 9:49?am, "Jeremy Bing" <jer...@mytrashmail.com> wrote:
> > > > > > "James Tursa" <aclassyguywithakno...@hotmail.com> wrote in message <h21081$dg...@fred.mathworks.com>...
> > > > > > > "James Tursa" <aclassyguywithakno...@hotmail.com> wrote in message <h20s1i$ev...@fred.mathworks.com>...
> > > > > > > > "Jeremy Bing" <jer...@mytrashmail.com> wrote in message <h20j9k$9o...@fred.mathworks.com>...
> > > > > > > > > Hi,
>
> > > > > > > > > I have data in the form of the following.
>
> > > > > > > > > X Y Z X Y Z ?>>
> > > > > > > > > X Y Z X Y Z
> > > > > > > > > X Y Z X Y Z
> > > > > > > > > X Y Z X Y Z
> > > > > > > > > ...
>
> > > > > > > > > But i need the data to be:
>
> > > > > > > > > X Y Z
> > > > > > > > > X Y Z
> > > > > > > > > X Y Z
>
> > > > > > > > > Do i used a case statement or a nested forloop to reshape the data? The dataset is quite large.
>
> > > > > > > > reshape(A',3,[])'
>
> > > > > > > > But this has a lot of data movement, the two transposes cause the entire dataset to be moved twice. If the time consumption is still too large then we could resort to a c-mex routine and limit the data movement to once, speeding things up.
>
> > > > > > > > James Tursa
>
> > > > > > > FYI, here is a bare-bones (no argument checking) c-mex file to do the same job. I get about a 20-30% speed improvement with this routine over the reshape method shown above. This may or may not be significant to you.
>
> > > > > > > James Tursa
>
> > > > > > > -----------------------------------
>
> > > > > > > #include "mex.h"
> > > > > > > void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
> > > > > > > {
> > > > > > > ? ? mwSize m, n, i, numel;
> > > > > > > ? ? double *target, *source1, *source2;
>
> > > > > > > ? ? m = mxGetM(prhs[0]);
> > > > > > > ? ? n = mxGetN(prhs[0]);
> > > > > > > ? ? numel = m * n / 2;
> > > > > > > ? ? plhs[0] = mxCreateDoubleMatrix(2*m, n/2, mxREAL);
> > > > > > > ? ? target = mxGetPr(plhs[0]);
> > > > > > > ? ? source1 = mxGetPr(prhs[0]);
> > > > > > > ? ? source2 = source1 + numel;
> > > > > > > ? ? for( i=0; i<numel; i++ ) {
> > > > > > > ? ? ? ? *target++ = *source1++;
> > > > > > > ? ? ? ? *target++ = *source2++;
> > > > > > > ? ? }
> > > > > > > }
>
> > > > > > Hi,
>
> > > > > > Thanks for the replies.
>
> > > > > > I'm not sure I understand the code. I have been trying to do it. My data has 63 columns and 294 rows however this depends on the file being input.
> > > > > > The format for each row is X Y Z X Y Z X Y Z X Y Z >>> 63. I need the all X, Y and Z data to be in one column rather in a row for every Row in the file. I.e I need every 3'd (3,6,9) column to be placed under column 1 (X). Same goes for Y (every second). Then when the first row is complete move to the second row and compile that.
>
> > > > > > Thanks.
>
> > > > > > Jeremy
>
> > > > > Have you tested any of this code? As far as I can tell, a there are a
> > > > > lot of ways to do what you are asking here (and it has been shown to
> > > > > be true).
> > > > > For example, the code I posted above works for your whole matrix.
> > > > > Tideman's code can be easily modified to make it work for your whole
> > > > > matrix.
> > > > > James Tursa's code (the one liner) will do the reshaping you are
> > > > > asking with minimal code (but as he said, it has a lot of data
> > > > > movement which can be a problem with larger matrices).
> > > > > If you are used to MEX files, you can compile his C code as a MEX file
> > > > > and run it through that (and I'm assuming it will STILL do exactly
> > > > > what you asked for).
>
> > > > > Why not try these things out and see which one you like best for your
> > > > > program?
> > > > > If you don't understand the code, just ask a specific question about
> > > > > specific code and I'm sure we will be more than willing to let you
> > > > > know what it does.
>
> > > > > -Nathan
>
> > > > Ok I will try again.
>
> > > This seems to work but how do i concatenate the variable B1 for the loop. (store the data).
>
> > > ? ? r = cell2mat(r);
> > > ? ? rnum = numel(r(:,1));
> > > ? ? for iii = 1:rnum
> > > ? ? ? ? ?X1=r(iii,1:3:end)';
> > > ? ? ? ? ?Y1=r(iii,2:3:end)';
> > > ? ? ? ? ?Z1=r(iii,3:3:end)';
> > > ? ? ? ? ?B1=[X1,Y1,Z1];
> > > ? ? end
>
> > > Thank you.
>
> > start out with B1 = [] and do B1 = [B1;X1,Y1,Z1];
>
> > B1 = [];
> > r = cell2mat(r);
> > rnum = numel(r(:,1));
> > for iii = 1:rnum
> >      X1=r(iii,1:3:end)';
> >      Y1=r(iii,2:3:end)';
> >      Z1=r(iii,3:3:end)';
> >      B1=[B1;X1,Y1,Z1];
> > end
>
> Hi,
>
> Thanks that worked. Although its telling me to preallocate B1?
>
> Regards,
>
> Jeremy

If you previously know the size of what B1 is supposed to be (in your
case 6174x3), do exactly what it says. Make B1 that big to begin with.
You have to change your code around a bit to make it still work,
however, but it will be faster with preallocation.

B1 = zeros(6174,3);
rnum = numel(r(:,1));
for iii = 1:rnum
     X1=r(iii,1:3:end)';
     Y1=r(iii,2:3:end)';
     Z1=r(iii,3:3:end)';
     B1((iii-1)*98+1:(iii)*98,1:3)=[X1,Y1,Z1]; %puts X1,Y1,Z1 into the
correct spots in B1
end

Subject: Reshaping Dataset

From: Jeremy Bing

Date: 26 Jun, 2009 20:17:01

Message: 15 of 17

Nathan <ngreco32@gmail.com> wrote in message <565e5a43-680a-4524-a407-f329449f22e3@u16g2000pru.googlegroups.com>...
> On Jun 26, 11:36?am, "Jeremy Bing" <jer...@mytrashmail.com> wrote:
> > Nathan <ngrec...@gmail.com> wrote in message <10ff833d-fad9-4a7a-8b86-2a46bca5a...@y10g2000prc.googlegroups.com>...
> > > On Jun 26, 10:55?am, "Jeremy Bing" <jer...@mytrashmail.com> wrote:
> > > > "Jeremy Bing" <jer...@mytrashmail.com> wrote in message <h22vsa$qk...@fred.mathworks.com>...
> > > > > Nathan <ngrec...@gmail.com> wrote in message <54b243f3-71a5-40cd-88b2-0b4b1efca...@l35g2000pra.googlegroups.com>...
> > > > > > On Jun 26, 9:49?am, "Jeremy Bing" <jer...@mytrashmail.com> wrote:
> > > > > > > "James Tursa" <aclassyguywithakno...@hotmail.com> wrote in message <h21081$dg...@fred.mathworks.com>...
> > > > > > > > "James Tursa" <aclassyguywithakno...@hotmail.com> wrote in message <h20s1i$ev...@fred.mathworks.com>...
> > > > > > > > > "Jeremy Bing" <jer...@mytrashmail.com> wrote in message <h20j9k$9o...@fred.mathworks.com>...
> > > > > > > > > > Hi,
> >
> > > > > > > > > > I have data in the form of the following.
> >
> > > > > > > > > > X Y Z X Y Z ?>>
> > > > > > > > > > X Y Z X Y Z
> > > > > > > > > > X Y Z X Y Z
> > > > > > > > > > X Y Z X Y Z
> > > > > > > > > > ...
> >
> > > > > > > > > > But i need the data to be:
> >
> > > > > > > > > > X Y Z
> > > > > > > > > > X Y Z
> > > > > > > > > > X Y Z
> >
> > > > > > > > > > Do i used a case statement or a nested forloop to reshape the data? The dataset is quite large.
> >
> > > > > > > > > reshape(A',3,[])'
> >
> > > > > > > > > But this has a lot of data movement, the two transposes cause the entire dataset to be moved twice. If the time consumption is still too large then we could resort to a c-mex routine and limit the data movement to once, speeding things up.
> >
> > > > > > > > > James Tursa
> >
> > > > > > > > FYI, here is a bare-bones (no argument checking) c-mex file to do the same job. I get about a 20-30% speed improvement with this routine over the reshape method shown above. This may or may not be significant to you.
> >
> > > > > > > > James Tursa
> >
> > > > > > > > -----------------------------------
> >
> > > > > > > > #include "mex.h"
> > > > > > > > void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
> > > > > > > > {
> > > > > > > > ? ? mwSize m, n, i, numel;
> > > > > > > > ? ? double *target, *source1, *source2;
> >
> > > > > > > > ? ? m = mxGetM(prhs[0]);
> > > > > > > > ? ? n = mxGetN(prhs[0]);
> > > > > > > > ? ? numel = m * n / 2;
> > > > > > > > ? ? plhs[0] = mxCreateDoubleMatrix(2*m, n/2, mxREAL);
> > > > > > > > ? ? target = mxGetPr(plhs[0]);
> > > > > > > > ? ? source1 = mxGetPr(prhs[0]);
> > > > > > > > ? ? source2 = source1 + numel;
> > > > > > > > ? ? for( i=0; i<numel; i++ ) {
> > > > > > > > ? ? ? ? *target++ = *source1++;
> > > > > > > > ? ? ? ? *target++ = *source2++;
> > > > > > > > ? ? }
> > > > > > > > }
> >
> > > > > > > Hi,
> >
> > > > > > > Thanks for the replies.
> >
> > > > > > > I'm not sure I understand the code. I have been trying to do it. My data has 63 columns and 294 rows however this depends on the file being input.
> > > > > > > The format for each row is X Y Z X Y Z X Y Z X Y Z >>> 63. I need the all X, Y and Z data to be in one column rather in a row for every Row in the file. I.e I need every 3'd (3,6,9) column to be placed under column 1 (X). Same goes for Y (every second). Then when the first row is complete move to the second row and compile that.
> >
> > > > > > > Thanks.
> >
> > > > > > > Jeremy
> >
> > > > > > Have you tested any of this code? As far as I can tell, a there are a
> > > > > > lot of ways to do what you are asking here (and it has been shown to
> > > > > > be true).
> > > > > > For example, the code I posted above works for your whole matrix.
> > > > > > Tideman's code can be easily modified to make it work for your whole
> > > > > > matrix.
> > > > > > James Tursa's code (the one liner) will do the reshaping you are
> > > > > > asking with minimal code (but as he said, it has a lot of data
> > > > > > movement which can be a problem with larger matrices).
> > > > > > If you are used to MEX files, you can compile his C code as a MEX file
> > > > > > and run it through that (and I'm assuming it will STILL do exactly
> > > > > > what you asked for).
> >
> > > > > > Why not try these things out and see which one you like best for your
> > > > > > program?
> > > > > > If you don't understand the code, just ask a specific question about
> > > > > > specific code and I'm sure we will be more than willing to let you
> > > > > > know what it does.
> >
> > > > > > -Nathan
> >
> > > > > Ok I will try again.
> >
> > > > This seems to work but how do i concatenate the variable B1 for the loop. (store the data).
> >
> > > > ? ? r = cell2mat(r);
> > > > ? ? rnum = numel(r(:,1));
> > > > ? ? for iii = 1:rnum
> > > > ? ? ? ? ?X1=r(iii,1:3:end)';
> > > > ? ? ? ? ?Y1=r(iii,2:3:end)';
> > > > ? ? ? ? ?Z1=r(iii,3:3:end)';
> > > > ? ? ? ? ?B1=[X1,Y1,Z1];
> > > > ? ? end
> >
> > > > Thank you.
> >
> > > start out with B1 = [] and do B1 = [B1;X1,Y1,Z1];
> >
> > > B1 = [];
> > > r = cell2mat(r);
> > > rnum = numel(r(:,1));
> > > for iii = 1:rnum
> > > ? ? ?X1=r(iii,1:3:end)';
> > > ? ? ?Y1=r(iii,2:3:end)';
> > > ? ? ?Z1=r(iii,3:3:end)';
> > > ? ? ?B1=[B1;X1,Y1,Z1];
> > > end
> >
> > Hi,
> >
> > Thanks that worked. Although its telling me to preallocate B1?
> >
> > Regards,
> >
> > Jeremy
>
> If you previously know the size of what B1 is supposed to be (in your
> case 6174x3), do exactly what it says. Make B1 that big to begin with.
> You have to change your code around a bit to make it still work,
> however, but it will be faster with preallocation.
>
> B1 = zeros(6174,3);
> rnum = numel(r(:,1));
> for iii = 1:rnum
> X1=r(iii,1:3:end)';
> Y1=r(iii,2:3:end)';
> Z1=r(iii,3:3:end)';
> B1((iii-1)*98+1:(iii)*98,1:3)=[X1,Y1,Z1]; %puts X1,Y1,Z1 into the
> correct spots in B1
> end

Great works thanks

Subject: Reshaping Dataset

From: Jeremy Bing

Date: 26 Jun, 2009 20:34:01

Message: 16 of 17

Nathan <ngreco32@gmail.com> wrote in message <565e5a43-680a-4524-a407-f329449f22e3@u16g2000pru.googlegroups.com>...
> On Jun 26, 11:36?am, "Jeremy Bing" <jer...@mytrashmail.com> wrote:
> > Nathan <ngrec...@gmail.com> wrote in message <10ff833d-fad9-4a7a-8b86-2a46bca5a...@y10g2000prc.googlegroups.com>...
> > > On Jun 26, 10:55?am, "Jeremy Bing" <jer...@mytrashmail.com> wrote:
> > > > "Jeremy Bing" <jer...@mytrashmail.com> wrote in message <h22vsa$qk...@fred.mathworks.com>...
> > > > > Nathan <ngrec...@gmail.com> wrote in message <54b243f3-71a5-40cd-88b2-0b4b1efca...@l35g2000pra.googlegroups.com>...
> > > > > > On Jun 26, 9:49?am, "Jeremy Bing" <jer...@mytrashmail.com> wrote:
> > > > > > > "James Tursa" <aclassyguywithakno...@hotmail.com> wrote in message <h21081$dg...@fred.mathworks.com>...
> > > > > > > > "James Tursa" <aclassyguywithakno...@hotmail.com> wrote in message <h20s1i$ev...@fred.mathworks.com>...
> > > > > > > > > "Jeremy Bing" <jer...@mytrashmail.com> wrote in message <h20j9k$9o...@fred.mathworks.com>...
> > > > > > > > > > Hi,
> >
> > > > > > > > > > I have data in the form of the following.
> >
> > > > > > > > > > X Y Z X Y Z ?>>
> > > > > > > > > > X Y Z X Y Z
> > > > > > > > > > X Y Z X Y Z
> > > > > > > > > > X Y Z X Y Z
> > > > > > > > > > ...
> >
> > > > > > > > > > But i need the data to be:
> >
> > > > > > > > > > X Y Z
> > > > > > > > > > X Y Z
> > > > > > > > > > X Y Z
> >
> > > > > > > > > > Do i used a case statement or a nested forloop to reshape the data? The dataset is quite large.
> >
> > > > > > > > > reshape(A',3,[])'
> >
> > > > > > > > > But this has a lot of data movement, the two transposes cause the entire dataset to be moved twice. If the time consumption is still too large then we could resort to a c-mex routine and limit the data movement to once, speeding things up.
> >
> > > > > > > > > James Tursa
> >
> > > > > > > > FYI, here is a bare-bones (no argument checking) c-mex file to do the same job. I get about a 20-30% speed improvement with this routine over the reshape method shown above. This may or may not be significant to you.
> >
> > > > > > > > James Tursa
> >
> > > > > > > > -----------------------------------
> >
> > > > > > > > #include "mex.h"
> > > > > > > > void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
> > > > > > > > {
> > > > > > > > ? ? mwSize m, n, i, numel;
> > > > > > > > ? ? double *target, *source1, *source2;
> >
> > > > > > > > ? ? m = mxGetM(prhs[0]);
> > > > > > > > ? ? n = mxGetN(prhs[0]);
> > > > > > > > ? ? numel = m * n / 2;
> > > > > > > > ? ? plhs[0] = mxCreateDoubleMatrix(2*m, n/2, mxREAL);
> > > > > > > > ? ? target = mxGetPr(plhs[0]);
> > > > > > > > ? ? source1 = mxGetPr(prhs[0]);
> > > > > > > > ? ? source2 = source1 + numel;
> > > > > > > > ? ? for( i=0; i<numel; i++ ) {
> > > > > > > > ? ? ? ? *target++ = *source1++;
> > > > > > > > ? ? ? ? *target++ = *source2++;
> > > > > > > > ? ? }
> > > > > > > > }
> >
> > > > > > > Hi,
> >
> > > > > > > Thanks for the replies.
> >
> > > > > > > I'm not sure I understand the code. I have been trying to do it. My data has 63 columns and 294 rows however this depends on the file being input.
> > > > > > > The format for each row is X Y Z X Y Z X Y Z X Y Z >>> 63. I need the all X, Y and Z data to be in one column rather in a row for every Row in the file. I.e I need every 3'd (3,6,9) column to be placed under column 1 (X). Same goes for Y (every second). Then when the first row is complete move to the second row and compile that.
> >
> > > > > > > Thanks.
> >
> > > > > > > Jeremy
> >
> > > > > > Have you tested any of this code? As far as I can tell, a there are a
> > > > > > lot of ways to do what you are asking here (and it has been shown to
> > > > > > be true).
> > > > > > For example, the code I posted above works for your whole matrix.
> > > > > > Tideman's code can be easily modified to make it work for your whole
> > > > > > matrix.
> > > > > > James Tursa's code (the one liner) will do the reshaping you are
> > > > > > asking with minimal code (but as he said, it has a lot of data
> > > > > > movement which can be a problem with larger matrices).
> > > > > > If you are used to MEX files, you can compile his C code as a MEX file
> > > > > > and run it through that (and I'm assuming it will STILL do exactly
> > > > > > what you asked for).
> >
> > > > > > Why not try these things out and see which one you like best for your
> > > > > > program?
> > > > > > If you don't understand the code, just ask a specific question about
> > > > > > specific code and I'm sure we will be more than willing to let you
> > > > > > know what it does.
> >
> > > > > > -Nathan
> >
> > > > > Ok I will try again.
> >
> > > > This seems to work but how do i concatenate the variable B1 for the loop. (store the data).
> >
> > > > ? ? r = cell2mat(r);
> > > > ? ? rnum = numel(r(:,1));
> > > > ? ? for iii = 1:rnum
> > > > ? ? ? ? ?X1=r(iii,1:3:end)';
> > > > ? ? ? ? ?Y1=r(iii,2:3:end)';
> > > > ? ? ? ? ?Z1=r(iii,3:3:end)';
> > > > ? ? ? ? ?B1=[X1,Y1,Z1];
> > > > ? ? end
> >
> > > > Thank you.
> >
> > > start out with B1 = [] and do B1 = [B1;X1,Y1,Z1];
> >
> > > B1 = [];
> > > r = cell2mat(r);
> > > rnum = numel(r(:,1));
> > > for iii = 1:rnum
> > > ? ? ?X1=r(iii,1:3:end)';
> > > ? ? ?Y1=r(iii,2:3:end)';
> > > ? ? ?Z1=r(iii,3:3:end)';
> > > ? ? ?B1=[B1;X1,Y1,Z1];
> > > end
> >
> > Hi,
> >
> > Thanks that worked. Although its telling me to preallocate B1?
> >
> > Regards,
> >
> > Jeremy
>
> If you previously know the size of what B1 is supposed to be (in your
> case 6174x3), do exactly what it says. Make B1 that big to begin with.
> You have to change your code around a bit to make it still work,
> however, but it will be faster with preallocation.
>
> B1 = zeros(6174,3);
> rnum = numel(r(:,1));
> for iii = 1:rnum
> X1=r(iii,1:3:end)';
> Y1=r(iii,2:3:end)';
> Z1=r(iii,3:3:end)';
> B1((iii-1)*98+1:(iii)*98,1:3)=[X1,Y1,Z1]; %puts X1,Y1,Z1 into the
> correct spots in B1
> end

Great works thanks

Subject: Reshaping Dataset

From: James Tursa

Date: 27 Jun, 2009 07:48:01

Message: 17 of 17

"Jeremy Bing" <jeremy@mytrashmail.com> wrote in message <h22u5u$4rm$1@fred.mathworks.com>...
 for the replies.
>
> I'm not sure I understand the code. I have been trying to do it. My data has 63 columns and 294 rows however this depends on the file being input.
> The format for each row is X Y Z X Y Z X Y Z X Y Z >>> 63. I need the all X, Y and Z data to be in one column rather in a row for every Row in the file. I.e I need every 3'd (3,6,9) column to be placed under column 1 (X). Same goes for Y (every second). Then when the first row is complete move to the second row and compile that.
>
> Thanks.
>
> Jeremy

A few comments:

1) I misunderstood your problem. I read your 1st post literally and thought that you had an M x 6 array. I did not realize you had an M x 3n array. So the c-mex routine I posted will not work for this more general case. I would have to modify it. I could post an updated routine, but I don't think you need it (see comment 3 below).

2) Even though I misunderstood your problem, my posted MATLAB method will still work for you:

    reshape(A',3,[])'

This is IMO the best method of all of the MATLAB based methods posted. A concise vectorized one-liner that is fairly easy to understand. The A' part just puts all of the relevant data next to each other in memory (the entire dataset is copied), the reshape command gets all of the x-y-z data into separate rows (without requiring a copy), and the final ' just transforms the x-y-z rows into columns (the entire dataset is copied again). Yes, it moves the entire dataset twice, but so do all of the other methods posted. In fact some of these other methods move the data three times, and they have a lot more overhead of temporary variables created than the reshape method shown above.

3) You mentioned that you had a large problem, so I figured a c-mex routine might make a difference to you in speed. However, 295 x 63 is not a large problem ... I wouldn't even call it a medium problem. For this small size, any MATLAB method will probably do just fine unless it is extremely poorly coded. So I wouldn't worry about a c-mex routine and probably wouldn't worry too much about fine tuning the method you ultimately choose ... you probably wouldn't even notice the difference.

James Tursa

Tags for this Thread

Add a New Tag:

Separated by commas
Ex.: root locus, bode

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

rssFeed for this Thread
 

MATLAB Central Terms of Use

NOTICE: Any content you submit to MATLAB Central, including personal information, is not subject to the protections which may be afforded information collected under other sections of The MathWorks, Inc. Web site. You are entirely responsible for all content that you upload, post, e-mail, transmit or otherwise make available via MATLAB Central. The MathWorks does not control the content posted by visitors to MATLAB Central and, does not guarantee the accuracy, integrity, or quality of such content. Under no circumstances will The MathWorks be liable in any way for any content not authored by The MathWorks, or any loss or damage of any kind incurred as a result of the use of any content posted, e-mailed, transmitted or otherwise made available via MATLAB Central. Read the complete Terms prior to use.

Contact us at files@mathworks.com