Thread Subject:

Data manipulation

Hi,
I have a block of data that I wish to process.
It has data then a set of zeros to signal then next block of data.

The each data block I wish to average and return the average into a matrix.
eg
1 3 5 6
4 5 7 2
0 0 0 0
0 0 0 0
2 3 4 5
5 8 8 9
0 0 0 0
0 0 0 0

How do I count how many rows in the data block(it could vary)?So that I can sum the individual block and avearge it? and then skip to the next data block a do the same.

I have tried for ,if loops but can't get the increments correct?

for length of data

A = [1 3 5 6; 4 5 7 2; 0 0 0 0; 0 0 0 0; 2 3 4 5; 5 8 8 9; 0 0 0 0; 0 0 0 0];
mean(reshape(transpose(A(~all((A == 0), 2), :)), 8, []))

"james bejon" <jamesbejon@yahoo.co.uk> wrote in message <iavjfo$npf$1@fred.mathworks.com>...
> A = [1 3 5 6; 4 5 7 2; 0 0 0 0; 0 0 0 0; 2 3 4 5; 5 8 8 9; 0 0 0 0; 0 0 0 0];
> mean(reshape(transpose(A(~all((A == 0), 2), :)), 8, []))

Hi James ,
Thanks but this sets the zero block to two and this may vary as well.I have an unknown length of data set ,unknown data block lengths and unknown break ie zero's between the data blocks.

Cheers
Danielle

"Danielle Botha" <delsatr1977@yahoo.com.au> wrote in message <iavfpm$rog$1@fred.mathworks.com>...
> Hi,
> I have a block of data that I wish to process.
> It has data then a set of zeros to signal then next block of data.
>
> The each data block I wish to average and return the average into a matrix.
> eg
> 1 3 5 6
> 4 5 7 2
> 0 0 0 0
> 0 0 0 0
> 2 3 4 5
> 5 8 8 9
> 0 0 0 0
> 0 0 0 0
>
> How do I count how many rows in the data block(it could vary)?So that I can sum the individual block and avearge it? and then skip to the next data block a do the same.
>
> I have tried for ,if loops but can't get the increments correct?
>
> for length of data
- - - - - - - - -
  I assume here that if a row is not all zeros, that whole row will be included in a block for averaging purposes. That is, if you had an isolated block equal to [0 1 2 3], for example, you would consider its average as 6/4 and not 6/3.

  With that assumption, do this:

p = ~all(A==0,2);
q = diff([false;p])>0;
q = cumsum(q(p));
B = accumarray(q,sum(A(p,:),2))./accumarray(q,size(A,2)*ones(size(q)));

B should be a vector with your block averages.

  Note: I haven't had the opportunity to fully test this, so let me know if it doesn't work properly.

Roger Stafford

So, here's a horrible one-liner you can use to handle cases where you can have any number of rows of zeros:

nonzeros(accumarray(cumsum(abs(diff([false; ~all(A==0, 2)]))), mean(A, 2), [], @mean))

"james bejon" <jamesbejon@yahoo.co.uk> wrote in message <ib21j3$9hc$1@fred.mathworks.com>...
> So, here's a horrible one-liner you can use to handle cases where you can have any number of rows of zeros:
>
> nonzeros(accumarray(cumsum(abs(diff([false; ~all(A==0, 2)]))), mean(A, 2), [], @mean))
- - - - - - - -
  If Danielle wanted to have some of the elements in A be negative, it is possible that some of the valid "data blocks" might turn out to have a mean value of zero. The 'nonzeros' call would unfortunately eliminate these along with all the zero-block spacers.

  However your solution does show that I could have simplified mine by using the @mean function in a single 'accumarray' call.

Roger Stafford

Good point Roger. Thanks.

Dear Danielle,

> 1 3 5 6
> 4 5 7 2
> 0 0 0 0
> 0 0 0 0
> 2 3 4 5
> 5 8 8 9
> 0 0 0 0
> 0 0 0 0

Can any number of the "data block" be zero?
I didn't understand your wanted output format. Can you post it for the above example?

Have you seen Matt Fig's:
http://www.mathworks.com/matlabcentral/fileexchange/28685-rcumsumc
This does not match perfectly, but it might be useful.

Kind regards, Jan

"Jan Simon" <matlab.THIS_YEAR@nMINUSsimon.de> wrote in message <ib3c0j$o2q$1@fred.mathworks.com>...
> Dear Danielle,
>
> > 1 3 5 6
> > 4 5 7 2
> > 0 0 0 0
> > 0 0 0 0
> > 2 3 4 5
> > 5 8 8 9
> > 0 0 0 0
> > 0 0 0 0
>
> Can any number of the "data block" be zero?
> I didn't understand your wanted output format. Can you post it for the above example?
>
> Have you seen Matt Fig's:
> http://www.mathworks.com/matlabcentral/fileexchange/28685-rcumsumc
> This does not match perfectly, but it might be useful.
>
> Kind regards, Jan

Hi Jan,
I wasn't very clear was I.
The output I wish for the columns to be averaged.For example if I have four rows of data ,average of each colmun in that block
ie
    1 2 3 9
    3 0 5 9
= 2 1 4 9 is the resulting row.
The zeros are just a pause in the signal I need as a flag to indicate the next block to average.

Thank you all for the feeback.
Danielle

A little bit cumbersome, but you could use this tool:
http://www.mathworks.com/matlabcentral/fileexchange/24255

A=[1 3 5 6;
   4 5 7 2;
   0 0 0 0;
   2 3 4 5;
   5 8 8 9;
   0 0 0 0;
   0 0 0 0];

l = any(A~=0,2);
m = size(A,1);
c = SplitVec([l (1:m).'],1,@(r) mean(A(r(:,2),:),1)); % FEX
m = cat(1,c{2-l(1):2:end})

Bruno