Thread Subject:

A function to count the number of occurrances

I'm trying to see if there's a more efficient way to write this function. I need to be able to count the number of occurrences of an integer in a vector up to each element. For example if I have a vector with identifiers in the first column and I want to count the number of occurrences it's appeared up to that point:

X =

     1
     1
     1
     2
     3
     3
     3

The output would be a second column such as:
Y =

     1 1
     1 2
     1 3
     2 1
     3 1
     3 2
     3 3

Here's some code that I've written that searches the vector length for the number of instances, but it runs slowly. Can anyone help speed this up? Thanks in advance.

function [X]= count_upto(Y,column)

%Calculates the number of occurrances up to that observation
% Y is the matrix, column is the vector you count over
if nargin < 2
    column = 1;
end

[I J] = size(Y);
X = zeros(I,J+1);
X(:,1:J) = Y;

for i = 1:I
    X(i,J+1)= sum(X(1:i,column) == X(i,column));
       
end

"Rawley " <ballheim@gmail.com> wrote in message <igl2l4$h11$1@fred.mathworks.com>...
> I'm trying to see if there's a more efficient way to write this function. I need to be able to count the number of occurrences of an integer in a vector up to each element. For example if I have a vector with identifiers in the first column and I want to count the number of occurrences it's appeared up to that point:
>
> X =
>
> 1
> 1
> 1
> 2
> 3
> 3
> 3
>
> The output would be a second column such as:
> Y =
>
> 1 1
> 1 2
> 1 3
> 2 1
> 3 1
> 3 2
> 3 3
>
> Here's some code that I've written that searches the vector length for the number of instances, but it runs slowly. Can anyone help speed this up? Thanks in advance.
>
> function [X]= count_upto(Y,column)
>
> %Calculates the number of occurrances up to that observation
> % Y is the matrix, column is the vector you count over
> if nargin < 2
> column = 1;
> end
>
> [I J] = size(Y);
> X = zeros(I,J+1);
> X(:,1:J) = Y;
>
> for i = 1:I
> X(i,J+1)= sum(X(1:i,column) == X(i,column));
>
> end

I also have same question. lol

"Rawley " <ballheim@gmail.com> wrote in message <igl2l4$h11$1@fred.mathworks.com>...
> I'm trying to see if there's a more efficient way to write this function. I need to be able to count the number of occurrences of an integer in a vector up to each element. For example if I have a vector with identifiers in the first column and I want to count the number of occurrences it's appeared up to that point:
>
> X =
>
> 1
> 1
> 1
> 2
> 3
> 3
> 3
>
> The output would be a second column such as:
> Y =
>
> 1 1
> 1 2
> 1 3
> 2 1
> 3 1
> 3 2
> 3 3
>


One way:
X =[ 1
     1
     1
     2
     3
     3
     3 ];
 
[trash, bin] = histc(X,unique(X));
Y = [bin, cell2mat(cellfun(@cumsum,accumarray(bin,1,[],@(x){x}),'uni',false))]

Actually, even easier:

[trash, bin] = histc(X,unique(X));
Y = [bin, cell2mat(accumarray(bin,1,[],@(x){cumsum(x)}))]

-SCd

Dear Rawley,

Have you seen Matt Fig's RCUMSUM:
  http://www.mathworks.com/matlabcentral/fileexchange/28685
I expect this Cpp-Mex to be much faster.

Kind regards, Jan

Or this, a 100% vectorized Matlab solution:

[XS IS] = sort(X);
[~, I J] = unique(XS,'first');
Y(IS) = (1:length(X))'-I(J)+1

% Bruno

"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <igl63o$37i$1@fred.mathworks.com>...
> Or this, a 100% vectorized Matlab solution:
>
> [XS IS] = sort(X);
> [~, I J] = unique(XS,'first');
> Y(IS) = (1:length(X))'-I(J)+1
>
> % Bruno

If X is sorted, the code can be simplified to

[~, I J] = unique(X,'first');
Y = (2:length(X)+1)'-I(J)

Bruno

Wow! Thanks everyone for the help. Life savers!

I have a similar code that I need a more efficient solution to. I need to count the number of times real valued elements have occurred per unique identifier. The first column is the unique identifier and the third column is the occurrence. Input:

X =

     1 NaN
     1 NaN
     1 NaN
     1 NaN
     1 87
     1 45
     1 NaN
     1 NaN
     1 15
     5 6
     5 NaN
     5 NaN
     5 6
     5 8
     5 NaN

output:
Y =

     1 NaN 0
     1 NaN 0
     1 NaN 0
     1 NaN 0
     1 87 1
     1 45 2
     1 NaN 2
     1 NaN 2
     1 15 3
     5 6 1
     5 NaN 1
     5 NaN 1
     5 6 2
     5 8 3
     5 NaN 3

Thanks again in advance.

"Rawley " <ballheim@gmail.com> wrote in message <igoed9$fhd$1@fred.mathworks.com>...
> I have a similar code that I need a more efficient solution to. I need to count the number of times real valued elements have occurred per unique identifier. The first column is the unique identifier and the third column is the occurrence. Input:
>
> X =
>
> 1 NaN
> 1 NaN
> 1 NaN
> 1 NaN
> 1 87
> 1 45
> 1 NaN
> 1 NaN
> 1 15
> 5 6
> 5 NaN
> 5 NaN
> 5 6
> 5 8
> 5 NaN
>
> output:
> Y =
>
> 1 NaN 0
> 1 NaN 0
> 1 NaN 0
> 1 NaN 0
> 1 87 1
> 1 45 2
> 1 NaN 2
> 1 NaN 2
> 1 15 3
> 5 6 1
> 5 NaN 1
> 5 NaN 1
> 5 6 2
> 5 8 3
> 5 NaN 3
>
> Thanks again in advance.

Here's my not very efficient version, better versions will be posted below this one for sure :)

y=[1 NaN
    1 NaN
    1 NaN
    1 NaN
    1 87
    1 45
    1 NaN
    1 NaN
    1 15
    5 6
    5 NaN
    5 NaN
    5 6
    5 8
    5 NaN];
ac=[];
num=0;
a=size(y);
y=[y zeros(a(1),1)];
for b=1:a(1)
    ac=[ac;y(b,1:3)];
    if (isfinite(ac(b,1)) & isfinite(ac(b,2)))
        s=size(ac);
        if (unique(ac, 'rows')~=s(1))
            if ((b~=1))
                if (ac(b-1,1)==ac(b,1))
                    num=num+1;
                else
                    num=1;
                end
            end
        end
    end
    y(b,3)=num;
end
y

"Rawley " <ballheim@gmail.com> wrote in message <igoed9$fhd$1@fred.mathworks.com>...
> I have a similar code that I need a more efficient solution to. I need to count the number of times real valued elements have occurred per unique identifier. The first column is the unique identifier and the third column is the occurrence. Input:
>

Just adapt the previous code as following:

X = [ 1 NaN
    1 NaN
    1 87
    1 45
    1 NaN
    1 15
    5 6
    5 NaN
    5 6
    5 8
    5 NaN]

isvalid = ~isnan(X(:,2));
[~, I J] = unique(X(isvalid,1),'first');
D = (2:sum(isvalid)+1)'-I(J);
Y = X;
Y(isvalid,3) = diff([0; D]);
Y(:,3) = cumsum(Y(:,3));

Y =

     1 NaN 0
     1 NaN 0
     1 87 1
     1 45 2
     1 NaN 2
     1 15 3
     5 6 1
     5 NaN 1
     5 6 2
     5 8 3
     5 NaN 3

% Bruno

"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <igosku$s11$1@fred.mathworks.com>...
> "Rawley " <ballheim@gmail.com> wrote in message <igoed9$fhd$1@fred.mathworks.com>...
> > I have a similar code that I need a more efficient solution to. I need to count the number of times real valued elements have occurred per unique identifier. The first column is the unique identifier and the third column is the occurrence. Input:
> >
>
> Just adapt the previous code as following:
>
> X = [ 1 NaN
> 1 NaN
> 1 87
> 1 45
> 1 NaN
> 1 15
> 5 6
> 5 NaN
> 5 6
> 5 8
> 5 NaN]
>
> isvalid = ~isnan(X(:,2));
> [~, I J] = unique(X(isvalid,1),'first');
> D = (2:sum(isvalid)+1)'-I(J);
> Y = X;
> Y(isvalid,3) = diff([0; D]);
> Y(:,3) = cumsum(Y(:,3));
>
> Y =
>
> 1 NaN 0
> 1 NaN 0
> 1 87 1
> 1 45 2
> 1 NaN 2
> 1 15 3
> 5 6 1
> 5 NaN 1
> 5 6 2
> 5 8 3
> 5 NaN 3
>
> % Bruno

Bruno, thanks for the solution: I had an imbalance in the second line: "> [~, I J] = unique(X(isvalid,1),'first');" so I changed it to [b,I,J]. After doing so it ran well, but I need to fix one thing. Once a new first column identifier is hit it doesn't click back to zero, it waits until the first real occurrence and clicks to 1. The output is this:

 > Y =
>
> 1 NaN 0
> 1 NaN 0
> 1 87 1
> 1 45 2
> 1 NaN 2
> 1 15 3
>---> 5 NaN 3 <---
> 5 6 1
> 5 NaN 1
> 5 6 2
> 5 8 3
> 5 NaN 3

I need this:

> Y =
>
> 1 NaN 0
> 1 NaN 0
> 1 87 1
> 1 45 2
> 1 NaN 2
> 1 15 3
> --> 5 NaN 0 <----
> 5 6 1
> 5 NaN 1
> 5 6 2
> 5 8 3
> 5 NaN 3

I'm going to need to take a few minutes to figure out these vector commands. Until then, I really appreciate the help.

I hate to follow-up with this question again, but I'm completely striking out on my own. Thanks for the help, Paulo, but I really do need a vectorized solution because my data is way too big to run loops on.

So using Bruno's code:

X = [ 1 NaN
    1 NaN
    1 87
    1 45
    1 NaN
    1 15
    5 NaN
    5 NaN
    5 6
    5 NaN
    5 6
    5 8
    5 NaN
    8 NaN
    8 NaN
    8 NaN]

isvalid = ~isnan(X(:,2));
[b, I, J] = unique(X(isvalid,1),'first');
D = (2:sum(isvalid)+1)'-I(J);
Y = X;
Y(isvalid,3) = diff([0; D]);
Y(:,3) = cumsum(Y(:,3))

Output is as follows, with what we need in parentheses:

Y =

     1 NaN 0
     1 NaN 0
     1 87 1
     1 45 2
     1 NaN 2
     1 15 3
     5 NaN 3 (0)
     5 NaN 3 (0)
     5 6 1
     5 NaN 1
     5 6 2
     5 8 3
     5 NaN 3
     8 NaN 3 (0)
     8 NaN 3 (0)
     8 NaN 3 (0)
-------------------------------
All I would need to do is have a command that would replace all of the first group of instances of NaN per first column indicator with a zero without turning all NaNs into zeros. Any help is greatly appreciated.

Try that:

isvalid = ~isnan(X(:,2));
valid = find(isvalid);
notvalid = find(~isvalid);

[~, I, J] = unique(X(valid,1),'first');
D = (2:length(valid)+1)'-I(J);
Y3 = zeros(size(X,1),1);
Y3(valid) = diff([0; D]);
Y3 = cumsum(Y3);

% Correction of NaN appears before
x1valid = X(valid,1);
x1notvalid = X(notvalid,1);
x1notvalid(:,2) = 1;
[b, i1] = ismember(x1notvalid , [x1valid D], 'rows');
i1(b) = valid(i1(b));
i1(~b) = Inf;
Y3(notvalid(notvalid<i1)) = 0;

Y = [X Y3]

% Bruno

I'm not sure why my post of a simplified version of fixing the issue of restarting does not appear in TMW Newsgroup page, so I post it again. Apologize if the post appears twice:

X = [ 1 NaN
    1 NaN
    1 87
    1 45
    1 NaN
    1 15
    5 NaN
    5 NaN
    5 6
    5 NaN
    5 6
    5 8
    5 NaN
    7 1
    8 NaN
    8 NaN
    8 NaN
    9 22]

% Engine
isvalid = ~isnan(X(:,2));
isbreak = [true; diff(X(:,1))>0];
kept = isvalid | isbreak;
xk = X(kept,:);
[~, I, J] = unique(xk(:,1),'first');
I = I(J);
D = (2:sum(kept)+1)'-I-isnan(xk(I,2));
Y3 = zeros(size(X,1),1);
Y3(kept) = diff([0; D]);
Y3 = cumsum(Y3);
Y = [X Y3]

% Bruno

Bruno = lifesaver. Thank you very very much!