Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
faster code

Subject: faster code

From: edward kabanyas

Date: 26 Apr, 2012 04:01:37

Message: 1 of 4

Hi all,

Previously I have asked about a chain process for many matrices, to get continuously data within them (http://www.mathworks.com/matlabcentral/newsreader/view_thread/319183)

Finally, I used a looping procedure to do it. As simple case, as the following example:


data(1).data1 = [1 2 3 4];
data(2).data1 = [3 1 5 6];
data(3).data1 = [1 10 3 4];
data(4).data1 = [1 3 5 4 2 7];
data(5).data1 = [4 5 7 1];
data(6).data1 = [1 7 5 4];
data(7).data1 = [1 4 5 7];

%real data is structure, therefore the above example is also a structure.

data=data';

idx =0;
for i=1:1:size(data,1)-1
  
    
    bidx =0;
   
        if i ==1
        for ii=1:1:size(data(i).data1,2)
        
     % 1st and 2nd layer
            Bx = (data(i+1).data1==data(i).data1(ii));
            
            if any(Bx)
                idx =idx+1;
                bidx =bidx+1;
                Sim1(idx).nest.temp = [data(i).data1(ii) data(i+1).data1(find(Bx))];
                ReadB(bidx,:) =[data(i+1).data1(find(Bx)) idx];
           
            end
        end
            
     % 2nd and next layers
    
        else
            bidx =0;
            for ii=1:1:size(data(i).data1,2)
            Bx = (data(i+1).data1==data(i).data1(ii));
            ReadB1 =ReadB;
            if any(Bx)
                
                Bxx = (ReadB1(:,1)==data(i+1).data1(find(Bx)));
                
                if any(Bxx)
                bidx=bidx+1;
                Sim1(ReadB(find(Bxx),2)).nest.temp = [Sim1(ReadB(find(Bxx),2)).nest.temp data(i+1).data1(find(Bx))];
                ReadB2(bidx,:) =[data(i+1).data1(find(Bx)) ReadB(find(Bxx),2)];
                
                
                else
                
                    idx = idx+1;
                    bidx =bidx+1;
                    Sim1(idx).nest.temp=[data(i).data1(ii) data(i+1).data1(find(Bx))];
                    ReadB2(bidx,:) =[data(i+1).data1(find(Bx)) idx];
                                      
                end
                      
            end
            end
            ReadB =ReadB2;
             bidx =0;
            
        end
         bidx =0;
end

Problem: the above example is small data file, in reality I analyzed about 100 Mb data files and the current code takes very long time (about 30 minute). Do you have any suggestion to modify this code, to become faster ?

Thank you,

Edward
                
               
              
        
        

Subject: faster code

From: Bruno Luong

Date: 26 Apr, 2012 06:47:07

Message: 2 of 4

data(1).a = [1 2 3 4];
data(2).a = [3 1 5 6];
data(3).a = [1 10 3 4];
data(4).a = [1 3 5 4 2 7];
data(5).a = [4 5 7 1];
data(6).a = [1 7 5 4];
data(7).a = [1 4 5 7];

data

%% Engine
c = {data.a};
[u, ~, V]= unique([c{:}]);
I = arrayfun(@(i) i+zeros(size(data(i).a)), 1:length(data), 'unif', 0);
I = [I{:}];
J = arrayfun(@(i) 1:length(data(i).a), 1:length(data), 'unif', 0);
J = [J{:}];
n = length(I);
loc = accumarray(V(:), (1:n)', [], @(i) {sort(i)})';
I = cellfun(@(x) I(x), loc, 'unif', 0);
J = cellfun(@(x) J(x), loc, 'unif', 0);
n = cellfun('length', I);
val = arrayfun(@(v,n) v+zeros(1,n), u, cellfun('length', I), 'unif', 0);
S = struct('val', val, ...
           'I', I, ...
           'J', J);
       
% Display
for s = S
    fprintf('similar: %s == [', mat2str(s.val));
    fprintf('data(%i).a(%i) ', [s.I; s.J]);
    fprintf(']\n');
end

% Bruno

Subject: faster code

From: edward kabanyas

Date: 26 Apr, 2012 07:57:07

Message: 3 of 4

Hi Bruno !

Thanks for your reply. It is really nice code. However, allow me to ask in more detail to my real code. My real code is as follow:

%--------code for checking consecutive clouds
%--------to further calculate the life time of cloud

load('200111IMCtest.mat'); %contain s structure,

idx =0;
for i=1:1:size(s,2)-1
    
    bidx =0;
        
        % for Layer-1 (hour-1) and Layer-2 (hour-2)
             
        if i ==1
        for ii=1:1:size(s(i).test.nest,1)
          
            %chek the possibility of iith cloud in first layer (1st hour) overlap with one of
            %clouds in sedond layer (2nd hour). Cloud is assumed overlap
            %if cx (percentage of overlap pixel) > 70%
            
            for k=1:size(s(i+1).test.nest,1)
             C(k) =size(intersect(s(i).test.nest(ii).PixelList, s(i+1).test.nest(k).PixelList,'rows'),1)/size(s(i).test.nest(ii).PixelList,1)*100;
            end
             [cx,ix]=nanmax(C);
            
            if cx > 70
                idx =idx+1;
                bidx =bidx+1;
                Sim1(idx).nest= [s(i).test.nest(ii) s(i+1).test.nest(ix)];
                ReadB(bidx,:) =[s(i+1).test.nest(ix).Centroid(1) s(i+1).test.nest(ix).Centroid(2) idx];
           
            end
        end
            
    
         %second layer and soon;
        else
            bidx =0;
            for ii=1:1:size(s(i).test.nest,1)
            for k=1:size(s(i+1).test.nest,1)
             C(k) =size(intersect(s(i).test.nest(ii).PixelList, s(i+1).test.nest(k).PixelList,'rows'),1)/size(s(i).test.nest(ii).PixelList,1)*100;
            end
             [cx,ix]=nanmax(C);
             
            ReadB1 =ReadB;
            
             if cx > 70
                   
                [C,ia,ib] = intersect(ReadB1(:,1:2),s(i).test.nest(ii).Centroid,'rows');
                
                if ia >0
                bidx=bidx+1;
              
                Sim1(ReadB(ia,3)).nest= [Sim1(ReadB(ia,3)).nest s(i+1).test.nest(ix)];
                ReadB2(bidx,:) =[s(i+1).test.nest(ix).Centroid(1) s(i+1).test.nest(ix).Centroid(2) ReadB(ia,3)];
                
                
                else
                
                    idx = idx+1;
                    bidx =bidx+1;
                    Sim1(idx).nest=[s(i).test.nest(ii) s(i+1).test.nest(ix)];
                    ReadB2(bidx,:) =[s(i+1).test.nest(ix).Centroid(1) s(i+1).test.nest(ix).Centroid(2) idx];
                
                        
                end
            
           
            end
            end
            ReadB =ReadB2;
            bidx =0;
            
            
        end
        
    
     bidx =0;

    
   
end


% ---code finish

Could you help me to adopt your proposed code for the above code ?

Thanks very much for help.

EDward

Subject: faster code

From: edward kabanyas

Date: 26 Apr, 2012 08:16:07

Message: 4 of 4

The input file is:

http://www.filefactory.com/file/3bf694j0jeu1/n/200111IMCtest.mat

My main concern is:

(1) How to make this code faster
(2) effective way to match the possible cloud in the first layer (fisrt hour) with those in second layer (second-hour). So far, I did by looping as:

   %chek the possibility of iith cloud in first layer (1st hour) overlap with
            %1..kth clouds in sedond layer (2nd hour). Cloud assume overlap
            %if cx (percentage of overlap pixel) > 70%
            
            for k=1:size(s(i+1).test.nest,1)
             C(k) =size(intersect(s(i).test.nest(ii).PixelList, s(i+1).test.nest(k).PixelList,'rows'),1)/size(s(i).test.nest(ii).PixelList,1)*100;
            end
             [cx,ix]=nanmax(C);

              if cx > 70, it is assumed overlap.

The Centroid of each possible cloud is also available in structure s, but I don't know which is the effective way and the most correct way to match to image, based on their Pixel or Centeroid. Hopefully you have suggestion.

Thanks for help
Edward







"edward kabanyas" <djuky_hmi@yahoo.com> wrote in message <jnav4j$s9r$1@newscl01ah.mathworks.com>...
> Hi Bruno !
>
> Thanks for your reply. It is really nice code. However, allow me to ask in more detail to my real code. My real code is as follow:
>
> %--------code for checking consecutive clouds
> %--------to further calculate the life time of cloud
>
> load('200111IMCtest.mat'); %contain s structure,
>
> idx =0;
> for i=1:1:size(s,2)-1
>
> bidx =0;
>
> % for Layer-1 (hour-1) and Layer-2 (hour-2)
>
> if i ==1
> for ii=1:1:size(s(i).test.nest,1)
>
> %chek the possibility of iith cloud in first layer (1st hour) overlap with one of
> %clouds in sedond layer (2nd hour). Cloud is assumed overlap
> %if cx (percentage of overlap pixel) > 70%
>
> for k=1:size(s(i+1).test.nest,1)
> C(k) =size(intersect(s(i).test.nest(ii).PixelList, s(i+1).test.nest(k).PixelList,'rows'),1)/size(s(i).test.nest(ii).PixelList,1)*100;
> end
> [cx,ix]=nanmax(C);
>
> if cx > 70
> idx =idx+1;
> bidx =bidx+1;
> Sim1(idx).nest= [s(i).test.nest(ii) s(i+1).test.nest(ix)];
> ReadB(bidx,:) =[s(i+1).test.nest(ix).Centroid(1) s(i+1).test.nest(ix).Centroid(2) idx];
>
> end
> end
>
>
> %second layer and soon;
> else
> bidx =0;
> for ii=1:1:size(s(i).test.nest,1)
> for k=1:size(s(i+1).test.nest,1)
> C(k) =size(intersect(s(i).test.nest(ii).PixelList, s(i+1).test.nest(k).PixelList,'rows'),1)/size(s(i).test.nest(ii).PixelList,1)*100;
> end
> [cx,ix]=nanmax(C);
>
> ReadB1 =ReadB;
>
> if cx > 70
>
> [C,ia,ib] = intersect(ReadB1(:,1:2),s(i).test.nest(ii).Centroid,'rows');
>
> if ia >0
> bidx=bidx+1;
>
> Sim1(ReadB(ia,3)).nest= [Sim1(ReadB(ia,3)).nest s(i+1).test.nest(ix)];
> ReadB2(bidx,:) =[s(i+1).test.nest(ix).Centroid(1) s(i+1).test.nest(ix).Centroid(2) ReadB(ia,3)];
>
>
> else
>
> idx = idx+1;
> bidx =bidx+1;
> Sim1(idx).nest=[s(i).test.nest(ii) s(i+1).test.nest(ix)];
> ReadB2(bidx,:) =[s(i+1).test.nest(ix).Centroid(1) s(i+1).test.nest(ix).Centroid(2) idx];
>
>
> end
>
>
> end
> end
> ReadB =ReadB2;
> bidx =0;
>
>
> end
>
>
> bidx =0;
>
>
>
> end
>
>
> % ---code finish
>
> Could you help me to adopt your proposed code for the above code ?
>
> Thanks very much for help.
>
> EDward

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us