Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Set interestions/Searching in large Matrices

Subject: Set interestions/Searching in large Matrices

From: Anthony Hopf

Date: 26 Apr, 2010 12:30:11

Message: 1 of 14

Ok, I posted a similar question a few weeks ago, but have determined that my old approach would have introduced unnecessary errors into my calculations. What I am trying to do now is set intersections/searching within a large matrix. I've created the following code to illustrate the issue... excuse the code for any inefficiencies in creating the matrices, it was hastily created.

The point of this exercise was to investigate how long it would take to find points within a value matrix based on criteria evaluated over index matrices. I have a sub sampled matrix val_ss that is indexed by r_ss and phi_ss. What I would like to do is sample over val_ss and populate a sampling matrix val_s using relational and logical operators comparing the sub sample matrix indexes (r_ss,phi_ss) to the sample matrix indexes (r_s,phi_s).

Even with a 64bit quad core machine running at ~3GHz in a Linux environment the set operation/search operation takes ~0.25sec each iteration on a large matrix. Making this matrix smaller helps, but that defeats my purpose.

Other than outsourcing this project to a C script... does anyone see a more efficient way to implement this operation?

Code:
%Subsampled matrix
deltar_ss = 10;
deltaphi_ss = 0.1;

[phi_ss,r_ss] = meshgrid(0.5*deltaphi_ss:deltaphi_ss:(2000-(0.5*deltaphi_ss)),0.5*deltar_ss:deltar_ss:(4000-(0.5*deltar_ss)));
val_ss = zeros(size(r_ss,1),size(phi_ss,2));
for i = 1:length(phi_ss);
    if (~~(i/2-floor(i/2)))
val_ss(:,i) = 1;
    end
end
clear i;

%%
%Resample the data
deltar_s = 100;
deltaphi_s = 1;
[phi_s,r_s] = meshgrid(0.5*deltaphi_s:deltaphi_s:(2000-(0.5*deltaphi_s)),0.5*deltar_s:deltar_s:(4000-(0.5*deltar_s)));
val_s = zeros(size(r_s,1),size(phi_s,2));
for i = 1:size(val_s,2)
    for j = 1:size(val_s,1)
        tic;
        val_s(j,i) = sum(val_ss(phi_ss > phi_s(j,i) - deltaphi_s/2 & phi_ss < phi_s(j,i) + deltaphi_s/2 & r_ss > r_s(j,i) - deltar_s/2 & r_ss < r_s(j,i) + deltar_s/2)); %needs to be faster or vectorized.
        toc;
    end
end

Subject: Set interestions/Searching in large Matrices

From: Jan Simon

Date: 26 Apr, 2010 14:02:21

Message: 2 of 14

Dear Anthony!

> deltar_ss = 10;
> deltaphi_ss = 0.1;
> [phi_ss, r_ss] = meshgrid(0.5*deltaphi_ss:deltaphi_ss:(2000-(0.5*deltaphi_ss)),0.5*deltar_ss:deltar_ss:(4000-(0.5*deltar_ss)));

Replace this:
> val_ss = zeros(size(r_ss,1),size(phi_ss,2));
> for i = 1:length(phi_ss);
> if (~~(i/2-floor(i/2)))
> val_ss(:,i) = 1;
> end
> end

with this:
  val_ss = zeros(size(r_ss,1),size(phi_ss,2));
  val_ss(:, 1:2:end) = 1;
You do not need a complicated operation with the slow double not, division and FLOOR function, if the wanted pattern is trivial.

> deltar_s = 100;
> deltaphi_s = 1;
> [phi_s,r_s] = meshgrid(0.5*deltaphi_s:deltaphi_s:(2000-(0.5*deltaphi_s)),0.5*deltar_s:deltar_s:(4000-(0.5*deltar_s)));
> val_s = zeros(size(r_s,1),size(phi_s,2));
> for i = 1:size(val_s,2)
> for j = 1:size(val_s,1)
> tic;
> val_s(j,i) = sum(val_ss(phi_ss > phi_s(j,i) - deltaphi_s/2 & phi_ss < phi_s(j,i) + deltaphi_s/2 & r_ss > r_s(j,i) - deltar_s/2 & r_ss < r_s(j,i) + deltar_s/2));
> toc;
> end
> end

At first move all unneded calculations out of the loops, because they need not be calculated repeatedly:
  deltaphi_s/2, deltar_s/2
Create temporary arrays for these calculations and access the indices instead of an elementwise processing:
  phi_s(j,i) - deltaphi_s/2
  phi_s(j,i) + deltaphi_s/2
  r_s(j,i) - deltar_s/2
  r_s(j,i) + deltar_s/2

Then I'd start to think of vectorization!

Jan

Subject: Set interestions/Searching in large Matrices

From: Anthony Hopf

Date: 26 Apr, 2010 14:32:20

Message: 3 of 14

Jan,

Wow, that was a quick response and thank you for the tips! I'll post back with some results.

T

"Jan Simon" <matlab.THIS_YEAR@nMINUSsimon.de> wrote in message <hr46dd$eqs$1@fred.mathworks.com>...
> Dear Anthony!
>
> > deltar_ss = 10;
> > deltaphi_ss = 0.1;
> > [phi_ss, r_ss] = meshgrid(0.5*deltaphi_ss:deltaphi_ss:(2000-(0.5*deltaphi_ss)),0.5*deltar_ss:deltar_ss:(4000-(0.5*deltar_ss)));
>
> Replace this:
> > val_ss = zeros(size(r_ss,1),size(phi_ss,2));
> > for i = 1:length(phi_ss);
> > if (~~(i/2-floor(i/2)))
> > val_ss(:,i) = 1;
> > end
> > end
>
> with this:
> val_ss = zeros(size(r_ss,1),size(phi_ss,2));
> val_ss(:, 1:2:end) = 1;
> You do not need a complicated operation with the slow double not, division and FLOOR function, if the wanted pattern is trivial.
>
> > deltar_s = 100;
> > deltaphi_s = 1;
> > [phi_s,r_s] = meshgrid(0.5*deltaphi_s:deltaphi_s:(2000-(0.5*deltaphi_s)),0.5*deltar_s:deltar_s:(4000-(0.5*deltar_s)));
> > val_s = zeros(size(r_s,1),size(phi_s,2));
> > for i = 1:size(val_s,2)
> > for j = 1:size(val_s,1)
> > tic;
> > val_s(j,i) = sum(val_ss(phi_ss > phi_s(j,i) - deltaphi_s/2 & phi_ss < phi_s(j,i) + deltaphi_s/2 & r_ss > r_s(j,i) - deltar_s/2 & r_ss < r_s(j,i) + deltar_s/2));
> > toc;
> > end
> > end
>
> At first move all unneded calculations out of the loops, because they need not be calculated repeatedly:
> deltaphi_s/2, deltar_s/2
> Create temporary arrays for these calculations and access the indices instead of an elementwise processing:
> phi_s(j,i) - deltaphi_s/2
> phi_s(j,i) + deltaphi_s/2
> r_s(j,i) - deltar_s/2
> r_s(j,i) + deltar_s/2
>
> Then I'd start to think of vectorization!
>
> Jan

Subject: Set interestions/Searching in large Matrices

From: Anthony Hopf

Date: 26 Apr, 2010 17:23:05

Message: 4 of 14

Jan,

I think I understand what you mean by doing the process by linear index but I don't know how to implement this technique.

using the find function you can retrieve the indexes where the evaluation is correct, but how do you do this for every point in my matrix... I am hoping to expand this to a much large 3D matrix where the number of elements summed increases as the indexes increase from the starting point.

I guess my mind can't wrap my head around vectorized linear indexing.... maybe someone has an example?

T

"Jan Simon" <matlab.THIS_YEAR@nMINUSsimon.de> wrote in message <hr46dd$eqs$1@fred.mathworks.com>...
> Dear Anthony!
>
> > deltar_ss = 10;
> > deltaphi_ss = 0.1;
> > [phi_ss, r_ss] = meshgrid(0.5*deltaphi_ss:deltaphi_ss:(2000-(0.5*deltaphi_ss)),0.5*deltar_ss:deltar_ss:(4000-(0.5*deltar_ss)));
>
> Replace this:
> > val_ss = zeros(size(r_ss,1),size(phi_ss,2));
> > for i = 1:length(phi_ss);
> > if (~~(i/2-floor(i/2)))
> > val_ss(:,i) = 1;
> > end
> > end
>
> with this:
> val_ss = zeros(size(r_ss,1),size(phi_ss,2));
> val_ss(:, 1:2:end) = 1;
> You do not need a complicated operation with the slow double not, division and FLOOR function, if the wanted pattern is trivial.
>
> > deltar_s = 100;
> > deltaphi_s = 1;
> > [phi_s,r_s] = meshgrid(0.5*deltaphi_s:deltaphi_s:(2000-(0.5*deltaphi_s)),0.5*deltar_s:deltar_s:(4000-(0.5*deltar_s)));
> > val_s = zeros(size(r_s,1),size(phi_s,2));
> > for i = 1:size(val_s,2)
> > for j = 1:size(val_s,1)
> > tic;
> > val_s(j,i) = sum(val_ss(phi_ss > phi_s(j,i) - deltaphi_s/2 & phi_ss < phi_s(j,i) + deltaphi_s/2 & r_ss > r_s(j,i) - deltar_s/2 & r_ss < r_s(j,i) + deltar_s/2));
> > toc;
> > end
> > end
>
> At first move all unneded calculations out of the loops, because they need not be calculated repeatedly:
> deltaphi_s/2, deltar_s/2
> Create temporary arrays for these calculations and access the indices instead of an elementwise processing:
> phi_s(j,i) - deltaphi_s/2
> phi_s(j,i) + deltaphi_s/2
> r_s(j,i) - deltar_s/2
> r_s(j,i) + deltar_s/2
>
> Then I'd start to think of vectorization!
>
> Jan

Subject: Set interestions/Searching in large Matrices

From: Anthony Hopf

Date: 26 Apr, 2010 20:02:05

Message: 5 of 14

Ok, so I have been staring at this for awhile and still having trouble. The section of the code that really needs help is this:
phi_low = phi_s - d_phi;
phi_hi = phi_s + d_phi;
r_low = r_s - d_r;
r_hi = r_s + d_r;
for i = 1:size(val_s,2)
    for j = 1:size(val_s,1)
        tic;
        val_s(j,i) = sum(val_ss(phi_ss > phi_low(j,i) & phi_ss < phi_hi(j,i) & r_ss > r_low(j,i) & r_ss < r_hi(j,i)));
        toc;
    end
end

What I would like to do is vectorize the for loop and I think I know what I need to do but don't know how to implement it. As you can see there is a range of values that I would like to sum over within my subsampled nxm matrix. In order to vectorize this I would first build a matrix that is kxv where k and v are arbitrary but smaller than nxm, this is the output matrix val_s. My next step would be to create an kxvxp matrix where p is a vector that will be the indexes that correspond to the values that fit within the range, from the nxm matrix, given by evaluating the inequality at the center of the for loops. The value of p will depend on the number of values that fit the inequality, in this case they would all be the same number of indexes, but this need not be the case. Finally I would sum over these indexes in the p vector pointing to values in the nxm matrix. Am I on the right
track? Am I looking at this "algorithm" incorrectly?

Thank you

T


"Anthony Hopf" <anthony.hopf@gmail.com> wrote in message <hr485k$f1h$1@fred.mathworks.com>...
> Jan,
>
> Wow, that was a quick response and thank you for the tips! I'll post back with some results.
>
> T
>
> "Jan Simon" <matlab.THIS_YEAR@nMINUSsimon.de> wrote in message <hr46dd$eqs$1@fred.mathworks.com>...
> > Dear Anthony!
> >
> > > deltar_ss = 10;
> > > deltaphi_ss = 0.1;
> > > [phi_ss, r_ss] = meshgrid(0.5*deltaphi_ss:deltaphi_ss:(2000-(0.5*deltaphi_ss)),0.5*deltar_ss:deltar_ss:(4000-(0.5*deltar_ss)));
> >
> > Replace this:
> > > val_ss = zeros(size(r_ss,1),size(phi_ss,2));
> > > for i = 1:length(phi_ss);
> > > if (~~(i/2-floor(i/2)))
> > > val_ss(:,i) = 1;
> > > end
> > > end
> >
> > with this:
> > val_ss = zeros(size(r_ss,1),size(phi_ss,2));
> > val_ss(:, 1:2:end) = 1;
> > You do not need a complicated operation with the slow double not, division and FLOOR function, if the wanted pattern is trivial.
> >
> > > deltar_s = 100;
> > > deltaphi_s = 1;
> > > [phi_s,r_s] = meshgrid(0.5*deltaphi_s:deltaphi_s:(2000-(0.5*deltaphi_s)),0.5*deltar_s:deltar_s:(4000-(0.5*deltar_s)));
> > > val_s = zeros(size(r_s,1),size(phi_s,2));
> > > for i = 1:size(val_s,2)
> > > for j = 1:size(val_s,1)
> > > tic;
> > > val_s(j,i) = sum(val_ss(phi_ss > phi_s(j,i) - deltaphi_s/2 & phi_ss < phi_s(j,i) + deltaphi_s/2 & r_ss > r_s(j,i) - deltar_s/2 & r_ss < r_s(j,i) + deltar_s/2));
> > > toc;
> > > end
> > > end
> >
> > At first move all unneded calculations out of the loops, because they need not be calculated repeatedly:
> > deltaphi_s/2, deltar_s/2
> > Create temporary arrays for these calculations and access the indices instead of an elementwise processing:
> > phi_s(j,i) - deltaphi_s/2
> > phi_s(j,i) + deltaphi_s/2
> > r_s(j,i) - deltar_s/2
> > r_s(j,i) + deltar_s/2
> >
> > Then I'd start to think of vectorization!
> >
> > Jan

Subject: Set interestions/Searching in large Matrices

From: Anthony Hopf

Date: 27 Apr, 2010 11:41:03

Message: 6 of 14

I had the big picture in mind when I posted this example code and I kept thinking about how I could do the resampling contingent on evaluating an inequality based on other indexing matrices. Speaking with one of my lab mates it had become clear that in doing the sampling this way I will see the same sampling errors I would if I just did a transform from cartesian coordinates to spherical, interpolate, and sample using evenly spaced indexes.

Comparing this to the problem I have posted here... i do not need to index the matrix v_ss in terms of r_ss and phi_ss indexes, but I can to the indexing in terms of v_ss since the indexing is uniform (the nth index is (0.5+n)*deltar_ss in the first column). I can then return to cartesian coordinates after sampling by interpolation. Now I just need to wrap my head around this problem... I'm sure I will find some literature or tutorial on matlab central.

Thank you

T


"Jan Simon" <matlab.THIS_YEAR@nMINUSsimon.de> wrote in message <hr46dd$eqs$1@fred.mathworks.com>...
> Dear Anthony!
>
> > deltar_ss = 10;
> > deltaphi_ss = 0.1;
> > [phi_ss, r_ss] = meshgrid(0.5*deltaphi_ss:deltaphi_ss:(2000-(0.5*deltaphi_ss)),0.5*deltar_ss:deltar_ss:(4000-(0.5*deltar_ss)));
>
> Replace this:
> > val_ss = zeros(size(r_ss,1),size(phi_ss,2));
> > for i = 1:length(phi_ss);
> > if (~~(i/2-floor(i/2)))
> > val_ss(:,i) = 1;
> > end
> > end
>
> with this:
> val_ss = zeros(size(r_ss,1),size(phi_ss,2));
> val_ss(:, 1:2:end) = 1;
> You do not need a complicated operation with the slow double not, division and FLOOR function, if the wanted pattern is trivial.
>
> > deltar_s = 100;
> > deltaphi_s = 1;
> > [phi_s,r_s] = meshgrid(0.5*deltaphi_s:deltaphi_s:(2000-(0.5*deltaphi_s)),0.5*deltar_s:deltar_s:(4000-(0.5*deltar_s)));
> > val_s = zeros(size(r_s,1),size(phi_s,2));
> > for i = 1:size(val_s,2)
> > for j = 1:size(val_s,1)
> > tic;
> > val_s(j,i) = sum(val_ss(phi_ss > phi_s(j,i) - deltaphi_s/2 & phi_ss < phi_s(j,i) + deltaphi_s/2 & r_ss > r_s(j,i) - deltar_s/2 & r_ss < r_s(j,i) + deltar_s/2));
> > toc;
> > end
> > end
>
> At first move all unneded calculations out of the loops, because they need not be calculated repeatedly:
> deltaphi_s/2, deltar_s/2
> Create temporary arrays for these calculations and access the indices instead of an elementwise processing:
> phi_s(j,i) - deltaphi_s/2
> phi_s(j,i) + deltaphi_s/2
> r_s(j,i) - deltar_s/2
> r_s(j,i) + deltar_s/2
>
> Then I'd start to think of vectorization!
>
> Jan

Subject: Set interestions/Searching in large Matrices

From: Bruno Luong

Date: 27 Apr, 2010 11:54:04

Message: 7 of 14

"Anthony Hopf" <anthony.hopf@gmail.com> wrote in message <hr6igf$2l5$1@fred.mathworks.com>...
> I had the big picture in mind when I posted this example code and I kept thinking about how I could do the resampling contingent on evaluating an inequality based on other indexing matrices. Speaking with one of my lab mates it had become clear that in doing the sampling this way I will see the same sampling errors I would if I just did a transform from cartesian coordinates to spherical, interpolate, and sample using evenly spaced indexes.
>
> Comparing this to the problem I have posted here... i do not need to index the matrix v_ss in terms of r_ss and phi_ss indexes, but I can to the indexing in terms of v_ss since the indexing is uniform (the nth index is (0.5+n)*deltar_ss in the first column). I can then return to cartesian coordinates after sampling by interpolation. Now I just need to wrap my head around this problem... I'm sure I will find some literature or tutorial on matlab central.
>

What you are doing is some sort of 2D binning. Look in FEX there are many 2d/nd histogram codes you can use to find which points fall into which rectangles. Next use accumarray() command to get the final result.

Bruno

Subject: Set interestions/Searching in large Matrices

From: Anthony Hopf

Date: 27 Apr, 2010 12:44:05

Message: 8 of 14

Bruno,

I went this route but couldn't find anything that didn't go point by point. Basically what I am doing is resampling the data in 3d space... I wanted to do directly from cart(fine grain) to spherical(course grain) in one step by sampling the cartesian grid. So far the only way I see to do this is to evaluate an inequality for every point in my spherical grid over the cartesian grid. This is element by element and takes ~0.25sec each element.

After talking with one of my lab colleagues it seems like I can do the same thing, with similar edge errors but faster by creating a matrix of interpolated cartesian data indexed uniformly in spherical coordintates and vectorize a summation into a courser grid spherically indexed matrix and transform back into a cartesian grid...also courser grid than the original cartesian grid. This all happens in 3d vectors, not I just have to figure out how to do the above transformations.

Maybe you have a file from the FEX that you had in mind... if so could you post the name or link?

Thanks for the help Bruno.

T

"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <hr6j8s$kt3$1@fred.mathworks.com>...
> "Anthony Hopf" <anthony.hopf@gmail.com> wrote in message <hr6igf$2l5$1@fred.mathworks.com>...
> > I had the big picture in mind when I posted this example code and I kept thinking about how I could do the resampling contingent on evaluating an inequality based on other indexing matrices. Speaking with one of my lab mates it had become clear that in doing the sampling this way I will see the same sampling errors I would if I just did a transform from cartesian coordinates to spherical, interpolate, and sample using evenly spaced indexes.
> >
> > Comparing this to the problem I have posted here... i do not need to index the matrix v_ss in terms of r_ss and phi_ss indexes, but I can to the indexing in terms of v_ss since the indexing is uniform (the nth index is (0.5+n)*deltar_ss in the first column). I can then return to cartesian coordinates after sampling by interpolation. Now I just need to wrap my head around this problem... I'm sure I will find some literature or tutorial on matlab central.
> >
>
> What you are doing is some sort of 2D binning. Look in FEX there are many 2d/nd histogram codes you can use to find which points fall into which rectangles. Next use accumarray() command to get the final result.
>
> Bruno

Subject: Set interestions/Searching in large Matrices

From: Bruno Luong

Date: 27 Apr, 2010 14:05:23

Message: 9 of 14

%Subsampled matrix
deltar_ss = 10;
deltaphi_ss = 0.1;

[phi_ss,r_ss] = meshgrid(0.5*deltaphi_ss:deltaphi_ss:(2000-(0.5*deltaphi_ss)),0.5*deltar_ss:deltar_ss:(4000-(0.5*deltar_ss)));
val_ss = zeros(size(r_ss,1),size(phi_ss,2));
for i = 1:length(phi_ss);
    if (~~(i/2-floor(i/2)))
val_ss(:,i) = 1;
    end
end
clear i;

%%
%Resample the data
deltar_s = 100;
deltaphi_s = 1;

%% New engine using HISTCN and ACCUMARRAY
% FEX: http://www.mathworks.com/matlabcentral/fileexchange/23897

phi_s = 0.5*deltaphi_s:deltaphi_s:(2000-(0.5*deltaphi_s));
r_s = 0.5*deltar_s:deltar_s:(4000-(0.5*deltar_s));

phir = [r_ss(:) phi_ss(:)];
[count edges mid loc] = histcn(phir, r_s, phi_s);
idx = all(loc,2);
val_s = accumarray(loc(idx,:), val_ss(idx));

Bruno

Subject: Set interestions/Searching in large Matrices

From: Bruno Luong

Date: 28 Apr, 2010 06:44:05

Message: 10 of 14

"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message
> %% New engine using HISTCN and ACCUMARRAY
> % FEX: http://www.mathworks.com/matlabcentral/fileexchange/23897
>
> phi_s = 0.5*deltaphi_s:deltaphi_s:(2000-(0.5*deltaphi_s));
> r_s = 0.5*deltar_s:deltar_s:(4000-(0.5*deltar_s));
>
> phir = [r_ss(:) phi_ss(:)];
> [count edges mid loc] = histcn(phir, r_s, phi_s);
> idx = all(loc,2);
> val_s = accumarray(loc(idx,:), val_ss(idx));
>

Sorry, it might be necessary to make a slight adjustment of the edges as following:

%%
%Resample the data
deltar_s = 100;
deltaphi_s = 1;

phi_s = 0:deltaphi_s:2000;
r_s = 0:deltar_s:4000;

phir = [r_ss(:) phi_ss(:)];
[count edges mid loc] = histcn(phir, r_s, phi_s);
val_s = accumarray(loc, val_ss(:), [length(r_s) length(phi_s)]-1);

Bruno

Subject: Set interestions/Searching in large Matrices

From: Anthony Hopf

Date: 5 May, 2010 12:36:05

Message: 11 of 14

"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <hr8lfl$3d9$1@fred.mathworks.com>...
> "Bruno Luong" <b.luong@fogale.findmycountry> wrote in message
> > %% New engine using HISTCN and ACCUMARRAY
> > % FEX: http://www.mathworks.com/matlabcentral/fileexchange/23897
> >
> > phi_s = 0.5*deltaphi_s:deltaphi_s:(2000-(0.5*deltaphi_s));
> > r_s = 0.5*deltar_s:deltar_s:(4000-(0.5*deltar_s));
> >
> > phir = [r_ss(:) phi_ss(:)];
> > [count edges mid loc] = histcn(phir, r_s, phi_s);
> > idx = all(loc,2);
> > val_s = accumarray(loc(idx,:), val_ss(idx));
> >
>
> Sorry, it might be necessary to make a slight adjustment of the edges as following:
>
> %%
> %Resample the data
> deltar_s = 100;
> deltaphi_s = 1;
>
> phi_s = 0:deltaphi_s:2000;
> r_s = 0:deltar_s:4000;
>
> phir = [r_ss(:) phi_ss(:)];
> [count edges mid loc] = histcn(phir, r_s, phi_s);
> val_s = accumarray(loc, val_ss(:), [length(r_s) length(phi_s)]-1);
>
> Bruno

Bruno,

In order to use this procedure the data would need to have a common delta in each direction correct? What if I had sorted data without a common spacing between data points? is here any way I could get to a corrected matrix easily?

Thanks again... I understand your function much better now.

T

Subject: Set interestions/Searching in large Matrices

From: Bruno Luong

Date: 5 May, 2010 12:45:23

Message: 12 of 14

"Anthony Hopf" <anthony.hopf@gmail.com> wrote in message <hrronl$sds$1@fred.mathworks.com>...
> "Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <hr8lfl$3d9$1@fred.mathworks.com>...
> > "Bruno Luong" <b.luong@fogale.findmycountry> wrote in message
> > > %% New engine using HISTCN and ACCUMARRAY
> > > % FEX: http://www.mathworks.com/matlabcentral/fileexchange/23897
> > >
> > > phi_s = 0.5*deltaphi_s:deltaphi_s:(2000-(0.5*deltaphi_s));
> > > r_s = 0.5*deltar_s:deltar_s:(4000-(0.5*deltar_s));
> > >
> > > phir = [r_ss(:) phi_ss(:)];
> > > [count edges mid loc] = histcn(phir, r_s, phi_s);
> > > idx = all(loc,2);
> > > val_s = accumarray(loc(idx,:), val_ss(idx));
> > >
> >
> > Sorry, it might be necessary to make a slight adjustment of the edges as following:
> >
> > %%
> > %Resample the data
> > deltar_s = 100;
> > deltaphi_s = 1;
> >
> > phi_s = 0:deltaphi_s:2000;
> > r_s = 0:deltar_s:4000;
> >
> > phir = [r_ss(:) phi_ss(:)];
> > [count edges mid loc] = histcn(phir, r_s, phi_s);
> > val_s = accumarray(loc, val_ss(:), [length(r_s) length(phi_s)]-1);
> >
> > Bruno
>
> Bruno,
>
> In order to use this procedure the data would need to have a common delta in each direction correct?

No, it's not correct. The (linear) grid is required to be strictly monotonic, that's all. There is no requirement to be a regular grid ("common delta" as you put it).

Bruno

Subject: Set interestions/Searching in large Matrices

From: Anthony Hopf

Date: 8 Jun, 2010 10:47:04

Message: 13 of 14

Bruno,

I hate to rehash this, but the code I am using now is inefficient (and you mentioned that this would be a more effective way to bin my data). I have a 3d matrix or values indexed by r,theta,phi (each are 3d matrices). I would like to bin this 3d value matrix conditioned on the three other matrices (r, theta, and phi). On the matrix r I would like to collect the index values binned by a binning vector:

r_edge = -(binsize/2):binsize:(N*binsize)-(binsize/2);

can I do this with histc? It looks like I need to reshape the 3d matrix r into a vector to do the binning with histc (or maybe I don't and I just don't understand the indexing in matlab, if I reshape does matlab remember the old index values that will correspond to the 3d value matrix?)

I will then bin on theta and phi:

theta_edge = -(tbin/2):tbin:(M*tbin)-(tbin/2);
phi_edge = -(pbin/2):pbin:(P*pbin)-(pbin/2);

in this case I was that I would use your procedure with histcn, but again it looks like I need to reshape my theta and phi 3d matrices and don't want to lose the corresponding index values.

I'm not trying to be laze ( i just feel like I should "stand on the shoulders of giants" by accessing your understanding of these functions)

Thank you,

Anthony

"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <hrrp93$4ok$1@fred.mathworks.com>...
> "Anthony Hopf" <anthony.hopf@gmail.com> wrote in message <hrronl$sds$1@fred.mathworks.com>...
> > "Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <hr8lfl$3d9$1@fred.mathworks.com>...
> > > "Bruno Luong" <b.luong@fogale.findmycountry> wrote in message
> > > > %% New engine using HISTCN and ACCUMARRAY
> > > > % FEX: http://www.mathworks.com/matlabcentral/fileexchange/23897
> > > >
> > > > phi_s = 0.5*deltaphi_s:deltaphi_s:(2000-(0.5*deltaphi_s));
> > > > r_s = 0.5*deltar_s:deltar_s:(4000-(0.5*deltar_s));
> > > >
> > > > phir = [r_ss(:) phi_ss(:)];
> > > > [count edges mid loc] = histcn(phir, r_s, phi_s);
> > > > idx = all(loc,2);
> > > > val_s = accumarray(loc(idx,:), val_ss(idx));
> > > >
> > >
> > > Sorry, it might be necessary to make a slight adjustment of the edges as following:
> > >
> > > %%
> > > %Resample the data
> > > deltar_s = 100;
> > > deltaphi_s = 1;
> > >
> > > phi_s = 0:deltaphi_s:2000;
> > > r_s = 0:deltar_s:4000;
> > >
> > > phir = [r_ss(:) phi_ss(:)];
> > > [count edges mid loc] = histcn(phir, r_s, phi_s);
> > > val_s = accumarray(loc, val_ss(:), [length(r_s) length(phi_s)]-1);
> > >
> > > Bruno
> >
> > Bruno,
> >
> > In order to use this procedure the data would need to have a common delta in each direction correct?
>
> No, it's not correct. The (linear) grid is required to be strictly monotonic, that's all. There is no requirement to be a regular grid ("common delta" as you put it).
>
> Bruno

Subject: Set interestions/Searching in large Matrices

From: Anthony Hopf

Date: 8 Jun, 2010 11:21:04

Message: 14 of 14

Here is the exact code I am using. My output is a sparse matrix (to conserve memory... there are many zeros... I use ~10% of the array for indexing)

function [ index_range ] = range_bin_sparse( r_4d, bin_size )
%RANGE_BIN Identifies the index values in r_4d that are within a bin of a given size
% each radar is done individually and the ouput file is a sparse
% matrix
% [ range_index ] = range_bin_sparse( r_4d, bin_size )

%single range bin binning of a single radar
r_bin_max = ceil((((nanmax(r_4d(:)))))/bin_size);
r_bin = (bin_size/2:bin_size:r_bin_max*bin_size+bin_size/2);%[m]
index_range = zeros(length(r_bin),250);

%filter variables
rH = r_bin + bin_size/2;
rL = r_bin - bin_size/2;
%find exceptable values for each range bin
for v=1:length(r_bin)
index_temp = find((r_4d) > rL(v) & (r_4d) <rH(v));
%if max_index < length(index_range)
%max_index = length(index_range);
%end
index_range(v,1:length(index_temp)) = index_temp;
%clear index_temp;
if isempty(index_temp)
v = length(r_bin);
end
clear index_temp;
end
index_range = sparse(index_range');

end


"Anthony Hopf" <anthony.hopf@gmail.com> wrote in message <hul738$b2p$1@fred.mathworks.com>...
> Bruno,
>
> I hate to rehash this, but the code I am using now is inefficient (and you mentioned that this would be a more effective way to bin my data). I have a 3d matrix or values indexed by r,theta,phi (each are 3d matrices). I would like to bin this 3d value matrix conditioned on the three other matrices (r, theta, and phi). On the matrix r I would like to collect the index values binned by a binning vector:
>
> r_edge = -(binsize/2):binsize:(N*binsize)-(binsize/2);
>
> can I do this with histc? It looks like I need to reshape the 3d matrix r into a vector to do the binning with histc (or maybe I don't and I just don't understand the indexing in matlab, if I reshape does matlab remember the old index values that will correspond to the 3d value matrix?)
>
> I will then bin on theta and phi:
>
> theta_edge = -(tbin/2):tbin:(M*tbin)-(tbin/2);
> phi_edge = -(pbin/2):pbin:(P*pbin)-(pbin/2);
>
> in this case I was that I would use your procedure with histcn, but again it looks like I need to reshape my theta and phi 3d matrices and don't want to lose the corresponding index values.
>
> I'm not trying to be laze ( i just feel like I should "stand on the shoulders of giants" by accessing your understanding of these functions)
>
> Thank you,
>
> Anthony
>
> "Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <hrrp93$4ok$1@fred.mathworks.com>...
> > "Anthony Hopf" <anthony.hopf@gmail.com> wrote in message <hrronl$sds$1@fred.mathworks.com>...
> > > "Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <hr8lfl$3d9$1@fred.mathworks.com>...
> > > > "Bruno Luong" <b.luong@fogale.findmycountry> wrote in message
> > > > > %% New engine using HISTCN and ACCUMARRAY
> > > > > % FEX: http://www.mathworks.com/matlabcentral/fileexchange/23897
> > > > >
> > > > > phi_s = 0.5*deltaphi_s:deltaphi_s:(2000-(0.5*deltaphi_s));
> > > > > r_s = 0.5*deltar_s:deltar_s:(4000-(0.5*deltar_s));
> > > > >
> > > > > phir = [r_ss(:) phi_ss(:)];
> > > > > [count edges mid loc] = histcn(phir, r_s, phi_s);
> > > > > idx = all(loc,2);
> > > > > val_s = accumarray(loc(idx,:), val_ss(idx));
> > > > >
> > > >
> > > > Sorry, it might be necessary to make a slight adjustment of the edges as following:
> > > >
> > > > %%
> > > > %Resample the data
> > > > deltar_s = 100;
> > > > deltaphi_s = 1;
> > > >
> > > > phi_s = 0:deltaphi_s:2000;
> > > > r_s = 0:deltar_s:4000;
> > > >
> > > > phir = [r_ss(:) phi_ss(:)];
> > > > [count edges mid loc] = histcn(phir, r_s, phi_s);
> > > > val_s = accumarray(loc, val_ss(:), [length(r_s) length(phi_s)]-1);
> > > >
> > > > Bruno
> > >
> > > Bruno,
> > >
> > > In order to use this procedure the data would need to have a common delta in each direction correct?
> >
> > No, it's not correct. The (linear) grid is required to be strictly monotonic, that's all. There is no requirement to be a regular grid ("common delta" as you put it).
> >
> > Bruno

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us