Path: news.mathworks.com!not-for-mail
From: "Anthony Hopf" <anthony.hopf@gmail.com>
Newsgroups: comp.soft-sys.matlab
Subject: Re: Index matrix to sparse matrix... saving memory
Date: Wed, 9 Jun 2010 14:59:08 +0000 (UTC)
Organization: The MathWorks, Inc.
Lines: 208
Message-ID: <huoa7s$dt4$1@fred.mathworks.com>
References: <htrfev$fo7$1@fred.mathworks.com> <htrk32$2vm$1@fred.mathworks.com> <htrlad$in3$1@fred.mathworks.com> <htrmbl$nn6$1@fred.mathworks.com> <hts2hl$3tk$1@fred.mathworks.com> <htt9rq$1s8$1@fred.mathworks.com> <htthum$sg$1@fred.mathworks.com> <httk3s$hb0$1@fred.mathworks.com> <httl5l$ns3$1@fred.mathworks.com> <hum6no$7rt$1@fred.mathworks.com> <hum8vg$65s$1@fred.mathworks.com> <huo80j$c5n$1@fred.mathworks.com>
Reply-To: "Anthony Hopf" <anthony.hopf@gmail.com>
NNTP-Posting-Host: webapp-02-blr.mathworks.com
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: fred.mathworks.com 1276095548 14244 172.30.248.37 (9 Jun 2010 14:59:08 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Wed, 9 Jun 2010 14:59:08 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 1899184
Xref: news.mathworks.com comp.soft-sys.matlab:643476

So I think a couple lines fixed the problem:

%%%%%%
bin_size = 100;
r_bin_max = ceil((((nanmax(double(r_4d(:))))))/bin_size);
r_bin = (0:bin_size:r_bin_max*bin_size);%[m]
[null,bin] = histc(r_4d(:),r_bin);
bin(bin==0)=max(bin(:))+1;
bool = sparse(1:numel(bin), (bin),true);
bool = bool(:,1:end-1);

I added the r_4d(:) so I can work with 3d matrices, and I took all the values of bin =0 and changed their value to something outside of the actual r_4d matrix index values and then lopped them off at the end.

Again, Thank you for all of your help.  Now to apply this to r,theta, and phi... I think i am going to do this sequentially.

Anthony


"Anthony Hopf" <anthony.hopf@gmail.com> wrote in message <huo80j$c5n$1@fred.mathworks.com>...
> Thank you Matt and Bruno!!  I apologize for not staying with this topic, but I have quite a bit on my plate... and I am taking your advice, and TRYING to learn.  You have both been very helpful and I hope you won't give up on me yet!!
> 
> The reason I didn't stick with Matt's suggestion is I could never get values into the cells I and J, and then I got pulled from my work (circumstances that I do not wish to bring up).
> 
> I am working with your suggestions and trying to apply the code to a 3d matrix, but first I started playing with your code to see what happens if there are NaN values within r and I run into a problem that I thought I could explain, the error I receive is:
> 
> ??? Error using ==> sparse
> Index into matrix must be positive.
> 
> I realize the problem is that bin will have a 0 values due to the fact there is an out of bounds points... so I thought a remedy would be:
> 
> r_4d=peaks(50)+7; % + 7 to make data positive
> r_4d(25:30,25:30) = NaN;
> bin_size = 0.1;
> %[ index_range ] = range_bin_sparse( r_4d, bin_size );
> 
> % Here is a way to use HISTC
> nbins = 151;%ceil(max(r_4d(:))/bin_size);
> r_bin = (0:nbins)*bin_size;
> [null,bin] = histc(r_4d,r_bin);
> bool = sparse(1:nnz(bin), nonzeros(bin),true);
> 
> As you notice I have modified r_4d(25,25) to be a NaN value and commented out index_range (this line also perplexes me).  My final two changes are in the sparse command where I try and compensate for the fact that there are out of bounds points... this seems to mess things up, when I go to a much larger data set with a 3d matrix r_4d the code runs, the number of nnz points in bool matches the number of ~isnan(r_4d) so I know histc is still working... but when I index using bool the returned values are not correct.
> 
> for the above code:
> without NaNs:                                        
> bool(:,50)                                            
> 
> ans =                                                 
> 
>  (525,1)        1                                     
>  (526,1)        1
>  (527,1)        1
>  (528,1)        1
>  (580,1)        1
>  (826,1)        1
>  (830,1)        1
> (1013,1)        1
> (1064,1)        1
> (1206,1)        1
> (1406,1)        1
> (1517,1)        1
> 
> with NaNs:
> bool(:,50)
> 
> ans =
> 
>  (525,1)        1
>  (526,1)        1
>  (527,1)        1
>  (528,1)        1
>  (580,1)        1
>  (826,1)        1
>  (830,1)        1
> (1013,1)        1
> (1064,1)        1
> (1206,1)        1
> (1382,1)        1
> (1481,1)        1
> 
> Can you please give me a little more advice on this?
> 
> Thank you
> 
> "Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <hum8vg$65s$1@fred.mathworks.com>...
> > First, this a an month-old thread. I'm sorry but if you haven't not pay enough attention to what has been suggested for month (including Matt's remark about the miss-use of sparse), the a lot time is waste.
> > 
> > Now it will be the last time to write something on this topic, then I'll move on. Here we go:
> > 
> > First I change your original function by removing the garbage inside, and also change one comparison test (from > to >=) to match HISTC.
> > 
> > %%
> > function [ index_range ] = range_bin_sparse( r_4d, bin_size )
> > %RANGE_BIN Identifies the index values in r_4d that are within a bin of a given size
> > % each radar is done individually and the ouput file is a sparse
> > % matrix
> > % [ range_index ] = range_bin_sparse( r_4d, bin_size )
> > 
> > %single range bin binning of a single radar
> > r_bin_max = ceil((((max(r_4d(:)))))/bin_size);
> > r_bin = (bin_size/2:bin_size:r_bin_max*bin_size+bin_size/2);%[m]
> > index_range = zeros(length(r_bin),250);
> > 
> > %filter variables
> > rH = r_bin + bin_size/2;
> > rL = r_bin - bin_size/2;
> > %find exceptable values for each range bin
> > for v=1:length(r_bin)
> >     index_temp = find((r_4d) >= rL(v) & (r_4d) <rH(v)); % change here
> >     index_range(v,1:length(index_temp)) = index_temp;
> > end
> > index_range = sparse(index_range');
> > 
> > end
> > 
> > % Now here is the data I use to test
> > r_4d=peaks(50)+7; % + 7 to make data positive
> > bin_size = 0.1;
> > [ index_range ] = range_bin_sparse( r_4d, bin_size );
> > 
> > % Here is a way to use HISTC
> > nbins = ceil(max(r_4d(:))/bin_size);
> > edges = (0:nbins)*bin_size;
> > [~,bin] = histc(r_4d,edges);
> > bool = sparse(1:numel(bin), bin,true);
> > 
> > % Note the the logical sparse matrix BOOL is not exactly like range, but it contains the
> > % same information, if you want to find which index on the 50th bins, you can do this:
> > 
> > >> index_range(:,50)
> > 
> > ans =
> > 
> >    (1,1)            525
> >    (2,1)            526
> >    (3,1)            527
> >    (4,1)            528
> >    (5,1)            580
> >    (6,1)            826
> >    (7,1)            830
> >    (8,1)           1013
> >    (9,1)           1064
> >   (10,1)           1206
> >   (11,1)           1406
> >   (12,1)           1517
> > 
> > % Or this:
> > 
> > >> find(bool(:,50))
> > 
> > ans =
> > 
> >          525
> >          526
> >          527
> >          528
> >          580
> >          826
> >          830
> >         1013
> >         1064
> >         1206
> >         1406
> >         1517
> > 
> > % You see that they contain the same thing.
> > 
> > In practice, next calculation does not need the FIND command, because BOOL can be used as matrix for LOGICAL indexing, that's what Matt suggest you. For example figure out values of thet array r_4d() falling the 50th bins
> > 
> > >> r_4d(bool(:,50)) 
> > 
> > ans =
> > 
> >     4.9948
> >     4.9059
> >     4.9003
> >     4.9737
> >     4.9775
> >     4.9397
> >     4.9658
> >     4.9965
> >     4.9447
> >     4.9068
> >     4.9602
> >     4.9365
> > 
> > % Check if they fall inside the bins
> > 
> >  >> r_4d(bool(:,50))>=edges(50) & r_4d(bool(:,50))<edges(51)
> > 
> > ans =
> > 
> >      1
> >      1
> >      1
> >      1
> >      1
> >      1
> >      1
> >      1
> >      1
> >      1
> >      1
> >      1
> > 
> > >>
> > 
> > % Bruno