http://www.mathworks.com/matlabcentral/newsreader/view_thread/240113
MATLAB Central Newsreader  taking a long time
Feed for thread: taking a long time
enus
©19942015 by MathWorks, Inc.
webmaster@mathworks.com
MATLAB Central Newsreader
http://blogs.law.harvard.edu/tech/rss
60
MathWorks
http://www.mathworks.com/images/membrane_icon.gif

Sat, 29 Nov 2008 16:35:02 +0000
taking a long time
http://www.mathworks.com/matlabcentral/newsreader/view_thread/240113#613868
Travis
I am using the distance function to find the distance between two sets of lat/longs. The data sets are 700,700 and 309 points long. It has been running for over 10 hours. Is this normal for it to take this long? Is there a quicker way to do this?

Sun, 30 Nov 2008 00:44:02 +0000
Re: taking a long time
http://www.mathworks.com/matlabcentral/newsreader/view_thread/240113#613923
Travis
OK, going on 24+ hours now. Again, is this normal? Is there a faster way?

Sun, 30 Nov 2008 02:00:18 +0000
Re: taking a long time
http://www.mathworks.com/matlabcentral/newsreader/view_thread/240113#613928
Roger Stafford
"Travis" <sinusoid2@hotmail.com> wrote in message <ggrqvm$k5l$1@fred.mathworks.com>...<br>
> I am using the distance function to find the distance between two sets of lat/longs. The data sets are 700,700 and 309 points long. It has been running for over 10 hours. Is this normal for it to take this long? Is there a quicker way to do this?<br>
<br>
> OK, going on 24+ hours now. Again, is this normal? Is there a faster way?<br>
<br>
Travis, you are placing a heavy burden on your computer's computational capacity. Every distance between a 700,700element set and a 309element set amounts to a total of 216,516,300 distances altogether, and each one of these presumably involves a great circle arc distance between pairs of longitude and latitude quantities. These are spherical trigonometric computations rather than the much simpler Euclidean formulas. If you are using an ellipsoidal earth, these computations would be even more timeconsuming.<br>
<br>
I would suggest you time your computer for computing, say, all the distances between two 100element sets and then multiply that by a factor of twentyone thousand or thereabouts to get the timing you might reasonably expect for your problem. If the 100 by a 100 case takes only five seconds, you could expect to have six hours more to go beyond your present 24 hour wait.<br>
<br>
As for there being a faster method, some of the trigonometric calculations, such as determining the sine and cosine of the longitude and latitude quantities could be precomputed just once for each set's elements and this would save some time as compared with starting each distance computation from scratch. However, I would expect that the people at Mathworks have already figured that out for themselves and would be taking such a shortcut, provided it is being done in a single call on the appropriate distance function.<br>
<br>
There is of course another (unlikely?) possibility. Your computer may not possess enough RAM memory to store the necessary large number of distances and it is doing a switch to virtual disk memory. That would certainly slow things down. Mathworks people can tell you whether such a thing might be happening. I always thought that Matlab would complain if it didn't possess a sufficient amount of contiguous RAM memory to store an allocated array.<br>
<br>
Roger Stafford

Sun, 30 Nov 2008 03:20:03 +0000
Re: taking a long time
http://www.mathworks.com/matlabcentral/newsreader/view_thread/240113#613940
Travis
"Roger Stafford" <ellieandrogerxyzzy@mindspring.com.invalid> wrote in message <ggss3i$9vo$1@fred.mathworks.com>...<br>
> "Travis" <sinusoid2@hotmail.com> wrote in message <ggrqvm$k5l$1@fred.mathworks.com>...<br>
> > I am using the distance function to find the distance between two sets of lat/longs. The data sets are 700,700 and 309 points long. It has been running for over 10 hours. Is this normal for it to take this long? Is there a quicker way to do this?<br>
> <br>
> > OK, going on 24+ hours now. Again, is this normal? Is there a faster way?<br>
> <br>
> Travis, you are placing a heavy burden on your computer's computational capacity. Every distance between a 700,700element set and a 309element set amounts to a total of 216,516,300 distances altogether, and each one of these presumably involves a great circle arc distance between pairs of longitude and latitude quantities. These are spherical trigonometric computations rather than the much simpler Euclidean formulas. If you are using an ellipsoidal earth, these computations would be even more timeconsuming.<br>
> <br>
> I would suggest you time your computer for computing, say, all the distances between two 100element sets and then multiply that by a factor of twentyone thousand or thereabouts to get the timing you might reasonably expect for your problem. If the 100 by a 100 case takes only five seconds, you could expect to have six hours more to go beyond your present 24 hour wait.<br>
> <br>
> As for there being a faster method, some of the trigonometric calculations, such as determining the sine and cosine of the longitude and latitude quantities could be precomputed just once for each set's elements and this would save some time as compared with starting each distance computation from scratch. However, I would expect that the people at Mathworks have already figured that out for themselves and would be taking such a shortcut, provided it is being done in a single call on the appropriate distance function.<br>
> <br>
> There is of course another (unlikely?) possibility. Your computer may not possess enough RAM memory to store the necessary large number of distances and it is doing a switch to virtual disk memory. That would certainly slow things down. Mathworks people can tell you whether such a thing might be happening. I always thought that Matlab would complain if it didn't possess a sufficient amount of contiguous RAM memory to store an allocated array.<br>
> <br>
> Roger Stafford<br>
<br>
Well, I knew it was a huge amount of number, but I was hoping with an AMD 6400 OC'd and 8 gb of good RAM it might not take so long. I think on my next run I will cut it down a little. It doesn't seem to be running out of memory, I have seen it use as much as 1.5gb at once though.