Faster or smarter way to sort and intersect large amount of data??

Dear Sir/Madam,

I was using "intersect" in my Matlab code to do the sorting where I want the following:

[ch] = sort(s, 'ascend');

[same, a] = intersect(s, ch);

For example:

input: s =[55 21 78 7]

output: ch = [7 21 55 78] a = [4 2 4 3]

I need to access ‘a’ where ‘a’ shows the original index prior to sorting so I can use it for further processing.

This method works exactly as what I want, but I guess it is taking a lot of time to do the sorting and intersect etc especially when the size of s approaching 100 or higher, are there other faster or smarter ways to do so?

Thank you very much.

[José-Luis]

What's a lot of time? Remember that the algorithmic complexity of sort is O(n log n) so there are limits to how fast you can make it, and Matlab is already pretty decent for this function.

[want2know]

Thanks for your input, Jose-Luis... when "s" is approaching 100, it took 4 hours to run a "an-hour simulation"

Answer 1

[Jos (10584)]

intersect is pretty fast, so I would not worry too much ...

a = rand(100000,1) ;
b = rand(100000,1) ;
tic ; [c,i] = intersect(a,b) ; toc ;
% Elapsed time is 0.029850 seconds.

[want2know]

Thanks a lot Jos, do you think if "sort" is worth replaced by other sorting algorithm such as "bubble sort" etc? I tried your method to check if "sort" is the culprit of having a real slow simulation

a = rand(100000,1) ;

tic ; [c] = sort(a,'ascend') ; toc ; Elapsed time is 0.027178 seconds.

[Jos (10584)]

You have to decide if this is really worth optimising. Perhaps there are other bottlenecks in your code. Did you run the profiler?

[want2know]

Thanks Jos, sorry I am new to Matlab, profiler? do you mean "profile on / profile viewer"? so that I can have a look about it

[Jos (10584)]

Yep. The profiler is a very useful tool if you want to know where improvements might be found.

http://www.mathworks.nl/help/matlab/ref/profile.html