First of all, look over my image segmentation demo. http://www.mathworks.com/matlabcentral/fileexchange/?term=authorid%3A31862 Then look at how I filter out coins based on size. You need to do the same thing. If they overlap the size will be a lot bigger than if they don't overlap. So you just need to tweak the area value in my code and it should work for you.