It's tough to tell precisely what is going on here. My first thought is that perhaps you have two copies of your tests in two separate locations that are both on the path. Try doing the following to see whether there are two packages of the same name showing up on the path:
If this is a scalar struct then there onlys exists one test package or folder. If it is a struct array with more than one element then it should show you where it is picking up tests from.
What does the suite look like? Does it look expected or are the tests showing up twice in the suite as well?
Also, another thing to look at is whether there is any conflict with the runtests function in MATLAB that is included in the framework (13b and beyond). What happens when you change your function to be named something else, or alternatively, if you just use the included runtests function by typing:
>> result = runtests('tests')
Any information you can provide would be helpful, this certainly doesn't sound like behavior that should be happening.
Does that help?