Clustering a set of Motion Vectors into Optimal Clusters based on position, direction?

Wondering if anyone can help me on this: I have a set of motion vectors representing people, I want to cluster these vectors into the different objects/people based on the vectors' proximity to one another as well as their direction and possibly past locations of clusters which I can also give as an input?

Anyone have any ideas on how to do this? I've been looking at agglomerative clustering and then evaluating the silhouette values and choosing clusters accordingly but perhaps there are better ways?

[jiapeng he]

I have similar questions with yours, have you worked it out?