Documentation |
Package: vision
Track points in video using Kanade-Lucas-Tomasi (KLT) algorithm
The point tracker object tracks a set of points using the Kanade-Lucas-Tomasi (KLT), feature-tracking algorithm. You can use the point tracker for video stabilization, camera motion estimation, and object tracking. It works particularly well for tracking objects that do not change shape and for those that exhibit visual texture. The point tracker is often used for short-term tracking as part of a larger tracking framework.
As the point tracker algorithm progresses over time, points can be lost due to lighting variation, out of plane rotation, or articulated motion. To track an object over a long period of time, you may need to reacquire points periodically.
pointTracker = vision.PointTracker returns a System object, pointTracker, that tracks a set of points in a video.
pointTracker = vision.PointTracker(Name,Value) configures the tracker object properties, specified as one or more name-value pair arguments. Unspecified properties have default values.
Code Generation Support |
---|
Supports MATLAB^{®} Function block: Yes |
System Objects in MATLAB Code Generation. |
Code Generation Support, Usage Notes, and Limitations. |
To initialize the tracking process, you must use the initialize method to specify the initial locations of the points and the initial video frame.
initialize(pointTracker,points,I) initializes points to track and sets the initial video frame. The initial locations POINTS, must be an M-by-2 array of [x y] coordinates. The initial video frame, I, must be a 2-D grayscale or RGB image and must be the same size and data type as the video frames passed to the step method.
The detectFASTFeatures, detectSURFFeatures, detectHarrisFeatures, and detectMinEigenFeatures functions are few of the many ways to obtain the initial points for tracking.
Define and set up your point tracker object using the constructor.
Call the step method with the input image, I, and the point tracker object, pointTracker. See the following syntax for using the step method.
After initializing the tracking process, use the step method to track the points in subsequent video frames. You can also reset the points at any time by using the setPoints method.
[points,point_validity] = step(pointTracker,I) tracks the points in the input frame, I using the point tracker object, pointTracker. The output points contain an M-by-2 array of [x y] coordinates that correspond to the new locations of the points in the input frame, I. The output, point_validity provides an M-by-1 logical array, indicating whether or not each point has been reliably tracked.
A point can be invalid for several reasons. The point can become invalid if it falls outside of the image. Also, it can become invalid if the spatial gradient matrix computed in its neighborhood is singular. If the bidirectional error is greater than the MaxBidirectionalError threshold, this condition can also make the point invalid.
[points,point_validity,scores] = step(pointTracker,I) additionally returns the confidence score for each point. The M-by-1 output array, scores, contains values between 0 and 1. These values correspond to the degree of similarity between the neighborhood around the previous location and new location of each point. These values are computed as a function of the sum of squared differences between the previous and new neighborhoods. The greatest tracking confidence corresponds to a perfect match score of 1.
setPoints(pointTracker, points) sets the points for tracking. The method sets the M-by-2 points array of [x y] coordinates with the points to track. You can use this method if the points need to be redetected because too many of them have been lost during tracking.
setPoints(pointTracker,points,point_validity) additionally lets you mark points as either valid or invalid. The input logical vector point_validity of length M, contains the true or false value corresponding to the validity of the point to be tracked. The length M corresponds to the number of points. A false value indicates an invalid point that should not be tracked. For example, you can use this method with the estimateGeometricTransform function to determine the transformation between the point locations in the previous and current frames. You can mark the outliers as invalid.
initialize | Initialize video frame and points to track |
setPoints | Set points to track |
step | Track points in video using Kanade-Lucas-Tomasi (KLT) algorithm |
Lucas, Bruce D. and Takeo Kanade. "An Iterative Image Registration Technique with an Application to Stereo Vision,"Proceedings of the 7th International Joint Conference on Artificial Intelligence, April, 1981, pp. 674–679.
Tomasi, Carlo and Takeo Kanade. Detection and Tracking of Point Features, Computer Science Department, Carnegie Mellon University, April, 1991.
Shi, Jianbo and Carlo Tomasi. "Good Features to Track," IEEE Conference on Computer Vision and Pattern Recognition, 1994, pp. 593–600.
Kalal, Zdenek, Krystian Mikolajczyk, and Jiri Matas. "Forward-Backward Error: Automatic Detection of Tracking Failures," Proceedings of the 20th International Conference on Pattern Recognition, 2010, pages 2756–2759, 2010.