Video applications present common but difficult challenges that require flexible analysis and processing functionality. Using MATLAB® and Simulink® products, you can develop solutions to common video processing challenges such as video stabilization, video mosaicking, target detection, and tracking.
Object tracking is an essential part of many applications including pedestrian avoidance, security and surveillance, and augmented reality. This examples shows motion-based tracking of moving people in a video from a stationary camera.
Object Detection and Counting
Video processing can be used to detect and count objects that move in video sequences. In this case study, scientists in Australia are using video footage to estimate the wildlife population of waterbirds.
MATLAB® provides tools and algorithms that let you view, analyze, read, and write videos. Video processing can be useful in applications like:
- Object recognition with deep learning
- Motion estimation such as optical flow
- Face detection and tracking
Video Processing in 4 Easy Steps
Video processing in MATLAB involves the following steps:
- Reading the video
- Displaying the video
- Processing the video
- Writing the video
Step 1. Reading the Video
You can read video from files or directly from cameras.
A single MATLAB command lets you read in videos from a file:
>> vid = VideoReader('filename.avi')
MATLAB supports webcams for video processing, while Image Acquisition Toolbox™ enables live acquisition from many industrial and scientific cameras.
MATLAB lets you read video files using a variety of codecs including OS-specific codecs for Microsoft® Windows®, Mac, and Linux®.
Step 2. Displaying the Video
There are two methods for displaying video in MATLAB:
deployableVideoPlayer: Efficiently view a series of video frames
implay: Launch the Video Viewer app for viewing videos
Step 3. Processing the Video
A video is a sequence of individual video frames, or images. This means an algorithm designed to perform edge detection on an image can be quickly converted to perform edge detection on a video.
Read single image
|Read image frame from video|
Video processing can be very simple, as in the example using edge detection, or significantly more complex, such as tracking algorithms that must account for an object’s location in previous frames.
For more information on advanced video processing, see examples for:
Step 4. Writing the Video
After processing, you can write each frame of a video back to a file. You can create a video file with the function:
>> vid_w = VideoWriter('newfile.avi'); >> open(vid_w)
vid_w can accumulate new frames to create a video.
A Complete MATLAB Example
Putting all the components together, let’s run through a complete example to show the steps of reading, displaying, processing, and writing video:
%% Read and process a video into MATLAB % Setup: create Video Reader and Writer videoFileReader = VideoReader('tilted_face.avi'); myVideo = VideoWriter('myFile.avi'); % Setup: create deployable video player and face detector depVideoPlayer = vision.DeployableVideoPlayer; faceDetector = vision.CascadeObjectDetector(); open(myVideo); %% Detect faces in each frame while hasFrame(videoFileReader) % read video frame videoFrame = readFrame(videoFileReader); % process frame bbox = faceDetector(videoFrame); videoFrame = insertShape(videoFrame, 'Rectangle', bbox); % Display video frame to screen depVideoPlayer(videoFrame); % Write frame to final video file writeVideo(myVideo, videoFrame); pause(1/videoFileReader.FrameRate); end close(myVideo)
You can download this code on MATLAB Central.
Video Processing Algorithms for Computer Vision
MATLAB algorithms that use temporal correlation for video processing are based on the concept of “state,” the idea that the algorithm is working on a current video frame but also uses previous frames to determine its output. This is crucial for object tracking algorithms, which rely on prior information to inform future action. A common example of tracking is the KLT algorithm, which tracks individual points in an object to keep track of an object’s location.
Developers of video processing algorithms can also use the vision-specific algorithms in Computer Vision System Toolbox™. The algorithms let you read and view high-resolution videos in a rapid and memory-efficient way. The toolbox also includes algorithms for 3D point cloud processing, stereo vision, object detection, tracking and recognition, and other applications.