Video and Image Processing Blockset 2.8

Disparity Estimation for Stereo Vision

Stereo vision refers to the ability to infer information about the 3-D structure of a scene using two images captured from different viewpoints. Using two stereo images, this demo shows how the Video and Image Processing Blockset™ can be used to compute the epipolar geometry [1][3] of a stereo system. Once it has this geometry, the demo estimates the disparity map (relative depth) between the stereo images.

The demo expects a reasonably configured stereo rig and will error out if it detects that the epipolar lines have slope greater than one. Furthermore, the demo can take a long time to run with larger images. This can be remedied by running the demo using the accelerator mode which compiles the demo into an executable.

Contents

Demo Model

The following figure shows the Disparity Estimation for Stereo Vision demo model.

Intrinsic Calibration Subsystem

Intrinsic calibration is the process of estimating the geometry of stereo cameras. The Intrinsic Calibration subsystem performs the following steps:

  1. Locates the features in each image using the Corner Detection
     block

  2. Identifies points that are common to both images using
     Sum of Squared Differences (SSD) algorithm [1][2]

  3. Computes the Fundamental Matrix [1][3] that describes the epipolar
     geometry of the stereo vision system. The demo computes
     this using the Eight-Point [4] algorithm. Additionally, it uses
     either the RANSAC algorithm [3] or the Least Median of Squares [5]
     algorithm to reduce the error in the estimation of the Fundamental
     Matrix. The error is computed using Sampson distance metric [3].

The Find Matching Features block is a masked subsystem. It requires that you specify the desired number of matching features for later use in computing the Fundamental Matrix of the stereo system. Looking under the mask of this subsystem reveals further implementation details.

The demo uses the Corner Detection block to find image features. If you would like to feed your own data into the demo you may wish to explore the various corner detection methods and other settings of the Corner Detection block.

The Point Feature Matching subsystem, shown above, uses Sum of Squared Differences (SSD) metric to determine which features in the stereo pair correspond to each other. It then returns an ordered list of matching feature coordinates. The ordering of matching features is necessary for the Least Median of Squares algorithm used to compute the Fundamental Matrix.

Disparity Estimation Subsystem

Disparity estimation is the process of computing the relative depth of objects within a scene. The Disparity Estimation subsystem performs the following steps:

  1. For each location in the left image, compute the corresponding
     epipolar line in the right image

  2. In the right image, search along the epipolar line to find the
     location of the pixel that matches the point in the left image
     using the SSD [1]

  3. Refine the location of the best match to subpixel accuracy by
     fitting a quadratic polynomial curve to the minimum SSD
     value and its two neighboring values. The location of the minimum
     value of the polynomial curve is the subpixel location

  4. Determine the disparity values by computing the Euclidean distance
     between each pair of corresponding points

  5. Assemble all the disparity values into a map, where pixel
     intensities describe the relative depth of points within a scene

Each block extracted by the Block Processing Subsystem is used to search along the epipolar line in the right image.

The matching takes place inside the Match Features Along Epipolar Line subsystem shown below.

Disparity Estimation for Stereo Vision Results

The figures below show the results of the Disparity Estimation for Stereo Vision demo. The Left Image Features and Right Image Features windows show the left and right stereo images, respectively. The demo marks the matched feature points on the images. The Disparity Map window shows the disparity (relative depth) computed by the demo. The higher the intensity value, the nearer the object is to the camera.

There is a utility model included with this demo that helps generate a stereo pair of images from a virtual reality scene using Simulink 3D Animation. These images can subsequently be fed into the demo.

Results with virtual reality generated input:

The utility model vipstereo_vrsource.mdl that accompanies this demo can be used to generate stereo pair of images from a virtual reality scene. This model includes a camera system that can be moved around the virtual reality world to capture desired stereo scenes. The scenes generated by this model can be written to image files using the IMWRITE command:

    imwrite(leftImage, 'vrscene_left.png', 'png');
    imwrite(rightImage, 'vrscene_right.png', 'png');

The demo model vipstereo.mdl can now be configured to read these files to generate a disparity map:

    open_system('vipstereo');
    set_param('vipstereo/Stereo Source/Left Image', 'ImageValue', ...
        'imread(''vrscene_left.png'')');
    set_param('vipstereo/Stereo Source/Right Image', 'ImageValue', ...
        'imread(''vrscene_right.png'')');

Results with input captured from a virtual reality scene:

Results with input captured from a stereo rig:

Supporting Files

Utility model to generate images from a virtual reality source

vipstereo_vrsource.mdl

References

[1] Trucco, E; Verri, A; Introductory Techniques for 3-D Computer Vision (1998) Prentice Hall

[2] Hannah M J; Computer Matching of Areas on Stereo Images, Ph.D Thesis, Stanford University (1974)

[3] Richard Hartley and Andrew Zisserman; Multiple View Geometry in computer vision (2003) Cambridge University Press

[4] Hartley R; In Defense of the Eight-Point Algorithm, IEEE® Transactions on Pattern Analysis and Machine Intelligence, v.19 n.6, pp 580-593, June 1997

[5] Rousseeuw P J; Least Median of Squares Regression, J. of Amer. Stat. Assoc., 79, pp. 871-880, 1984

[6] L. Irsara and A. Fusiello; Quasi-euclidean uncalibrated epipolar rectification Rapporto di Ricerca RR 43/2006, Dipartimento di Informatica - Universit? di Verona, 2006; http://profs.sci.univr.it/~fusiello/demo/rect