Disparity map from stereo vision not accurate

I have a problem trying to do depht axtraction from a stereo vision system.

I'm using two cameras (iDS, no RGB but B&W images) with two different optics both with the same focal lenght (8 mm).

After the calibration done with the 'stereo camera calibrator' app from Matlab the result was an overall mean error of 0.32 pixels.

Then I took these photos of an object with both cameras:

and I tried to obtain the disparity map with this code:

left=imread('spinaleft.png');
right=imread('spinaright.png');
figure
imshowpair(left,right,"montage")
%% rectification
[leftRect,rightRect,reprojectionMatrix]=rectifyStereoImages(left,right,stereoParams2607);
figure
A=stereoAnaglyph(leftRect,rightRect);
imshow(A)
%% disparity
leftRect=im2gray(leftRect);
rightRect=im2gray(rightRect);
disparityRange = [0 432];
disparityMap = disparityBM(leftRect,rightRect,'DisparityRange',disparityRange,'UniquenessThreshold',10,'BlockSize',9,'ContrastThreshold',0.5);
figure
imshow(disparityMap,disparityRange)
title('Disparity Map')
colormap jet
colorbar

but unfortunately the disparity map is too different from the reality. This was the best one I've got:

I really don't understand what I'm doing wrong, and I have some doubts:

1) could it be better if I use RGB photos instead of B&W photos?

2) why there's a blue area in the left side of the disparity map?

3) why are the windows in red? It looks like the system sees them closer than they are.

Thanks for helping me!

[Giridharan Kumaravelu]

I don't think the results are poor because they are B&W photos. There might have been something gone wrong during the calibration process. Could you share the following for a deeper investigation?

1. The calibration images that you used to calibrate the camera.
2. The calibration results that you got (in a MAT file).
3. Link to spec sheet of the camera that you are using.

[Luca]

I created a link to a Matlab Drive where you can see the pictures I used to calibrate the stereo system

https://drive.matlab.com/sharing/1ff3b01a-be78-4d15-adb8-e35f9de6eb60

Here are the stereoParams and the estimationerror I got from the the calibration process. And also the datasheet of the camera.

To calibrate I used a checkerboard pattern with a square size of 23 mm.

Let me know if you need any other data.

Answer 1

[VINAYAK LUHA]

Hi Luca,

As per my understanding, the result is decent given the camera calibration accuracy and the use case scenario. However, the following suggestions may help to enhance the result quality.

1.Improvement in the camera calibrations

• In the images used for calibration the pattern should be at a distance roughly equal to the distance from the camera to the object of interest with auto-focus disabled.
• Ensure that the region of interest is fully visible from both cameras of the stereo camera pair.

2. Image post-processing

• Consider applying Morphological transformation such as dilation or smoothing using gaussian blurr to the disparityMap.

To address your questions

1. Yes, colour variations in RGB images can help in distinguishing objects or surfaces with similar intensities in grayscale (B&W) and can provide depth cues in certain scenarios.
2. The disparity of pixels which are less unique or less textured are unreliable and are marked as "nan" by the block matching algorithm which manifests as the blue area on the left of the disparityMap.
3. To improve the results, consider implementing the suggestions mentioned above, such as refining the camera calibration and exploring post-processing techniques.

Here are the articles you may find helpful for your references:

• Best practices to prepare camera and capture images for “stereo vision camera calibration app” https://in.mathworks.com/help/vision/ug/prepare-camera-and-capture-images.html
• MATLAB documentation of “disparitybm” function https://in.mathworks.com/help/vision/ref/disparitybm.html [KA6] [VL7]
• MATLAB documentation on Morphological filtering https://in.mathworks.com/help/images/morphological-filtering.html

I hope it helps!

[Luca]

Hi!

Thank you so much for the explanation, I will try to improve my results following your suggestions and then I’ll let you know if I’ll have some problems.

Thank you again, have a good day!