Find corresponding positions in stereo rectified images on 3D pointCloud

In a 1000 X 1000 image (rectified stereo pair), I_left , I have features detected at positions:

P_left =   124.914794921875  155.905288696289
           125.438186645508  206.946701049805
           126.040199279785  258.120697021484
           126.659095764160  309.096038818359
           127.287139892578  360.007171630859.

I_right has the corresponding features detected at positions:

P_right = 73.1327972412109  155.756408691406
          73.8827133178711  206.970779418945
          74.3082046508789  258.157165527344
          75.1555709838867  309.194702148438
          76.6253433227539  360.086303710938

As you can see, the y positions of corresponding points are almost similar (slight difference may be due to calibration, but doesn't matter, I can take the mean and make my y values same, if necessary.)

After Rectification, I find disparity and reconstructScene , to get a 3D pointCloud.

From the 3D pointCloud, I want to find the 3D values of the 5 feature points positions I had detected in I_left and I_right.

How should I do this?

I am not able to map these positions detected in rectified images onto the point cloud.

Answer 1

[Dima Lisin]

Hi Sanya,

reconstructScene returns pointCloud, which is an M-by-N-by-3 array. pointCloud(i, j, :) gives you x, y, and z coordinates corresponding to pixel i,j in the rectified image from camera 1.

[Meghana Dinesh]

Hi Dima,

These positions detected in my example are not integers. They are at a subpixel level, hence this doubt.

I want to avoid rounding off to nearest integer.

[Dima Lisin]

Hi Sanya,

currently the reconstructScene function only gives you the 3D coordinates corresponding to the integer pixel grid. If you want 3D coordinates for specific non-integer points, then you can undistort each image, do the feature matching, and then use the triangulate function. Note, that in this case you would not use rectifyStereoImages.

[Meghana Dinesh]

Will this affect my accuracy? I have tried feature matching......but it does not consider my stereoParameters anywhere. Hence, I thought it would be less accurate.

Also, unlike reconstructScene (where I obtain a point cloud), triangulation is only for a single point at a time. Am I right?

[Dima Lisin]

triangulate takes two sets of matching points, not just a single pair. It also returns the reprojection errors for the resulting 3D points, which lets you eliminate the bad matches.