## What is the scientific reason for removing high reprojection error images from the stereo calibration app?

### Marco Leonardi (view profile)

on 13 Jan 2017
Latest activity Edited by hani mohammed

### hani mohammed (view profile)

on 13 Aug 2018
In the stere camera calibration app page (https://www.mathworks.com/help/vision/ug/stereo-camera-calibrator-app.html) there is a statemente that says:
Consider removing image pairs if the images:
• Have a high mean reprojection error.
• Are blurry.
• Contain a checkerboard at an angle greater than 45 degrees relative to the camera plane.
• Contain incorrectly detected checkerboard points.
What is the scientific reason to do so? I mean if the image is sharp, corners are identified correctly, the checkerboard is not at an angle greater than 45 degrees and we have an high reprojection error, why we should remove this image/couple of images anyway?

### hani mohammed (view profile)

on 13 Aug 2018
Edited by hani mohammed

### hani mohammed (view profile)

on 13 Aug 2018

This is the way, the reprojection error is computed:
-If you have a pair of images (No Calibration), then you have the camera matrix already. In such case, you use the triangulation using both cameras to find the point X(x,y,z) in space corresponding to the points x and x' in the left and right images respectively. Then, you use each camera P and P' to reproject the world point X back to the image to be x_hat and x'_hat respectively. Usually if the matches are correct ( and the cameras are correct as well), then the reprojection error ( which is the difference between x and x_hat) should be small, if it is large, it means that either the camera matrix is incorrect or the matches are incorrect.
- If you are calibrating the camera, then you have the coordinates of the image points (x) and the coordinates of the world points X, and you need to find the optimal camera P. Then you use the camera P as an operator on X to find the value x_hat=PX ( you project the world point into the image plane using the camera matrix). Again, the difference between (x and x_hat ) is the reprojection error. If it is small, it means the camera matrix ( that is estimated) is correct. If it is large it means that the camera matrix is wrong.
Hani