Whats causing my length measurement to be so unprecise?

Hey folks,

I am working on an image processing algorithm that helps me calculate finger lengths. But the pictures I take lead to deviations up to 10mm, what makes the results unacceptable. I am looking for ideas, what’s wrong with my pictures.

The pictures I take are created in a photo-box where the distances between the camera, the hand and the light are always the same. I take pictures with the same camera (same configuration) and the same light every time. The camera is a Canon SX540 HS, so no scientific camera . The plane of the hand and the camera are parallel. The algorithm transforms the pictures I take into a binary matrix/picture and then calculates the most left pixel of each finger. When I have the pixels of the four fingertips, I calculate the horizontal distance to a line (78° from the horizontal) starting at the lowest point between thumb and index finger. This distance is the main subject of my research.

With the pictures I take, I get to small results at the bottom (small finger) and to large results at the top (index and middle). The ring finger is usually measured correct as it is the switching point in this trend. The checkboard pattern, wich I use for calculating the proportion of pixel and mm has 185 px/cm at the left side, 190 in the middle and goes down to 182px/cm on the right. In reality each box is exactly 10x10 mm.

Lens distortion. The trend of bottom to small, middle fine and top to large and the obvious not straight lines in the picture let me assume lens distortion is the problem. I use the camera calibration app and the tutorial given there to undistort my images. I used a larger correct checkbox pattern and the photo-box. I tried a bunch of pictures and configurations.

The results are only a shift of the deviations. The edge near the wrist should be a straight line, it doesn’t get undistorted, the distances I get are now too small everywhere and the largest deviation at the ring finger. The checkboard starts at 190px/cm on the left and goes down constantly to 186 on the right.

My idea now is distortion by perspective. Even though the plains of the hand and the camera are only 17cm away and parallel, it is the only idea I have left. I dont expect to have used the camera calibration app so wrong, that with correct use it would be able to undistort the images as much as i need it (the black reference at the right should be straight and not bended).

Due to the finish date of my work and my uncertainty if I am even on the right track, I am asking here for input, advise or specific help. If someone has an idea or experience what I might did wrong, or what obvious I didn’t see, please speak frankly. If you had a similar problem, how did you solve it? Did I miss a previous thread in the forum related to my question? If it is distortion by perspective, what matlab functions should I dive into first?

I am thankfull for everyone reading, even if you dont have help.

Greetings from Germany and stay healthy, Nils.

[dpb]

I'm no image processing guru, but first Q? I would have is whether the results are consistent from trial to trial with adequate precision between them (even if inaccurate)? IOW, is it truly reproducible?

If that is the case, then it should be possible to derive and apply an empirical distortion-removing transformation.

If it isn't, then there's no hope until you can resolve what are changes altho I presume from the discourse it is.

[Nils Schröder]

Thanks for your help 👍. I think about this as the remaining option if I dont find the problems origin.

[dpb]

But I think the answer to that question is critical to know, anyway...be curious to hear how it turns out.

There are some serious image-processing folks amongst the regulars here, highly likely one or more of them will see this after if not over weekend.

Answer 1

[Bjorn Gustavsson]

I would simply use the checker-board to get a mapping between image-coordinates and spatial coordinates. Simply put the checker-board on the plane where the hands will be, then register the intersections between the squares (this will need to be done with pixe/sub-pixel accuracy). This will give you two mappings, u(x,y) and v(x,y) where u and v are the horizontal and vertical image coordinates and x and y are the checkerboard coordinates. Since you want the inverse mapping you can use scatteredInterpolant to get that, something like this:

f_x = scatteredInterpolant(u(:),v(:),x_board(:),'natural');
f_y = scatteredInterpolant(u(:),v(:),y_board(:),'natural');

Then you use these functions to convert your pixel-coordinates to real-world coordinates. It will require a bit of work to extract the coordinates of the checker-board, but not too much.

(But have you checked your algebra/geometry/algorithm for calculating the finger-length? The problem you describe have the smell of a bug of some geometric sort...)

HTH

[Nils Schröder]

Thank you very much for this Idea. I will go after this. Though I have not a 100% fix camera position, I might need to adjust it a bit. It think about printing a (mostly) transparent grid to the hand plane, and then detect it by matlab to calculate the real-world coordinates and continue with your advise.

Yeah I checked that, by meassuring the amount of pixel for each finger manually in paint. The algorithm calculates the correct amount of pixel, but apparently the pixel/mm proportion is different in different areas of the picture.

[Bjorn Gustavsson]

For manual registration-tasks like this I've found that grabit is a very useful tool.