Thread Subject: Correction of Perspective Distortion

I have to analyze images that are distorted because the
optical axis is not perpendicular to the plane being
imaged. Therefore, lines that are parallel in the real
world are not parallel in the image and rectangular objects
appear as parallelograms. How can I correct for this?

Is the any Matlab function that calculates the images that
would have been taken perpendicular to the plane to be
imaged?

"OkinawaDolphin " <OkinawaDolphin@Hotmail.com> wrote in message
<fnpft6$lod$1@fred.mathworks.com>...
> I have to analyze images that are distorted because the
> optical axis is not perpendicular to the plane being
> imaged. Therefore, lines that are parallel in the real
> world are not parallel in the image and rectangular objects
> appear as parallelograms. How can I correct for this?
>
> Is the any Matlab function that calculates the images that
> would have been taken perpendicular to the plane to be
> imaged?
-------------
  There are two kinds of image "distortion" that you might have in mind here.
You could be thinking of a draftsman-like image as viewed from an infinite
distance above a plane. Then a parallel projection onto a tilted plane would
be a simple stretching along the direction orthogonal to the intersection of
the two planes. Rectangles would change to parallelograms and parallel lines
would always remain parallel. This is not usually what is referred to as a
"perspective distortion".

  Or you might be talking about a projection from a fixed perspective point
above one plane onto a tilted plane. In that case parallel lines in the one
plane do not, in general, translate to parallel lines in the other, and rectangles
become quadrilaterals which are not, in general, parallelograms. (Think of
parallel train rails and railroad ties seeming to converge to an infinitely
distant point.)

  Which do you have in mind? If you have this second circumstance in mind,
there are five different parameters that enter into the situation. First, there is
the position in the first plane at the base of an orthogonal line (optical axis)
from the given perspective point, which takes two parameters. Next, is the
distance above the plane to that perspective point, a third parameter. The
fourth and fifth parameters are the direction and amount of tilt to the second
plane relative to the first one.

  It isn't clear to me just how you would go about determining all these
necessary parameters before carrying out the kind of correction you are
seeking. It is quite possible to give a transformation which could make the
necessary correction that you ask for, once these five parameters are known.
As I recall from some work I did a very long time ago, the equations involved
are simple rational functions. However, (perhaps out of laziness,) I am
reluctant to go to the trouble of again working it all out without having some
kind of assurance that you have devised a method for finding the necessary
five parameters.

Roger Stafford

> There are two kinds of image "distortion" that you
might have in mind here.
> You could be thinking of a draftsman-like image as viewed
from an infinite
> distance above a plane. Then a parallel projection onto
a tilted plane would
> be a simple stretching along the direction orthogonal to
the intersection of
> the two planes. Rectangles would change to
parallelograms and parallel lines
> would always remain parallel. This is not usually what
is referred to as a
> "perspective distortion".
>
> Or you might be talking about a projection from a fixed
perspective point
> above one plane onto a tilted plane. In that case
parallel lines in the one
> plane do not, in general, translate to parallel lines in
the other, and rectangles
> become quadrilaterals which are not, in general,
parallelograms. (Think of
> parallel train rails and railroad ties seeming to
converge to an infinitely
> distant point.)

It seems that it is the second kind of distortion. In
Matlab there are the functions such as maketform and
cp2tform for defining spatial transformations. Both affine
and projective transforms can be performed using these
functions. Parallel lines seem to converge and rectangles
are changed to rectangles or quadrilaterals. Therefore I
think that a projective transformation is suitable.

If I had a photograph taken above the imaged area, I could
use it as a reference image. However, such an image does
not exist. So there are two possibilities:

1. Creating a simplified artificial reference image.

2. Arbitrarily define mappings between points on converging
lines in an image to points on a rectangle.

Can you recommend a method of chosing or defining pairs of
points?

"OkinawaDolphin " <OkinawaDolphin@Hotmail.com> wrote in message
<fnrtqt$sed$1@fred.mathworks.com>...
> It seems that it is the second kind of distortion. In
> Matlab there are the functions such as maketform and
> cp2tform for defining spatial transformations. Both affine
> and projective transforms can be performed using these
> functions. Parallel lines seem to converge and rectangles
> are changed to rectangles or quadrilaterals. Therefore I
> think that a projective transformation is suitable.
>
> If I had a photograph taken above the imaged area, I could
> use it as a reference image. However, such an image does
> not exist. So there are two possibilities:
>
> 1. Creating a simplified artificial reference image.
>
> 2. Arbitrarily define mappings between points on converging
> lines in an image to points on a rectangle.
>
> Can you recommend a method of chosing or defining pairs of
> points?
----------
  It looks to me as though you have posed for yourself a challenging problem,
Okinawa Dolphin.

  Plane projective transformations are defined by equations of the rational
form

 u = (A*x+B*y+C)/(G*x+H*y+I)
 v = (D*x+E*y+F)/(G*x+H*y+I)

where (u,v) are the transformed coordinates from (x,y) coordinates and A, B,
C, D, E, F, G, H, I are nine arbitrary parameters. Because of the division
involved above, changing all nine parameters by the same scale factor doesn't
alter the transformation, so there are really only eight degrees of freedom for
such transformations. This means that, in general, if you know four different
pairs of corresponding (u,v) and (x,y) points, representing eight equations,
the transformation for all points will then be uniquely determined.
Accordingly, the 'cp2tform' and 'maketform' functions in the Images Toolbox
you mentioned are indeed able to set up such transformations from (at least)
four pairs of corresponding points. (Happily, these functions get me off the
hook for having to go through the necessary derivation from scratch.)

  However, faced with just a transformed image and no particular knowledge
of the original image from which it was derived, leaves you with a lot of
variation to investigate in trying to correct back to the original. It may not be
quite as hopeless as it first seems, though. For example, suppose you know
that four certain points in your image correspond to some four points in the
original image with known relative positions, such as the four corners of what
was formerly a square, the four corners of some quadrilateral of known
shape, or perhaps four points in a figure of known original shape. You just
don't know the location, orientation, or size of that original shape.

  It is my understanding that if you use those first four points paired with any
four points having the known original shape as the arguments of 'cp2tform'
or 'maketform', they will give you a tranformation that is correct for the entire
image except of course for its orientation, location, and size in the original
form. All that had been squares would be restored as squares, circles as
circles, letter 'Q's as letter 'Q's. The whole restored image would look like the
original, except that you might have to stand on your head or use a
magnifying glass to see it properly. The trick, therefore, is to be able to
recognize what the original shape was of some one reasonably large object in
your distorted image and choose four corresponding point pairs in it to hand
to 'cp2tform' or 'maketform'.

  In other words, the task of seeking a valid original image from a distorted
one, involves only four degrees of freedom, rather than eight, because the
other four degrees are involved in all the possible orientations, positions, and
sizes that can occur in the original image. Among those remaining four
degrees of freedom, there should be one and only one combination that
would look right.

  Is any of this of use to you?

Roger Stafford

Roger Stafford,

choosing four points that correspond to the corners of a
rectangle in the original image (which is not available)
might indeed solve the problem. In the images I have there
are lines that can be assumed to be parallel and small
objects that can be taken as corners of rectangles. There
are also medium sized objects that are approximately
rectangular in the real world.

I wonder how I should choose a quadrilateral. What is large
enough? Should the quadrilateral in the image cover more
than half the image area? Should it cover 2/3 or 3/4? Would
1/10 be enough? It seems that this is the challenge.