The Image Processing Toolbox™ provides functionality for applying geometric transformations to register images.
For control point registration, the fitgeotrans function can infer the parameters for the following types of transformations, listed in order of complexity.
'polynomial' (Order 2, 3, or 4)
The first four transformations, 'nonreflective similarity', 'affine', 'projective', and 'polynomial' are global transformations. In these transformations, a single mathematical expression applies to an entire image. The last two transformations, 'piecewise linear' and 'lwm' (local weighted mean), are local transformations. In these transformations, different mathematical expressions apply to different regions within an image. When exploring how different transformations affect the images you are working with, try the global transformations first. If these transformations are not satisfactory, try the local transformations: the piecewise linear transformation first, and then the local weighted mean transformation.
Your choice of transformation type affects the number of control point pairs you need to select. For example, a nonreflective similarity transformation requires at least two control point pairs. A polynomial transformation of order 4 requires 15 control point pairs. For more information about these transformation types, and the special syntaxes they require, see cpselect.
For information about the specific transformation types available for intensity-based automatic image registration, see the imregister function reference page.