Thank you very much Brett to provide me a comprehensive information. Appreciate your help and looking forward to see more great jobs from you. If I face any issue, i will back to ask you.
Image Registration App is just a front end for the functionality of IMREGISTER.
IMREGISTER is actually a collection of algorithms (optimization + similarity metrics). Because of this, the correct citation would depend on what flavor of optimization and metric you're using.
Here are some applicable references:
Mutual Information based registration:
"Nonrigid multimodality image registration" D. Mattes, D. R. Haynor, H. Vesselle, T. Lewellen and W. Eubank Medical Imaging 2001: Image Processing, 2001, pp. 1609-1620.  "PET-CT Image Registration in the Chest Using Free-form Deformations" D. Mattes, D. R. Haynor, H. Vesselle, T. Lewellen and W. Eubank IEEE Transactions in Medical Imaging. Vol.22, No.1, January 2003. pp.120-128.  "Optimization of Mutual Information for MultiResolution Image Registration" P. Thevenaz and M. Unser IEEE Transactions in Image Processing, 9(12) December 2000.
One Plus One Evolutionary Optimizer:
"Parametric estimate of intensity inhomogeneities applied to MRI" Martin Styner, G. Gerig, Christian Brechbuehler, Gabor Szekely, IEEE TRANSACTIONS ON MEDICAL IMAGING; 19(3), pp. 153-165, 2000, (http://www.cs.unc.edu/~styner/docs/tmi00.pdf)
"Evaluation of 2D/3D bias correction with 1+1ES-optimization" Martin Styner, Prof. Dr. G. Gerig (IKT, BIWI, ETH Zuerich), TR-197 (http://www.cs.unc.edu/~styner/docs/StynerTR97.pdf)
Most of the literature seems to take the basic similarity metric/ optimization framework for granted without citing references. The references are usually for specific parts of the framework (numeric optimization, image similarity metrics, ways of pyramiding, etc.).
Hope that helps,