Error in Comparing 2 images using correlation

Hi, I have 2 images, visible and a thermal image. I cropped the face in the visible image, and i have to match this cropped face to the thermal image face. I used correlation function to match between the 2 images, but i get wrong results.

thermal=imread('C:\Users\USER\Desktop\memstech\database\thermal.jpg');
thermal=rgb2gray(thermal);
folder = fullfile('C:\Users\USER\Desktop\memstech\database');
baseFileName = 'visible.jpg';
% Get the full filename, with path prepended.
fullFileName = fullfile(folder, baseFileName);
rgbImage = imread(fullFileName);
rgbImage=rgb2gray(rgbImage);
[rows columns numberOfColorBands] = size(rgbImage);
% Display the original color image.
subplot(2, 2, 1);
imshow(rgbImage, []);
axis on;
title('Original Color Image', 'FontSize', fontSize);
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);
smallSubImage = imcrop(rgbImage, [203.510000000000,76.5100000000000,42.9800000000000,80.9800000000000]);
subplot(2, 2, 2);
imshow(smallSubImage, []);
axis on;
title('Template Image to Search For', 'FontSize', fontSize);
correlationOutput = normxcorr2(smallSubImage, thermal);
subplot(2, 2, 3);
imshow(correlationOutput, []);
title('Correlation Output', 'FontSize', fontSize);
[maxCorrValue, maxIndex] = max(abs(correlationOutput(:)));
[ypeak, xpeak] = ind2sub(size(correlationOutput),maxIndex(1));
corr_offset = [(xpeak-size(smallSubImage,2)) (ypeak-size(smallSubImage,1))];
subplot(2, 2, 4);
imshow(thermal);
hold on;
rectangle('position',[corr_offset(1) corr_offset(2) 100 100],...
          'edgecolor','g','linewidth',5);
title('Template Image Found in Original Image', 'FontSize', fontSize);

where did this algorithm went wrong? How to find the face patch in the thermal image. Kindly help me with your suggestions.

I am trying this method, because image registration failed to align 2 images with different field of view.