How to align the whole image according to a line ?

I have this code below which detects lines on my image but I can't align the whole image( scanned image) to make the lines straight, therefore align the text as well.

This is the image: http://tinypic.com/view.php?pic=106hgjs&s=6

The left is the input image and the right should the fixed one.

Any solutions??

Thank you

%# read and crop image
I = imread('C:\Users\Victoras\Desktop\attempt2.bmp');
%I = imread('http://i.stack.imgur.com/CJHaA.png');

I = I(:,1:end-3,:);     %# remove small white band on the side

%# egde detection
BW = edge(rgb2gray(I), 'canny');

%# hough transform
[H T R] = hough(BW);
P  = houghpeaks(H, 4, 'threshold',ceil(0.75*max(H(:))));
lines = houghlines(BW, T, R, P);

% shearing transforma
slopes = vertcat(lines.point2) - vertcat(lines.point1);
slopes = slopes(:,2) ./ slopes(:,1);
TFORM = maketform('affine', [1 -slopes(1) 0 ; 0 1 0 ; 0 0 1]);
II = imtransform(I, TFORM);

%# show edges
figure, imshow(BW)

%# show accumlation matrix and peaks
figure, imshow(imadjust(mat2gray(H)), [], 'XData',T, 'YData',R, 'InitialMagnification','fit')
xlabel('\theta (degrees)'), ylabel('\rho'), colormap(hot), colorbar
hold on, plot(T(P(:,2)), R(P(:,1)), 'gs', 'LineWidth',2), hold off
axis on, axis normal

%# show image with lines overlayed, and the aligned/rotated image
figure
subplot(121), imshow(I), hold on
for k = 1:length(lines)
    xy = [lines(k).point1; lines(k).point2];
    plot(xy(:,1), xy(:,2), 'g.-', 'LineWidth',2);
end, hold off
subplot(122), imshow(II)

Reformatting:

 %# read and crop image 
 I = imread('C:\Users\Victoras\Desktop\attempt2.bmp'); 
     %I = imread('http://i.stack.imgur.com/CJHaA.png');

I = I(:,1:end-3,:); %# remove small white band on the side

%# egde detection 
BW = edge(rgb2gray(I), 'canny');

%# hough transform 
[H T R] = hough(BW); 
P = houghpeaks(H, 4, 'threshold',ceil(0.75*max(H(:)))); 
lines = houghlines(BW, T, R, P);

% shearing transforma 
slopes = vertcat(lines.point2) - vertcat(lines.point1); 
slopes = slopes(:,2) ./ slopes(:,1); 
TFORM = maketform('affine', [1 -slopes(1) 0 ; 0 1 0 ; 0 0 1]); 
II = imtransform(I, TFORM);

%# show edges figure, imshow(BW)

%# show accumlation matrix and peaks figure, imshow(imadjust(mat2gray(H)), [], 'XData',T, 'YData',R, 'InitialMagnification','fit') 
xlabel('\theta (degrees)'), 
ylabel('\rho'), 
colormap(hot), 
colorbar hold on, 
plot(T(P(:,2)), R(P(:,1)), 'gs', 'LineWidth',2), 
hold off 
axis on, 
axis normal

%# show image with lines overlayed, and the aligned/rotated image   figure subplot(121), 
imshow(I), 
hold on 
for k = 1:length(lines) 
 xy = [lines(k).point1; lines(k).point2]; 
 plot(xy(:,1), xy(:,2), 'g.-', 'LineWidth',2); 
end, 
hold off 
subplot(122), 
imshow(II)

Answer 1

Not sure why you would apply a shear. Shouldn't it just be a rotation+translation?

Using Hough Transform to compute line angles so that I can perform a Shearing Transform to align the image. Isn't that the idea? Any ideas?

Thanks

I already told you my idea - to apply a rotation/translation instead of a shear.

However, there could be something I don't understand about the problem. That's why I also asked you what made you decide on a shear.

Answer 2

Once you determine the angle of that long line in the center, simply call imrotate(). Then crop if desired.

I will try this but say I have the value of the line angle stores as I, how can use imrotate??

Thanks

OutputImage = imrotate(InputImage, I);

Note, imrotate alone will only be sufficient if a relative translation between the two images doesn't matter.

I don't think imrotate will fix the problem, because the scanned image might not only change rotation but maybe a slight position to the right or left. Firstly I scan the image(answer sheet) as best as I can and then scan multiple images in order to compare. But some scanned images like in the example here have this problem. I need somehow when I imfuse the two images to be in the same position (maybe according to the line in the middle)

Then see my Answer. You should be using imtransform, but instead of performing a shear, you should perform a rototranslation. You can surely compute the required rotation based on the angle between the 2 center lines. Since you have the endpoints of the line, in both images, you can also compute the required translation to make the endpoints(post-rotation) coincide.

Matt (regarding your first comment) - right, and if a relative translation is not desired, then that's taken into account by the crop. You position the cropping rectangle to eliminate any lateral shift. You could not even worry about translating if you used a smarter but more complicated algorithm that looks at the lines and array of circles to set up a coordinate system. The other way is to translate the image after rotation so that the circles land in predefined locations. Then you can use a predefined mask to check the "filled" or "empty" status of the circles. Like Matt said, I think you could do the rotation and translation all in one shot if you just specify the endpoints of the long central line and the endpoints of where you want it to be in the desired final image.

Things are getting interesting here, thanks for all the help and advice!!! In my mind right now the best way to do it is to compare the orientation of the line between the two images to find the rotation of the scanned image and using that difference apply a rotation. After this, I will use translation to find the the end point of the lines and again compare the (x,y) coordinates.

Am entirely new at this and if the above I wrote are correct could please provide me with the necessary functions from start to bottom in order to find a way do it on my own?

Many thanks!!!

Here's a start:

% Read in a demo image.
folder = 'C:\Users\Vic\Documents\Temporary';
baseFileName = 'testsheet.png';
% Get the full filename, with path prepended.
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~exist(fullFileName, 'file')
	% File doesn't exist -- didn't find it there.  Check the search path for it.
	fullFileName = baseFileName; % No path this time.
	if ~exist(fullFileName, 'file')
		% Still didn't find it.  Alert user.
		errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
		uiwait(warndlg(errorMessage));
		return;
	end
end
grayImage = imread(fullFileName);
% Get the dimensions of the image.  
% numberOfColorBands should be = 1.
[rows columns numberOfColorBands] = size(grayImage)
if numberOfColorBands > 1
	% Convert to grayscale.
	grayImage = rgb2gray(grayImage);
end
% Display the original gray scale image.
subplot(1, 2, 1);
imshow(grayImage, []);
title('Original Grayscale Image', 'FontSize', fontSize);
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);
% Give a name to the title bar.
set(gcf,'name','Demo by ImageAnalyst','numbertitle','off') 

uiwait(msgbox('Locate the two endpoints of the line.'));
[actualx actualy] = ginput(2)

% Determine where the end points should be in the final, aligned image.
desiredx = mean(actualx);
desiredy = zeros(2,1);
desiredy(1) = actualy(1);
lineLength = hypot(actualx(1)-actualx(2), actualy(1)-actualy(2))
desiredy(2) = desiredy(1) + lineLength;

Then call maketform(), followed by tformfwd().