Asked by Carine
on 6 Mar 2013

Hello,

I have got this image https://www.dropbox.com/s/5hmy6tb3ym42tic/rect.jpg?m which is rectangle-like but not exactly.

I would like to find rectangular contours and I have tried to use edge and hough but it doesn't find continuous line. I have also tried to erode and dilate but it can't erase the difference of one pixel on the horizontal upper most line for example so it still doesn't find the continuous line. The second part of the problem would be to straighten it.

Another idea I got was to find the corners to link them with straight lines but it doesn't work either.

I am sure it shouldn't be complicated but I can't make it. Any ideas will be welcome!

Cheers

Answer by Teja Muppirala
on 7 Mar 2013

Accepted Answer

I've done this before: http://www.mathworks.com/videos/solving-a-sudoku-puzzle-using-a-webcam-68773.html

This is the strategy that I used and it seemed to work on your image as well. Basically you search for the corners, and then use IMTRANSFORM to transorm the corners into a rectangle. (I'll assume your image is called I).

%%1. Get rid of the white border

I2 = imclearborder(im2bw(I));

%%2. Find each of the four corners

[y,x] = find(I2);

[~,loc] = min(y+x);

C = [x(loc),y(loc)];

[~,loc] = min(y-x);

C(2,:) = [x(loc),y(loc)];

[~,loc] = max(y+x);

C(3,:) = [x(loc),y(loc)];

[~,loc] = max(y-x);

C(4,:) = [x(loc),y(loc)];

%%3. Plot the corners

imshow(I); hold all

plot(C([1:4 1],1),C([1:4 1],2),'r','linewidth',3);

%%4. Find the locations of the new corners

L = mean(C([1 4],1));

R = mean(C([2 3],1));

U = mean(C([1 2],2));

D = mean(C([3 4],2));

C2 = [L U; R U; R D; L D];

%%5. Do the image transform

figure

T = cp2tform(C ,C2,'projective');

IT = imtransform(im2bw(I),T); %IM2BW is not necessary

imshow(IT); hold all

Sign in to comment.

Answer by Image Analyst
on 7 Mar 2013

Carine:

You can straighten your image with imtransform(), like in this demo I wrote for you using your uploaded image:

clc; % Clear the command window.

close all; % Close all figures (except those of imtool.)

imtool close all; % Close all imtool figures if you have the Image Processing Toolbox.

clear; % Erase all existing variables. Or clearvars if you want.

workspace; % Make sure the workspace panel is showing.

format long g;

format compact;

fontSize = 20;

% Change the current folder to the folder of this m-file.

if(~isdeployed)

cd(fileparts(which(mfilename)));

end

% Check that user has the Image Processing Toolbox installed.

hasIPT = license('test', 'image_toolbox');

if ~hasIPT

% User does not have the toolbox installed.

message = sprintf('Sorry, but you do not seem to have the Image Processing Toolbox.\nDo you want to try to continue anyway?');

reply = questdlg(message, 'Toolbox missing', 'Yes', 'No', 'Yes');

if strcmpi(reply, 'No')

% User said No, so exit.

return;

end

end

% Read in a standard MATLAB gray scale demo image.

folder = 'C:\Users\Carine\Documents';

baseFileName = 'rect.jpg';

% Get the full filename, with path prepended.

fullFileName = fullfile(folder, baseFileName);

% Check if file exists.

if ~exist(fullFileName, 'file')

% File doesn't exist -- didn't find it there. Check the search path for it.

fullFileName = baseFileName; % No path this time.

if ~exist(fullFileName, 'file')

% Still didn't find it. Alert user.

errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);

uiwait(warndlg(errorMessage));

return;

end

end

grayImage = imread(fullFileName);

% Get the dimensions of the image.

% numberOfColorBands should be = 1.

[rows columns numberOfColorBands] = size(grayImage);

if numberOfColorBands > 1

grayImage = grayImage(:,:,2); % Take green channel

end

% Binarize the image

binaryImage1 = grayImage > 128;

% Get rid of the white border

binaryImage1 = imclearborder(binaryImage1);

% Display the original gray scale image.

subplot(2, 2, 1);

imshow(binaryImage1, []);

title('Binary Image', 'FontSize', fontSize);

% Enlarge figure to full screen.

set(gcf, 'units','normalized','outerposition',[0 0 1 1]);

% Give a name to the title bar.

set(gcf,'name','Demo by ImageAnalyst','numbertitle','off')

% Fill the object

binaryImage2 = imfill(binaryImage1, 'holes');

% Find the boundaries

boundaries = bwboundaries(binaryImage2);

xCoords = boundaries{1}(:, 2);

yCoords = boundaries{1}(:, 1);

% Find out which index is closest to the upper left corner

distances = sqrt((xCoords - 1).^2 + (yCoords - 1).^2);

[minDistanceUL, minIndexUL] = min(distances)

% Find out which index is closest to the upper right corner

distances = sqrt((xCoords-columns).^2 + (yCoords - 1).^2);

[minDistanceUR, minIndexUR] = min(distances)

% Find out which index is closest to the lower left corner

distances = sqrt((xCoords - 1).^2 + (yCoords - rows).^2);

[minDistanceLL, minIndexLL] = min(distances)

% Find out which index is closest to the lower right corner

distances = sqrt((xCoords - columns).^2 + (yCoords - rows).^2);

[minDistanceLR, minIndexLR] = min(distances)

% Plot circles over the corners, just for visualization purposes - for fun.

xCorners = [xCoords(minIndexUL), xCoords(minIndexUR), xCoords(minIndexLR), xCoords(minIndexLL)]

yCorners = [yCoords(minIndexUL), yCoords(minIndexUR), yCoords(minIndexLR), yCoords(minIndexLL)]

hold on;

plot(xCorners, yCorners, 'ro');

title('Bad image with corners located', 'FontSize', fontSize);

% Determine ideal corner locations - aligned with raster lines

x1 = mean([xCorners(1), xCorners(4)])

x2 = mean([xCorners(2), xCorners(3)])

y1 = mean([yCorners(1), yCorners(2)])

y2 = mean([yCorners(3), yCorners(4)])

% Show this

subplot(2, 2, 2);

imshow(binaryImage1);

hold on;

plot([x1 x2 x2 x1 x1], [y1 y1 y2 y2 y1], 'r-', 'LineWidth', 3);

title('Bad image with perfect rectangular overlaid', 'FontSize', fontSize);

% Warp the image to straighten it.

badXY = [xCorners; yCorners]'

desiredXY = [x1 x2 x2 x1; y1 y1 y2 y2]'

% Transform to a quadrilateral with vertices badXY

% into a quadrilateral with vertices desiredXY.

tform = maketform('projective', badXY, desiredXY);

% Fix/warp the image.

[binaryImage3, xdata, ydata] = imtransform(binaryImage1, ...

tform, 'bicubic', 'size', size(binaryImage1));

% Display the fixed image.

subplot(2, 2, 3);

imshow(binaryImage3);

title('Fixed image', 'FontSize', fontSize);

Sign in to comment.

Answer by Carine
on 11 Mar 2013

Thank you to the two of you, your answers have been very useful and the tutorial on sudoku solver impressive! I have mainly used the answer of Teja Muppirala but both give nearly the same result. To get rid off the white border I have inverted the colors first:

I2 = imclearborder(1-I); I2 = 1-I2;

Now I've still got a problem as the area size is important in my application. Here is a picture of nearly undeformed image (taken with an angle 0º more or less): <https://www.dropbox.com/s/2vpk39qi4hy9zsw/im40_0.JPG> and here is another one taken from an angle of 30º more or less: <https://www.dropbox.com/s/t0qv7yb6l9u68im/im40_30.JPG>. I would not only like to "straighten" it but also to recover more or less the same size (area). So what I do from Teja Muppirala's code is modifying the coordinates of the new corners in the following way:

C21 = C2;

% 5. Scale the new corners so that the size of the rectangle is the size of the largest borders of the original picture

% Ratio between horizontal borders

dh1 = distance(C(1,:),C(2,:));

dh2 = distance(C(4,:),C(3,:));

if dh1<dh2

dmh = dh1; dMh = dh2;

imax = 2;

else

dmh = dh2; dMh = dh1;

imax = 1;

end

C21(:,1) = C21(:,1)*dMh/dmh;

% Ratio between vertical borders

dh1 = distance(C(1,:),C(4,:));

dh2 = distance(C(2,:),C(3,:));

if dh1<dh2

dmv = dh1; dMv = dh2;

if imax == 1

imax = 2; %imaxv = 23, imaxh = 12

else

imax = 3; %imaxv = 23, imaxh = 34

end

else

dmv = dh2; dMv = dh1;

if imax == 1

imax = 1; %imaxv = 14, imaxh = 12

else

imax = 4; %imaxv = 14, imaxh = 34

end

end

C21(:,2) = C21(:,2)*dMv/dmv;

%imax is the index of the largest and longest border. Translate the new rectangle so that it passes by C2 of that index

for index = 1

C22(:,index) = C21(:,index)-C21(imax,index)+C2(imax,index);

end

where distance is just d = sqrt(sum(C1-C2).^2);.

What do you think of the idea? Any better one?

Thank you again!

Sign in to comment.

Opportunities for recent engineering grads.

Apply Today
## 0 Comments

Sign in to comment.