Asked by Carine
on 6 Mar 2013

Hello,

I have got this image https://www.dropbox.com/s/5hmy6tb3ym42tic/rect.jpg?m which is rectangle-like but not exactly.

I would like to find rectangular contours and I have tried to use **edge** and **hough** but it doesn't find continuous line. I have also tried to **erode** and **dilate** but it can't erase the difference of one pixel on the horizontal upper most line for example so it still doesn't find the continuous line.
The second part of the problem would be to straighten it.

Another idea I got was to find the **corners** to link them with straight lines but it doesn't work either.

I am sure it shouldn't be complicated but I can't make it. Any ideas will be welcome!

Cheers

Answer by Teja Muppirala
on 7 Mar 2013

Accepted answer

I've done this before: http://www.mathworks.com/videos/solving-a-sudoku-puzzle-using-a-webcam-68773.html

This is the strategy that I used and it seemed to work on your image as well. Basically you search for the corners, and then use IMTRANSFORM to transorm the corners into a rectangle. (I'll assume your image is called I).

%% 1. Get rid of the white border I2 = imclearborder(im2bw(I));

%% 2. Find each of the four corners [y,x] = find(I2);

[~,loc] = min(y+x); C = [x(loc),y(loc)];

[~,loc] = min(y-x); C(2,:) = [x(loc),y(loc)];

[~,loc] = max(y+x); C(3,:) = [x(loc),y(loc)];

[~,loc] = max(y-x); C(4,:) = [x(loc),y(loc)];

%% 3. Plot the corners imshow(I); hold all plot(C([1:4 1],1),C([1:4 1],2),'r','linewidth',3);

%% 4. Find the locations of the new corners L = mean(C([1 4],1)); R = mean(C([2 3],1)); U = mean(C([1 2],2)); D = mean(C([3 4],2)); C2 = [L U; R U; R D; L D];

%% 5. Do the image transform figure T = cp2tform(C ,C2,'projective'); IT = imtransform(im2bw(I),T); %IM2BW is not necessary imshow(IT); hold all

Answer by Image Analyst
on 7 Mar 2013

Carine:

You can straighten your image with imtransform(), like in this demo I wrote for you using your uploaded image:

clc; % Clear the command window. close all; % Close all figures (except those of imtool.) imtool close all; % Close all imtool figures if you have the Image Processing Toolbox. clear; % Erase all existing variables. Or clearvars if you want. workspace; % Make sure the workspace panel is showing. format long g; format compact; fontSize = 20;

% Change the current folder to the folder of this m-file. if(~isdeployed) cd(fileparts(which(mfilename))); end % Check that user has the Image Processing Toolbox installed. hasIPT = license('test', 'image_toolbox'); if ~hasIPT % User does not have the toolbox installed. message = sprintf('Sorry, but you do not seem to have the Image Processing Toolbox.\nDo you want to try to continue anyway?'); reply = questdlg(message, 'Toolbox missing', 'Yes', 'No', 'Yes'); if strcmpi(reply, 'No') % User said No, so exit. return; end end

% Read in a standard MATLAB gray scale demo image. folder = 'C:\Users\Carine\Documents'; baseFileName = 'rect.jpg'; % Get the full filename, with path prepended. fullFileName = fullfile(folder, baseFileName); % Check if file exists. if ~exist(fullFileName, 'file') % File doesn't exist -- didn't find it there. Check the search path for it. fullFileName = baseFileName; % No path this time. if ~exist(fullFileName, 'file') % Still didn't find it. Alert user. errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName); uiwait(warndlg(errorMessage)); return; end end grayImage = imread(fullFileName); % Get the dimensions of the image. % numberOfColorBands should be = 1. [rows columns numberOfColorBands] = size(grayImage); if numberOfColorBands > 1 grayImage = grayImage(:,:,2); % Take green channel end % Binarize the image binaryImage1 = grayImage > 128; % Get rid of the white border binaryImage1 = imclearborder(binaryImage1);

% Display the original gray scale image. subplot(2, 2, 1); imshow(binaryImage1, []); title('Binary Image', 'FontSize', fontSize); % Enlarge figure to full screen. set(gcf, 'units','normalized','outerposition',[0 0 1 1]); % Give a name to the title bar. set(gcf,'name','Demo by ImageAnalyst','numbertitle','off')

% Fill the object binaryImage2 = imfill(binaryImage1, 'holes'); % Find the boundaries boundaries = bwboundaries(binaryImage2); xCoords = boundaries{1}(:, 2); yCoords = boundaries{1}(:, 1);

% Find out which index is closest to the upper left corner distances = sqrt((xCoords - 1).^2 + (yCoords - 1).^2); [minDistanceUL, minIndexUL] = min(distances) % Find out which index is closest to the upper right corner distances = sqrt((xCoords-columns).^2 + (yCoords - 1).^2); [minDistanceUR, minIndexUR] = min(distances)

% Find out which index is closest to the lower left corner distances = sqrt((xCoords - 1).^2 + (yCoords - rows).^2); [minDistanceLL, minIndexLL] = min(distances) % Find out which index is closest to the lower right corner distances = sqrt((xCoords - columns).^2 + (yCoords - rows).^2); [minDistanceLR, minIndexLR] = min(distances)

% Plot circles over the corners, just for visualization purposes - for fun. xCorners = [xCoords(minIndexUL), xCoords(minIndexUR), xCoords(minIndexLR), xCoords(minIndexLL)] yCorners = [yCoords(minIndexUL), yCoords(minIndexUR), yCoords(minIndexLR), yCoords(minIndexLL)] hold on; plot(xCorners, yCorners, 'ro'); title('Bad image with corners located', 'FontSize', fontSize);

% Determine ideal corner locations - aligned with raster lines x1 = mean([xCorners(1), xCorners(4)]) x2 = mean([xCorners(2), xCorners(3)]) y1 = mean([yCorners(1), yCorners(2)]) y2 = mean([yCorners(3), yCorners(4)])

% Show this subplot(2, 2, 2); imshow(binaryImage1); hold on; plot([x1 x2 x2 x1 x1], [y1 y1 y2 y2 y1], 'r-', 'LineWidth', 3); title('Bad image with perfect rectangular overlaid', 'FontSize', fontSize);

% Warp the image to straighten it. badXY = [xCorners; yCorners]' desiredXY = [x1 x2 x2 x1; y1 y1 y2 y2]' % Transform to a quadrilateral with vertices badXY % into a quadrilateral with vertices desiredXY. tform = maketform('projective', badXY, desiredXY);

% Fix/warp the image. [binaryImage3, xdata, ydata] = imtransform(binaryImage1, ... tform, 'bicubic', 'size', size(binaryImage1)); % Display the fixed image. subplot(2, 2, 3); imshow(binaryImage3); title('Fixed image', 'FontSize', fontSize);

Answer by Carine
on 11 Mar 2013

Thank you to the two of you, your answers have been very useful and the tutorial on sudoku solver impressive! I have mainly used the answer of Teja Muppirala but both give nearly the same result. To get rid off the white border I have inverted the colors first:

I2 = imclearborder(1-I); I2 = 1-I2;

Now I've still got a problem as the area size is important in my application. Here is a picture of nearly undeformed image (taken with an angle 0º more or less): <https://www.dropbox.com/s/2vpk39qi4hy9zsw/im40_0.JPG> and here is another one taken from an angle of 30º more or less: <https://www.dropbox.com/s/t0qv7yb6l9u68im/im40_30.JPG>. I would not only like to "straighten" it but also to recover more or less the same size (area). So what I do from Teja Muppirala's code is modifying the coordinates of the new corners in the following way:

C21 = C2;

% 5. Scale the new corners so that the size of the rectangle is the size of the largest borders of the original picture % Ratio between horizontal borders dh1 = distance(C(1,:),C(2,:)); dh2 = distance(C(4,:),C(3,:)); if dh1<dh2 dmh = dh1; dMh = dh2; imax = 2; else dmh = dh2; dMh = dh1; imax = 1; end

C21(:,1) = C21(:,1)*dMh/dmh;

% Ratio between vertical borders dh1 = distance(C(1,:),C(4,:)); dh2 = distance(C(2,:),C(3,:)); if dh1<dh2 dmv = dh1; dMv = dh2; if imax == 1 imax = 2; %imaxv = 23, imaxh = 12 else imax = 3; %imaxv = 23, imaxh = 34 end else dmv = dh2; dMv = dh1; if imax == 1 imax = 1; %imaxv = 14, imaxh = 12 else imax = 4; %imaxv = 14, imaxh = 34 end end

C21(:,2) = C21(:,2)*dMv/dmv; %imax is the index of the largest and longest border. Translate the new rectangle so that it passes by C2 of that index for index = 1 C22(:,index) = C21(:,index)-C21(imax,index)+C2(imax,index); end

where distance is just d = sqrt(sum(C1-C2).^2);.

What do you think of the idea? Any better one?

Thank you again!

Opportunities for recent engineering grads.

## 0 Comments