How can I read all 0 values from all rows and columns?

Hi guys,

I'm trying to draw a centre point for all the rectangles from the picture below.

When I import the image and convert using the function "rgb2gray" I end up getting a 2D table.

The point here is to draw a line so I can follow a trajectory.

I tried to use some computer vision functions but it wasn't working as expected so I think the best approach will be getting the centre point and then draw a line connecting all the dots.

The zero values from my table represent the black lines from the image.

This is what I have.

 screen=imread('screenshoot_S10+_screen_test.jpg')  %importing image
 
gray_rows =rgb2gray(screen) %converting into a 2D table
gray= gray_rows'     %converting previews rows into columns

l = find((gray_rows( ) == 0))
c= find( (gray()== 0))'

For some reasson my c var is not reading the rows of table "grey".

Answer 1

[Image Analyst]

Pedro:

Upload a PNG file. You can't really count on the image if it's a JPG image. The lines may be blurry due to lossy compression artifacts. If you just find the centroid of every square and rectangle (ones with no other ones niterior to them) then you're going to have centroids all over the place. How is it then supposed to know what trajectory you want?

In the meantime, this is what I have so far:

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 22;

%--------------------------------------------------------------------------------------------------------
%    READ IN IMAGE
folder = pwd;
baseFileName = 'rectangles.jpg';
% Get the full filename, with path prepended.
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~exist(fullFileName, 'file')
	% The file doesn't exist -- didn't find it there in that folder.
	% Check the entire search path (other folders) for the file by stripping off the folder.
	fullFileNameOnSearchPath = baseFileName; % No path this time.
	if ~exist(fullFileNameOnSearchPath, 'file')
		% Still didn't find it.  Alert user.
		errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
		uiwait(warndlg(errorMessage));
		return;
	end
end
grayImage = imread(fullFileName);

% Get the dimensions of the image.
% numberOfColorChannels should be = 1 for a gray scale image, and 3 for an RGB color image.
[rows, columns, numberOfColorChannels] = size(grayImage)
if numberOfColorChannels > 1
	% It's not really gray scale like we expected - it's color.
	% Use weighted sum of ALL channels to create a gray scale image.
	grayImage = rgb2gray(grayImage);
	% ALTERNATE METHOD: Convert it to gray scale by taking only the green channel,
	% which in a typical snapshot will be the least noisy channel.
	% grayImage = grayImage(:, :, 2); % Take green channel.
end
% Display the image.
subplot(1, 2, 1);
imshow(grayImage);
axis('on', 'image');
title('Original Grayscale Image', 'FontSize', fontSize, 'Interpreter', 'None');
impixelinfo;
hFig = gcf;
hFig.WindowState = 'maximized'; % May not work in earlier versions of MATLAB.
drawnow;

%--------------------------------------------------------------------------------------------------------
% Get a mask that is the entire image.
mask = grayImage == 255;
% Get rid of small things
mask = bwareaopen(mask, 50);
% xline(threshold, 'LineWidth', 2, 'Color', 'r');
h3 = subplot(1, 2, 2);
imshow(mask, []);
axis('on', 'image');
title('Mask Image with Rect Centroids', 'FontSize', fontSize, 'Interpreter', 'None');
impixelinfo;

% Get the centroid of each blob.
props = regionprops(mask, grayImage, 'Centroid', 'Area');
allAreas = sort([props.Area])
xy = vertcat(props.Centroid)
numBlobs = length(props)
hold on
plot(xy(:, 1), xy(:, 2), 'r.', 'MarkerSize', 20);
msgbox('Done');

[Pedro Marques]

Thank you for your reply.

In fact, changing screenshot from jpg to a png did make a significant difference in the image processing.

And you're right! It's going to be too much centroids to deal with.

My first approach was this:

printpath=imread('Screenshot_MT.png') %import

gray =rgb2gray(printpath) %2D table

mask= gray > 254    % mask 
imshow(mask)

se=strel('rectangle',[70 50])  %this is not the appropiate size of rectangle, I'm still iterating this one

imerod = imerode(mask,se)
imshow(imerod)

imerod = bwdist(imerod,'euclidean'); %convert into the negative image
imshow(imerod)

%incomplement(imerode)

imerod =imerod >50 %thickness of line

out= bwmorph(imerod,'skel',inf)
imshow(out)

imshow(labeloverlay(printpath,out,'Transparency',0))

This is what I get:

[Image Analyst]

I'm not really sure what your goal is. What do you want as your desired output? If you want equations of the lines try fitPolynomialRANSAC(), if you have the Computer Vision Toolbox.

[Pedro Marques]

This image corresponds to a touchscreen test, perform on Samsung devices. The code to activate this test is *#0*# and then "Touch" button.

The goal is to define a trajectory that will be replicated with a robot arm.

When the rectangles are reached they turn green, see pic below.

I could try to define the path and then verify is the goal was achieved by searching for green rectangles(not sure how to do this yet).

I'm still new to image processing and I'm trying to find the best approach to do this. Do you have any suggestions?

[Image Analyst]

I don't think you can determine the path if you don't also have timing information. If all you have is an image, those locations could have been made in millions of different ways. For example you could draw the outer box first then the X or you could draw 4 triangles. There is no way to know without timing info.