How can I have Matlab use deep learning to identify key items in an image?

I am trying to have MATLAB use deep learning to automatically identify specific items within an image (namely, all the triangles located at the nodes). Can anyone give any input on how I would go about having MATLAB do this? Attached is:

1. The raw input image
2. A modified version of the input image that shows the objects of interest in the image (i.e., the triangles)

The x,y coordinates of the triangle vertices is what I ultimately need. Thanks in advance for your help!

1)

2)

Answer 1

[Image Analyst]

I'm sure if you had tried, you would have figured this out.

% Demo by Image Analyst
clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 22;
markerSize = 40;
%--------------------------------------------------------------------------------------------------------
%    READ IN IMAGE
folder = pwd;
baseFileName = 'triangles.png';
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~exist(fullFileName, 'file')
    % The file doesn't exist -- didn't find it there in that folder.
    % Check the entire search path (other folders) for the file by stripping off the folder.
    fullFileNameOnSearchPath = baseFileName; % No path this time.
    if ~exist(fullFileNameOnSearchPath, 'file')
        % Still didn't find it.  Alert user.
        errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
        uiwait(warndlg(errorMessage));
        return;
    end
end
rgbImage = imread(fullFileName);
% Get the dimensions of the image.
% numberOfColorChannels should be = 1 for a gray scale image, and 3 for an RGB color image.
[rows, columns, numberOfColorChannels] = size(rgbImage)
if numberOfColorChannels > 1
    % It's not really gray scale like we expected - it's color.
    fprintf('It is not really gray scale like we expected - it is color\n');
    % Extract the blue channel.
    grayImage = rgbImage(:, :, 3);
else
    grayImage = rgbImage;
end
% Update the dimensions of the image.
% numberOfColorChannels should be = 1 for a gray scale image, and 3 for an RGB color image.
[rows, columns, numberOfColorChannels] = size(grayImage)
%--------------------------------------------------------------------------------------------------------
% Display the image.
subplot(2, 2, 1);
imshow(grayImage);
impixelinfo;
axis('on', 'image');
title('Original Gray Scale Image', 'FontSize', fontSize, 'Interpreter', 'None');
% Maximize window.
g = gcf;
g.WindowState = 'maximized';
drawnow;
% Display histogram.
subplot(2, 2, 2);
histogram(grayImage);
grid on;
drawnow;
title('Histogram of Image', 'FontSize', fontSize, 'Interpreter', 'None');
%--------------------------------------------------------------------------------------------------------
% Binarize the image to get a mask.
lowThreshold = 0;
highThreshold = 182;
% https://www.mathworks.com/matlabcentral/fileexchange/29372-thresholding-an-image?s_tid=srchtitle
% threshold(lowThreshold, highThreshold, grayImage);
mask = grayImage >= lowThreshold & grayImage <= highThreshold;
% Put red threshold line on histogram so they know where it was thresholded at.
xline(highThreshold, 'Color', 'r', 'LineWidth', 2)
% Get rid of thin tendrils.
mask = imopen(mask, true(7));
% Take only the largest blobs.
mask = bwareafilt(mask, 2500);
% Display mask image.
subplot(2, 2, 3);
imshow(mask);
hold on;
impixelinfo;
axis('on', 'image');
drawnow;
title('Mask, a Binary Image', 'FontSize', fontSize, 'Interpreter', 'None');
% Label the two blobs so we can fit a line through wach one, one at a time.
[labeledImage, numBlobs] = bwlabel(mask);
% Get areas
props = regionprops(mask, 'Area', 'Centroid');
allAreas = [props.Area]
xy = vertcat(props.Centroid)
subplot(2, 2, 4);
histogram(allAreas, 10);
caption = sprintf('Size distribution of %d triangles', length(allAreas))
title(caption, 'FontSize',fontSize);
grid on;
xlabel('Area in Pixels', 'FontSize',fontSize)
ylabel('Count', 'FontSize',fontSize)

[Steven D]

Truly amazing! Thanks so much for taking the time to write up that beautiful code. You are awesome! I see that the code correctly identified every triangle in the input image. Is it possible to adjust the tolerance of some parameters in the code so that the output image is more like image number 2 that I had originally posted? Thanks again for your help!!!

[Steven D]

I managed to get closer to the image by adjusting the "highThreshold" setting. However, because the output image was binary, the triangle blobs were still a bit blocky. I may need to add another step to "feather" out the triangles by blurring, and then re-binarizing them in the code somewhere. The reason why i need to have more fine detail is that I will need to locate the precise endpoints/vertices of the triangles (in a future step) for analysis.

[Image Analyst]

I thought you just need the location of the triangles. What difference does it make what the shapes of the blobs are?

[Steven D]

I need the blobs to be the actual triangles (that are seen in the input image) because I need the exact locations (x,y coordinates) of the endpoints/vertices of the triangles for future analysis.

[Image Analyst]

Then I'd work on finding the cell walls instead of the triangles. Then call bwskel. If you have a mask of the cell walls then you can find the connection points with bwmorph. Then erase each connection point by setting it to false. Then call bwmorph again to get the endpoints of the lines, which will be where the points of the triangles were.

[Steven D]

Once I have accurate, crisp, triangles, I skel them into tripods. Then, I find the endpoints/verticies from those. Please see attached pics that show this:

I have this part all worked out. I was just looking for a quicker (and preferably automated) way to identify and isolate the triangles from the input image.

[Steven D]

Ideally, I would like to know how to have MATLAB instantly recognize (maybe via deep learning), all the triangles' vertices (see white dots in attached pic) and then store the coordinates of the same.

[Image Analyst]

You can use segnet but you'll have to have lots of example images. I am working on a general purpose segnet training app but it's not ready to be released yet. You might just use what I gave you and for each centroid crop out a little box and then try to better threshold the dark triangles. Then find the vertices using bwboundaries() and findpeaks() on the centroid to border distances.

[Steven D]

I'll definitely give it a shot. Thanks again for all your help!

Answer 2

[Image Analyst]

You can probably get those with traditional methods like thresholding and blob analysis.

For a deep learning solution you can use SegNet.

[Steven D]

Thanks for your comment. Would you happen to have any specific code examples that I could try?