How do I integrate a trained neural network into an application

I am trying to compare an uploaded image to a trained neural network in an application. I have been ablet o upload an image using a push button, however, I am not able to have that image analyzed by the trained neural network. Below is the code for the application and I am unsure where the failure is occuring. Please let me know what you think would be the best way to go about this. Thank You

methods (Access = private)        
        function results = startupfunc(app)
            x_net = load("xraycat.mat");
            x_net = app.net.net;
        end
    end
    

    methods (Access = private)

        % Button pushed function: UploadImageButton
        function UploadImageButtonPushed(app, event)
[File_Name, Path_Name] = uigetfile('PATHNAME');
imshow([Path_Name,File_Name],'Parent',app.UIAxes);
        end

        % Button pushed function: AnalyzeImageButton
        function AnalyzeImageButtonPushed(app, event)
[YPred, Scores] = predict(app.net, [File_name, Path_Name]);
imshow([YPred, Scores],'Parent',app.UIAxes2);
        end
    end

Answer 1

[Kojiro Saito]

You need add a property to pass that variable between functions.

In Code Browser panel in Code View, click Properties and click plus icon.

In properties, you can add a property, for example, variable name is filepath.

properties (Access = private)
    filepath % file path
    net % Trained Neural Network
end

After that, change your code as below.

function startupfunc(app)
    app.net = load("xraycat.mat");
    app.net = app.net.net;
end

% Button pushed function: UploadImageButton
function UploadImageButtonPushed(app, event)
    [File_Name, Path_Name] = uigetfile('PATHNAME');
    app.filepath = fullfile(Path_Name,File_Name);
    imshow(app.filepath,'Parent',app.UIAxes);
end

% Button pushed function: AnalyzeImageButton
function AnalyzeImageButtonPushed(app, event)
    imds = imageDatastore(app.filepath);
    [YPred, Scores] = classify(app.net, imds);
    % or,  
    % YPred = predict(app.net, imds);
    % or, for SVM classification
    % im = imread(app.filepath);
    % featureLayer = 'fc7'; % For AlexNet
    % imageFeatures = activations(app.net, im, featureLayer);
    % [YPred, Scores] = predict(app.net, imageFeatures);
    imshow([YPred, Scores],'Parent', app.UIAxes2);
end

I'm not sure which predict function you're using because there are some functions in MATLAB such as

• Deep Learning Toolbox's predict (https://www.mathworks.com/help/deeplearning/ref/predict.html) which returns YPred.
• Deep Learning Toolbox's classify (https://www.mathworks.com/help/deeplearning/ref/classify.html) which returns YPred and Scores.
• Statistics and Machine Learning Toolbox's SVM predict (https://www.mathworks.com/help/stats/classificationlinear.predict.html) which returns YPred and Scores.

But none of them can allow image's file path, so I changed your second input argument to imageDatastore in the above code.

[Patrick Kuczwara]

Using Classify I recieve the following error: Error using classify (line 123)

Requires at least three arguments.

Using Predict the following error: Error using predict (line 84)

No valid system or dataset was specified.

As the trained neural network is a retrained googlenet, I'm not sure if that makes a difference. Since both 'classify' and 'predict' are contained within the deep learning toolbox I am unsure as to why they are not working. I appreciate all of your help up to this point.

[Kojiro Saito]

The errors of classify and predict are those of Statistics and Machine Learning Toolbox. Could you tell me what class types of your retrained network? You can find the class types from,

load("xraycat.mat")
whos net

[Patrick Kuczwara]

Here is what was returned.

 Name      Size               Bytes  Class         Attributes

  net       1x1             24721426  DAGNetwork   

I am not sure if what I am aiming to do is even possible. However I greatly appreciate your help.

As a reference, I retrained a googlenet neural network to a new data set which was compiled. Thank you for all of your help with this project.

[Kojiro Saito]

All right, you're loading DAGNetwork, so function DAGNetwork/classify should be called instead of stats/classify.

Could you add function pragma in the first line of AnalyzeImageButtonPushed to force MATLAB Compiler to use DAGNetwork class?

function AnalyzeImageButtonPushed(app, event)
  %#function DAGNetwork
  im = imread(app.filepath);
  im = imresize(im, [224 224]); % Resize image to GoogleNetwork input size
  [YPred, Scores] = classify(app.net, im);
  sMax = max(Scores);
  frame = insertText(im, [0 5], [char(YPred) ':' num2str(sMax)]); % Inser label text to image
  imshow(frame, 'Parent', app.UIAxes2);
end

[Patrick Kuczwara]

I added the suggested pragma and code, however I am still receiving this error:

Error using classify (line 123)
Requires at least three arguments.

I tried reverting it to an imageDatastore, however this returned the same error. I do believe the rest is working, this is just the final hurdle.

Thank You.

[Kojiro Saito]

Thank you for testing. Instead of function pragma code, how about the following code. This is forcing to use DAGNetwork class.

function startupfunc(app)
    app.net = load("xraycat.mat");
    app.net = DAGNetwork.loadobj(app.net.net);
end

Could you try this?

[Patrick Kuczwara]

This as well gives me the same error as above. Is there a way to ensure that the network is being loaded into the application properly? Just to ensure that the issue lies in the analyzing of the image itself, rather than there possibly being an issue somewhere else.

Thank You, for all of the help to this point.

[Kojiro Saito]

You can know network was loaded properly by adding a message box with the network's classname.

function startupfunc(app)
    app.net = load("xraycat.mat");
    app.net = DAGNetwork.loadobj(app.net.net);
    msgbox(sprintf("Network is %s", class(app.net)))
end

Also, could you confirm your app works fine in MATLAB desktop before compiling? In MATLAB desktop, we can place break points so we can trace what is working fine line by line.

[Patrick Kuczwara]

So I loaded all of the functions into a script that were contained within the app, minust the startup functions, button pushes, etc. As below:

%% Loading Network   
app.net = load("xraycat.mat");
app.net = DAGNetwork.loadobj(app.net.net);
% msgbox(sprintf("Network is %s", class(app.net))) % Commented out after proper loading was seen.

%% Retrieving Image
[File_Name, Path_Name] = uigetfile('PATHNAME');
app.filepath = fullfile(Path_Name,File_Name);

%% Analyzing Image  
im = imread(app.filepath);
im = imresize(im, [224 224]); % Resize image to GoogleNetwork input size
[YPred, Scores] = classify(app.net, im);
sMax = max(Scores);
frame = insertText(im, [0 5], [char(YPred) ':' num2str(sMax)]); % Insert label text to image
imshow(frame)

When this was run everything worked well and the frame was shown with the score and name on the image. Therefore we know that everything works, however, there is just something that does not add up within the application itself. As I would like to be able to share this with other non-MATLAB computers, how else could a user interface be built?

[Kojiro Saito]

A simple way is this:

1. Change the script to function (Insert function at the top) and name the m file as the same as function name.

For example,

myFunc.m

function myFunc()
    %% Loading Network   
    app.net = load("xraycat.mat");
    app.net = DAGNetwork.loadobj(app.net.net);
    % msgbox(sprintf("Network is %s", class(app.net))) % Commented out after proper loading was seen.
    %% Retrieving Image
    [File_Name, Path_Name] = uigetfile('PATHNAME');
    app.filepath = fullfile(Path_Name,File_Name);
    %% Analyzing Image  
    im = imread(app.filepath);
    im = imresize(im, [224 224]); % Resize image to GoogleNetwork input size
    [YPred, Scores] = classify(app.net, im);
    sMax = max(Scores);
    frame = insertText(im, [0 5], [char(YPred) ':' num2str(sMax)]); % Insert label text to image
    imshow(frame)
end

Then, from Apps, click Application Compiler (installation and license of MATLAB Compiler are needed) and convert the myFunc.m to exe file which runs with MATLAB Runtime in non-MATLAB users machine.

If the exe file works, it your App Designer app will also work fine, so you can convert your app file (.mlapp) to Standalone application (.exe) or Web Apps (.ctf).

[Patrick Kuczwara]

Now since this works, I would like to output the top 5 proabable matches with their categorical lable. If possible, I would like to output standard images for each one that is matched. Please let me know if this is possible. Thanks

[Malina Balint]

I also had the exact same problem with my application and I followed the steps above, but I don't understand how can I create an interface with buttons that works, because from what I've understood so far if I put my function in app compiler it creates a application without an interface an so I still need an interface for my application. Thanks in advance