# How can I use the Genetic Algorithm (GA) to train a Neural Network in Neural Network Toolbox?

367 views (last 30 days)
MathWorks Support Team on 9 Jul 2013
Answered: Rahul Biswas on 28 Sep 2021
How can I use the Global Optimization Toolbox "ga" to optimize the Neural Network?

MathWorks Support Team on 12 Nov 2020
Edited: MathWorks Support Team on 12 Nov 2020
The ability to set the algorithm to ga in the train function is not currently directly available in Neural Network Toolbox (as of R2017a at least).
To work around this issue, use the steps outlined below to optimize a neural network using a genetic algorithm.
The "ga" function requires a function handle as an input argument to which it passes a 1xN vector, where N is the number of variables in the system to be optimized.
For a neural network, the weights and biases are a Mx1 vector. These may be optimized using "ga".
A function can be written to accept the network, weights and biases, inputs and targets. This function may return the mean squared error based on the outputs and the targets as "ga" requires a function handle that only returns a scalar value.
The following code example describes a function that returns the mean squared error for a given input of weights and biases, a network, its inputs and targets.
function mse_calc = mse_test(x, net, inputs, targets)
% 'x' contains the weights and biases vector
% in row vector form as passed to it by the
% genetic algorithm. This must be transposed
% when being set as the weights and biases
% vector for the network.
%
% To set the weights and biases vector to the
% one given as input
net = setwb(net, x');
%
% To evaluate the ouputs based on the given
% weights and biases vector
y = net(inputs);
%
% Calculating the mean squared error
mse_calc = sum((y-targets).^2)/length(y);
end
The following code example describes a separate script that sets up a basic Neural Network problem and the definition of a function handle to be passed to "ga". It uses the above function to calculate the Mean Squared Error.\n
% INITIALIZE THE NEURAL NETWORK PROBLEM %
%
% inputs for the neural net
inputs = (1:10);
% targets for the neural net
targets = cos(inputs.^2);
%
% number of neurons
n = 2;
%
% create a neural network
net = feedforwardnet(n);
%
% configure the neural network for this dataset
net = configure(net, inputs, targets);
%
% create handle to the MSE_TEST function, that
% calculates MSE
h = @(x) mse_test(x, net, inputs, targets);
%
% Setting the Genetic Algorithms tolerance for
% minimum change in fitness function before
% terminating algorithm to 1e-8 and displaying
% each iteration's results.
ga_opts = gaoptimset('TolFun', 1e-8,'display','iter');
%
% PLEASE NOTE: For a feed-forward network
% with n neurons, 3n+1 quantities are required
% in the weights and biases column vector.
%
% a. n for the input weights
% b. n for the input biases
% c. n for the output weights
% d. 1 for the output bias
%
% running the genetic algorithm with desired options
[x_ga_opt, err_ga] = ga(h, 3*n+1, ga_opts);
Please note that the above example makes use of
, which was first introduced in R2010b.
merve ülkü on 12 Apr 2021
Hello MatWorks Team, I have 3x28 Input and 1x28 Target. How ı change this code for it working ? My code is below;
ı want to decided optimized 3 values for my target...
% inputs for the neural net
% targets for the neural net
% number of neurons
n = 2;
% create a neural network
net = feedforwardnet(n);
% configure the neural network for this dataset
net = configure(net, inputs, targets);
% create handle to the MSE_TEST function, that
% calculates MSE
h = @(x) mse_test(x, net, inputs, targets);
% Setting the Genetic Algorithms tolerance for
% minimum change in fitness function before
% terminating algorithm to 1e-8 and displaying
% each iteration's results.
ga_opts = gaoptimset('TolFun', 1e-8,'display','iter');
% PLEASE NOTE: For a feed-forward network
% with n neurons, 3n+1 quantities are required
% in the weights and biases column vector.
%
% a. n for the input weights
% b. n for the input biases
% c. n for the output weights
% d. 1 for the output bias
% running the genetic algorithm with desired options
[x_ga_opt, err_ga] = ga(h, 3*n+1, ga_opts);
function mse_calc = mse_test(x, net, inputs, targets)
% 'x' contains the weights and biases vector
% in row vector form as passed to it by the
% genetic algorithm. This must be transposed
% when being set as the weights and biases
% vector for the network.
% To set the weights and biases vector to the
% one given as input
net = setwb(net, x');
% To evaluate the ouputs based on the given
% weights and biases vector
y = net(inputs);
% Calculating the mean squared error
mse_calc = sum((y-targets).^2)/length(y);
end
Please share the right code thank you so much for helping.

Michael on 11 Jul 2014
Let me see if I understood. By executing this function ga at the end, "net" will be be optimized such that the mean squared error is minimized?
Best, Mike
merve ülkü on 12 Apr 2021
Hello MatWorks Team, I have 3x28 Input and 1x28 Target. How ı change this code for it working ? My code is below;
ı want to decided optimized 3 values for my target...
% inputs for the neural net
% targets for the neural net
% number of neurons
n = 2;
% create a neural network
net = feedforwardnet(n);
% configure the neural network for this dataset
net = configure(net, inputs, targets);
% create handle to the MSE_TEST function, that
% calculates MSE
h = @(x) mse_test(x, net, inputs, targets);
% Setting the Genetic Algorithms tolerance for
% minimum change in fitness function before
% terminating algorithm to 1e-8 and displaying
% each iteration's results.
ga_opts = gaoptimset('TolFun', 1e-8,'display','iter');
% PLEASE NOTE: For a feed-forward network
% with n neurons, 3n+1 quantities are required
% in the weights and biases column vector.
%
% a. n for the input weights
% b. n for the input biases
% c. n for the output weights
% d. 1 for the output bias
% running the genetic algorithm with desired options
[x_ga_opt, err_ga] = ga(h, 3*n+1, ga_opts);
function mse_calc = mse_test(x, net, inputs, targets)
% 'x' contains the weights and biases vector
% in row vector form as passed to it by the
% genetic algorithm. This must be transposed
% when being set as the weights and biases
% vector for the network.
% To set the weights and biases vector to the
% one given as input
net = setwb(net, x');
% To evaluate the ouputs based on the given
% weights and biases vector
y = net(inputs);
% Calculating the mean squared error
mse_calc = sum((y-targets).^2)/length(y);
end
Please share the right code thank you so much for helping.

Don Mathis on 7 Jun 2017
The code posted by MathWorks Support Team on 18 Oct 2013 throws an error if you put it in a file and run it as-is, because the function definition precedes the script. To fix it, you just need to put the function definition last. Here is a corrected version that seems to work fine. Just paste it into the MATLAB editor and hit the Run button.
% INITIALIZE THE NEURAL NETWORK PROBLEM %
% inputs for the neural net
inputs = (1:10);
% targets for the neural net
targets = cos(inputs.^2);
% number of neurons
n = 2;
% create a neural network
net = feedforwardnet(n);
% configure the neural network for this dataset
net = configure(net, inputs, targets);
% create handle to the MSE_TEST function, that
% calculates MSE
h = @(x) mse_test(x, net, inputs, targets);
% Setting the Genetic Algorithms tolerance for
% minimum change in fitness function before
% terminating algorithm to 1e-8 and displaying
% each iteration's results.
ga_opts = gaoptimset('TolFun', 1e-8,'display','iter');
% PLEASE NOTE: For a feed-forward network
% with n neurons, 3n+1 quantities are required
% in the weights and biases column vector.
%
% a. n for the input weights
% b. n for the input biases
% c. n for the output weights
% d. 1 for the output bias
% running the genetic algorithm with desired options
[x_ga_opt, err_ga] = ga(h, 3*n+1, ga_opts);
function mse_calc = mse_test(x, net, inputs, targets)
% 'x' contains the weights and biases vector
% in row vector form as passed to it by the
% genetic algorithm. This must be transposed
% when being set as the weights and biases
% vector for the network.
% To set the weights and biases vector to the
% one given as input
net = setwb(net, x');
% To evaluate the ouputs based on the given
% weights and biases vector
y = net(inputs);
% Calculating the mean squared error
mse_calc = sum((y-targets).^2)/length(y);
end
Caroline on 18 Sep 2020
I'm just starting to work with neural networks and I think I understand the theory, but I don't know how to apply it in practice. I did this exemplary example and the result does not match the function, why?
net = setwb(net, x_ga_opt);
net.trainFcn = 'trainlm';
net1 = train(net,inputs,targets);
w1 = net1(targets);
plot (inputs,w1)
hold on
plot (inputs,targets); what can i do to match the result with the function?

anurag kulshrestha on 3 May 2015
the above code is obsolete in Matlab R2010a. this code is not minimizing the mse function of ann.
Greg Heath on 17 May 2017
If you search on
greg genetic
and/or
greg ga
You will see that I cannot find a good working example of the MATLAB function ga.
If you are successful, please post
in the NEWSGROUP AND post a reference in ANSWERS.
Hope this helps.
Greg

kailash sahw shaw on 21 Jun 2017
Let us clear some point out of Logic used GA for Updating Weight of NN
Previously we have studied that weight keeps on updating for better optimized answer. but here the weight are fixed as they are just changing their position... So how come one can guarantee this will lead to optimize answer as the weight initially taken is random. Once random value assigned then we are not changing it, we just change the position of value for multiplication...
It means for every alternate time you run you will get different result...

Greg Heath on 21 Jun 2017
>Let us clear some point out of Logic used GA for Updating Weight of NN >Previously we have studied that weight keeps on updating for better >optimized answer.
That is because at each stage, MANY sets of random candidates are created
before one is found that decreases the error. Then that set of weights is accepted
and the multiple random searches are continued.
> but here the weight are fixed as they are just changing their position...
No! Absolutely not!
>So how come one can guarantee this will lead to optimize answer as the weight initially taken is random.
No. You do not understand.
1. ALL of the weights are randomly changed
2. The resulting error is calculated
3. If the new error is not lower than the existing error, the new set of
weights is discarded and the algorithm goes back to step 1.
4. If the new error is lower than the existing error, the new set of
weights are accepted.
a. If the new error is smaller or equal to the goal the algorithm is
terminated.
b. If the new error not smaller or equal to the goal, the algorithm
goes back to step1.
>Once random value assigned then we are not changing it, we just change >the position of value for multiplication...
No.
>It means for every alternate time you run you will get different result...
That is the point: Keep generating sets of weights until one lowers the error.
Accept that set. If the final error has reached the goal you are finished.
Otherwise start over with generating more randon sets.
Hope this helps.
Thank you for formally accepting my answer
Greg

Ahmed Abubakr on 20 Sep 2017
I was able to run the algorithm using multiple inputs I just had to change the code at which the new Y is calculated from the inputs using a simple for loop however the code is taking too long to run and i don't know how to optimize it, does any one have an idea on how to increase the code speed? I have 1920*3 inputs and 1920*1 output
##### 2 CommentsShowHide 1 older comment
merve ülkü on 12 Apr 2021
Hi Ahmed , can you share your working multiple input code and your datas ?

Joseph Ribeiro on 20 Nov 2017
i am using Matlab R2010b. The code does not work for me. It throws errors back at me. Any way I can a working one?
Thanks, Joseph

Joseph Ribeiro on 21 Nov 2017
Dear Greg, I would be grateful if the GA could be integrated into "bodyfat" code which is an example so i can follow it better. This is because I tried the code again as two separate m-files (since I am using the R2010b version) and it works. However, I am at a loss as to where exactly in the code I should insert the GA script.
If anyone else has tried it, I would be grateful if it could be pasted here.
Thanks.
Joseph
Greg Heath on 2 Dec 2017
Please do not misuse the ANSWER box. Obviously, you did not post an answer to the original question.
I have posted revisions of the MATLAB GA code.
Hope this helps,
Greg

Ayman Alrayes on 27 Jul 2018
why it takes too long time executing ?? all noticed that or it happens only with me?!! single run takes around half an hour!!
Greg Heath on 25 Feb 2019
Genetic Design of Neural Networks
1. Bring a lunch and a bottle of wine
2. ...
Greg

gan xiedang on 2 Nov 2019
the above code is obsolete in Matlab R2016b. this code is not minimizing the mse function of ann.
##### 2 CommentsShowHide 1 older comment
Morteza on 11 Jun 2020
Please let me know finally: Does the above code is correct to learn a ANN or that is not true?
MP

eakandar mazen on 3 May 2021
Radom Color Image Encryption by using the Genetic Algorithm and neural network can i get the code to run the simulatin

Rahul Biswas on 28 Sep 2021
How to divide the data the for training and testing. Kindly help.

R2010b

### Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!