# How can I use the Genetic Algorithm (GA) to train a Neural Network in Neural Network Toolbox?

367 views (last 30 days)

Show older comments

How can I use the Global Optimization Toolbox "ga" to optimize the Neural Network?

### Accepted Answer

MathWorks Support Team
on 12 Nov 2020

Edited: MathWorks Support Team
on 12 Nov 2020

The ability to set the algorithm to ga in the train function is not currently directly available in Neural Network Toolbox (as of R2017a at least).

To work around this issue, use the steps outlined below to optimize a neural network using a genetic algorithm.

The "ga" function requires a function handle as an input argument to which it passes a 1xN vector, where N is the number of variables in the system to be optimized.

For a neural network, the weights and biases are a Mx1 vector. These may be optimized using "ga".

A function can be written to accept the network, weights and biases, inputs and targets. This function may return the mean squared error based on the outputs and the targets as "ga" requires a function handle that only returns a scalar value.

The following code example describes a function that returns the mean squared error for a given input of weights and biases, a network, its inputs and targets.

function mse_calc = mse_test(x, net, inputs, targets)

% 'x' contains the weights and biases vector

% in row vector form as passed to it by the

% genetic algorithm. This must be transposed

% when being set as the weights and biases

% vector for the network.

%

% To set the weights and biases vector to the

% one given as input

net = setwb(net, x');

%

% To evaluate the ouputs based on the given

% weights and biases vector

y = net(inputs);

%

% Calculating the mean squared error

mse_calc = sum((y-targets).^2)/length(y);

end

The following code example describes a separate script that sets up a basic Neural Network problem and the definition of a function handle to be passed to "ga". It uses the above function to calculate the Mean Squared Error.\n

% INITIALIZE THE NEURAL NETWORK PROBLEM %

%

% inputs for the neural net

inputs = (1:10);

% targets for the neural net

targets = cos(inputs.^2);

%

% number of neurons

n = 2;

%

% create a neural network

net = feedforwardnet(n);

%

% configure the neural network for this dataset

net = configure(net, inputs, targets);

%

% create handle to the MSE_TEST function, that

% calculates MSE

h = @(x) mse_test(x, net, inputs, targets);

%

% Setting the Genetic Algorithms tolerance for

% minimum change in fitness function before

% terminating algorithm to 1e-8 and displaying

% each iteration's results.

ga_opts = gaoptimset('TolFun', 1e-8,'display','iter');

%

% PLEASE NOTE: For a feed-forward network

% with n neurons, 3n+1 quantities are required

% in the weights and biases column vector.

%

% a. n for the input weights

% b. n for the input biases

% c. n for the output weights

% d. 1 for the output bias

%

% running the genetic algorithm with desired options

[x_ga_opt, err_ga] = ga(h, 3*n+1, ga_opts);

Please note that the above example makes use of

, which was first introduced in R2010b.

##### 9 Comments

merve ülkü
on 12 Apr 2021

Hello MatWorks Team, I have 3x28 Input and 1x28 Target. How ı change this code for it working ? My code is below;

ı want to decided optimized 3 values for my target...

% inputs for the neural net

inputs = xlsread('myinputs.xlsx');

% targets for the neural net

targets = xlsread('mytargets.xlsx');

% number of neurons

n = 2;

% create a neural network

net = feedforwardnet(n);

% configure the neural network for this dataset

net = configure(net, inputs, targets);

% create handle to the MSE_TEST function, that

% calculates MSE

h = @(x) mse_test(x, net, inputs, targets);

% Setting the Genetic Algorithms tolerance for

% minimum change in fitness function before

% terminating algorithm to 1e-8 and displaying

% each iteration's results.

ga_opts = gaoptimset('TolFun', 1e-8,'display','iter');

% PLEASE NOTE: For a feed-forward network

% with n neurons, 3n+1 quantities are required

% in the weights and biases column vector.

%

% a. n for the input weights

% b. n for the input biases

% c. n for the output weights

% d. 1 for the output bias

% running the genetic algorithm with desired options

[x_ga_opt, err_ga] = ga(h, 3*n+1, ga_opts);

function mse_calc = mse_test(x, net, inputs, targets)

% 'x' contains the weights and biases vector

% in row vector form as passed to it by the

% genetic algorithm. This must be transposed

% when being set as the weights and biases

% vector for the network.

% To set the weights and biases vector to the

% one given as input

net = setwb(net, x');

% To evaluate the ouputs based on the given

% weights and biases vector

y = net(inputs);

% Calculating the mean squared error

mse_calc = sum((y-targets).^2)/length(y);

end

Please share the right code thank you so much for helping.

### More Answers (12)

Michael
on 11 Jul 2014

Hi. Thanks for this answer.

Let me see if I understood. By executing this function ga at the end, "net" will be be optimized such that the mean squared error is minimized?

Best, Mike

##### 9 Comments

merve ülkü
on 12 Apr 2021

Hello MatWorks Team, I have 3x28 Input and 1x28 Target. How ı change this code for it working ? My code is below;

ı want to decided optimized 3 values for my target...

% inputs for the neural net

inputs = xlsread('myinputs.xlsx');

% targets for the neural net

targets = xlsread('mytargets.xlsx');

% number of neurons

n = 2;

% create a neural network

net = feedforwardnet(n);

% configure the neural network for this dataset

net = configure(net, inputs, targets);

% create handle to the MSE_TEST function, that

% calculates MSE

h = @(x) mse_test(x, net, inputs, targets);

% Setting the Genetic Algorithms tolerance for

% minimum change in fitness function before

% terminating algorithm to 1e-8 and displaying

% each iteration's results.

ga_opts = gaoptimset('TolFun', 1e-8,'display','iter');

% PLEASE NOTE: For a feed-forward network

% with n neurons, 3n+1 quantities are required

% in the weights and biases column vector.

%

% a. n for the input weights

% b. n for the input biases

% c. n for the output weights

% d. 1 for the output bias

% running the genetic algorithm with desired options

[x_ga_opt, err_ga] = ga(h, 3*n+1, ga_opts);

function mse_calc = mse_test(x, net, inputs, targets)

% 'x' contains the weights and biases vector

% in row vector form as passed to it by the

% genetic algorithm. This must be transposed

% when being set as the weights and biases

% vector for the network.

% To set the weights and biases vector to the

% one given as input

net = setwb(net, x');

% To evaluate the ouputs based on the given

% weights and biases vector

y = net(inputs);

% Calculating the mean squared error

mse_calc = sum((y-targets).^2)/length(y);

end

Please share the right code thank you so much for helping.

Don Mathis
on 7 Jun 2017

The code posted by MathWorks Support Team on 18 Oct 2013 throws an error if you put it in a file and run it as-is, because the function definition precedes the script. To fix it, you just need to put the function definition last. Here is a corrected version that seems to work fine. Just paste it into the MATLAB editor and hit the Run button.

% INITIALIZE THE NEURAL NETWORK PROBLEM %

% inputs for the neural net

inputs = (1:10);

% targets for the neural net

targets = cos(inputs.^2);

% number of neurons

n = 2;

% create a neural network

net = feedforwardnet(n);

% configure the neural network for this dataset

net = configure(net, inputs, targets);

% create handle to the MSE_TEST function, that

% calculates MSE

h = @(x) mse_test(x, net, inputs, targets);

% Setting the Genetic Algorithms tolerance for

% minimum change in fitness function before

% terminating algorithm to 1e-8 and displaying

% each iteration's results.

ga_opts = gaoptimset('TolFun', 1e-8,'display','iter');

% PLEASE NOTE: For a feed-forward network

% with n neurons, 3n+1 quantities are required

% in the weights and biases column vector.

%

% a. n for the input weights

% b. n for the input biases

% c. n for the output weights

% d. 1 for the output bias

% running the genetic algorithm with desired options

[x_ga_opt, err_ga] = ga(h, 3*n+1, ga_opts);

function mse_calc = mse_test(x, net, inputs, targets)

% 'x' contains the weights and biases vector

% in row vector form as passed to it by the

% genetic algorithm. This must be transposed

% when being set as the weights and biases

% vector for the network.

% To set the weights and biases vector to the

% one given as input

net = setwb(net, x');

% To evaluate the ouputs based on the given

% weights and biases vector

y = net(inputs);

% Calculating the mean squared error

mse_calc = sum((y-targets).^2)/length(y);

end

##### 5 Comments

Caroline
on 18 Sep 2020

I'm just starting to work with neural networks and I think I understand the theory, but I don't know how to apply it in practice. I did this exemplary example and the result does not match the function, why?

net = setwb(net, x_ga_opt);

net.trainFcn = 'trainlm';

net1 = train(net,inputs,targets);

w1 = net1(targets);

plot (inputs,w1)

hold on

plot (inputs,targets);

what can i do to match the result with the function?

anurag kulshrestha
on 3 May 2015

the above code is obsolete in Matlab R2010a. this code is not minimizing the mse function of ann.

##### 7 Comments

Greg Heath
on 17 May 2017

If you search on

greg genetic

and/or

greg ga

You will see that I cannot find a good working example of the MATLAB function ga.

If you are successful, please post

in the NEWSGROUP AND post a reference in ANSWERS.

Hope this helps.

Greg

kailash sahw shaw
on 21 Jun 2017

Let us clear some point out of Logic used GA for Updating Weight of NN

Previously we have studied that weight keeps on updating for better optimized answer. but here the weight are fixed as they are just changing their position... So how come one can guarantee this will lead to optimize answer as the weight initially taken is random. Once random value assigned then we are not changing it, we just change the position of value for multiplication...

It means for every alternate time you run you will get different result...

##### 0 Comments

Greg Heath
on 21 Jun 2017

>Let us clear some point out of Logic used GA for Updating Weight of NN >Previously we have studied that weight keeps on updating for better >optimized answer.

That is because at each stage, MANY sets of random candidates are created

before one is found that decreases the error. Then that set of weights is accepted

and the multiple random searches are continued.

> but here the weight are fixed as they are just changing their position...

No! Absolutely not!

>So how come one can guarantee this will lead to optimize answer as the weight initially taken is random.

No. You do not understand.

1. ALL of the weights are randomly changed

2. The resulting error is calculated

3. If the new error is not lower than the existing error, the new set of

weights is discarded and the algorithm goes back to step 1.

4. If the new error is lower than the existing error, the new set of

weights are accepted.

a. If the new error is smaller or equal to the goal the algorithm is

terminated.

b. If the new error not smaller or equal to the goal, the algorithm

goes back to step1.

>Once random value assigned then we are not changing it, we just change >the position of value for multiplication...

No.

>It means for every alternate time you run you will get different result...

That is the point: Keep generating sets of weights until one lowers the error.

Accept that set. If the final error has reached the goal you are finished.

Otherwise start over with generating more randon sets.

Hope this helps.

Thank you for formally accepting my answer

Greg

##### 0 Comments

Ahmed Abubakr
on 20 Sep 2017

##### 2 Comments

merve ülkü
on 12 Apr 2021

Hi Ahmed , can you share your working multiple input code and your datas ?

Joseph Ribeiro
on 20 Nov 2017

i am using Matlab R2010b. The code does not work for me. It throws errors back at me. Any way I can a working one?

Thanks, Joseph

##### 0 Comments

Joseph Ribeiro
on 21 Nov 2017

Dear Greg, I would be grateful if the GA could be integrated into "bodyfat" code which is an example so i can follow it better. This is because I tried the code again as two separate m-files (since I am using the R2010b version) and it works. However, I am at a loss as to where exactly in the code I should insert the GA script.

If anyone else has tried it, I would be grateful if it could be pasted here.

Thanks.

Joseph

##### 1 Comment

Greg Heath
on 2 Dec 2017

Please do not misuse the ANSWER box. Obviously, you did not post an answer to the original question.

I have posted revisions of the MATLAB GA code.

Check the NEWSGROUP and ANSWERS.

Hope this helps,

Greg

Ayman Alrayes
on 27 Jul 2018

##### 1 Comment

Greg Heath
on 25 Feb 2019

Genetic Design of Neural Networks

- Bring a lunch and a bottle of wine
- ...

Greg

gan xiedang
on 2 Nov 2019

the above code is obsolete in Matlab R2016b. this code is not minimizing the mse function of ann.

##### 2 Comments

Morteza
on 11 Jun 2020

Please let me know finally: Does the above code is correct to learn a ANN or that is not true?

Please let me know.

MP

eakandar mazen
on 3 May 2021

##### 0 Comments

Rahul Biswas
on 28 Sep 2021

How to divide the data the for training and testing. Kindly help.

##### 0 Comments

### See Also

### Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!