# To optimize a neural network of multiple inputs using a genetic algorithm.

50 views (last 30 days)
Chuthamat on 26 Feb 2015
I have found the answer from the matlab team but the code is applied for a single input. I have tried to modify for 4 inputs but it was not success. Could you please give me the code to modify the below code for the 4 inputs?
The GA function requires a function handle as an input argument to which it passes a 1xN vector, where N is the number of variables in the system to be optimized.
For a neural network, the weights and biases are a Mx1 vector. These may be optimized using GA.
A function can be written to accept the network, weights and biases, inputs and targets. This function may return the mean squared error based on the outputs and the targets as GA requires a function handle that only returns a scalar value.
The following code example describes a function that returns the mean squared error for a given input of weights and biases, a network, its inputs and targets.
function mse_calc = mse_test(x, net, inputs, targets)
% 'x' contains the weights and biases vector
% in row vector form as passed to it by the
% genetic algorithm. This must be transposed
% when being set as the weights and biases
% vector for the network.
% To set the weights and biases vector to the
% one given as input
net = setwb(net, x');
% To evaluate the ouputs based on the given
% weights and biases vector
y = net(inputs);
% Calculating the mean squared error
mse_calc = sum((y-targets).^2)/length(y);
end
The following code example describes a script that sets up a basic Neural Network problem and the definition of a function handle to be passed to GA. It uses the above function to calculate the Mean Squared Error.
% INITIALIZE THE NEURAL NETWORK PROBLEM %
% inputs for the neural net
inputs = (1:10);
% targets for the neural net
targets = cos(inputs.^2);
% number of neurons
n = 2;
% create a neural network
net = feedforwardnet(n);
% configure the neural network for this dataset
net = configure(net, inputs, targets);
% create handle to the MSE_TEST function, that
% calculates MSE
h = @(x) mse_test(x, net, inputs, targets);
% Setting the Genetic Algorithms tolerance for
% minimum change in fitness function before
% terminating algorithm to 1e-8 and displaying
% each iteration's results.
ga_opts = gaoptimset('TolFun', 1e-8,'display','iter');
% PLEASE NOTE: For a feed-forward network
% with n neurons, 3n+1 quantities are required
% in the weights and biases column vector.
%
% a. n for the input weights
% b. n for the input biases
% c. n for the output weights
% d. 1 for the output bias
% running the genetic algorithm with desired options
[x_ga_opt, err_ga] = ga(h, 3*n+1, ga_opts);
##### 2 CommentsShowHide 1 older comment
Matt Talebi on 27 May 2015
Hi Greg, two things regarding the use of these codes: first, unless adding x = getwb(net) to the fitness function it returns error (Undefined function or variable 'x'). Following this modification and by running the [x_ga_opt, err_ga] = ga(h, 3*n+1, ga_opts), it shows no improvement in minimizing the MSE. The MSE that I get is equal to the one that I obtain for the trained network without using GA. I appreciate your comment on this. Cheers/Matt.

Greg Heath on 1 Mar 2015
Edited: Greg Heath on 1 Mar 2015
function NMSE_calc = NMSE( wb, net, input, target)
% wb is the weights and biases row vector obtained from the genetic algorithm.
% It must be transposed when transferring the weights and biases to the network net.
net = setwb(net, wb');
% The net output matrix is given by net(input). The corresponding error matrix is given by
error = target - net(input);
% The mean squared error normalized by the mean target variance is
NMSE_calc = mean(error.^2)/mean(var(target',1));
% It is independent of the scale of the target components and related to the Rsquare statistic via
% Rsquare = 1 - NMSEcalc ( see Wikipedia)
Greg Heath on 13 Apr 2015
My advice is to search in either the NEWSGROUP or answers using the search words
greg Nw
You can probably just search on Nw, the number of unknown weights to estimate.
For I-dimensional inputs, O-dimensional outputs and H hidden nodes
Nw = (I+1)*H +(H+1)*O
where the "1"s indicate bias connections.

anurag kulshrestha on 25 Apr 2015
I applied the above code for a single input with 3 hidden neurons and a single output in Matlab 7.10.1(R 2010a). But mse is not getting minimized. Code is given below with few change in function
net = newff(inputs,targets,n); where n=3
x= getx(net);
y=sim(net,inputs); where y is the output
e= targets-y; where e is error
h = @(x)mse(e,y,x);
ga_opts = gaoptimset('TolFcn', 1e-8,'display','iter');
[x_ga_opt, err_ga] = ga(h,3*n+1,ga_opts);
Greg Heath on 26 Sep 2016
h = @(x)mse(e,y,x);
is an error. Check the documentation
help mse
doc mse
Hope this helps.
Greg

Alexandra Sikinioti-Lock on 18 Jul 2016
I had the same problem it seems to have been resolved with the formwb. I altered the function like this.
function mse_calc = mse_test(wb, net, inputs, targets, inindelst, inlaydelst)
% 'x' contains the weights and biases vector
% in row vector form as passed to it by the
% genetic algorithm. This must be transposed
% when being set as the weights and biases
% vector for the network.
% To set the weights and biases vector to the
% one given as input
wb = formwb(net,net.b,net.IW,net.LW);
net = setwb(net, wb');
% To evaluate the ouputs based on the given
% weights and biases vector
%y = net(inputs);
y = net(inputs,inindelst,inlaydelst);
target=cell2mat(targets);
target1=target(3:254208);
% Calculating the mean squared error
mse_calc = sum((cell2mat(y)-target1).^2)/length(y);
end
Greg Heath on 9 Oct 2016
mse(target-output)/mean(var(target',1)) = ?

MD NOUSHAD JAVED on 5 Oct 2017
Respected Sir, Want to implement GA tool for multi-objective optimization.
I have three different objectives Y1 , Y2 and Y3. All three are polynomial equations of independent variables x1, x2 and x3.
The problem is that I am finding a solution to maximized the Y1 while minimize Y3 , while Y2 are of moderate importance (no such preference).
I would be rally helpful of you if you help me to do same functions in matlab. How will I write script code and define such importance to each variables ?
I will be thankful of you.
Greg Heath on 25 Aug 2018
I would try maximizing something like
f = Y1 + 0.5*Y2 - Y3
Hope this helps.
Greg

Ahmed Ryad on 26 Oct 2018
You can change the number of variables in the ga function from (3*n+1) to the number of the weight and bias of your net where the number of weight and bias is (no) wb=getwb(net); no=length(wb);