MATLAB Answers

Optimization of hidden layer dimension in neural network (number of neurons)

47 views (last 30 days)
David Franco
David Franco on 14 Dec 2017
Edited: PIYUSHA GARG on 12 Jun 2020
I am using GA optimization to train my neural network (to try to find the best weights set):
% Neural Network trained by Genetic Algorithm (GA)
% Reference:
% Thx Greg!
[x, t] = simplefit_dataset;
[I, ~] = size(x);
[O, N] = size(t);
% Reference MSE: Average Target Variance
var_t = mean(var(t,1,2));
% Hidden Node Choice
H = 4;
Nw = (I + 1)*H + (H + 1)*O;
% Create Regression/Curve-Fitting Neural Network:
net = feedforwardnet(H);
net.divideFcn = 'dividetrain';
% Configure the Net for the Simplefit Dataset
net = configure(net, x, t);
% Initial Weights and Errors
wb = getwb(net)';
% Create handle to the error function,
fun = @(wb) nmse(wb, net, x, t); % NMSE
% Set the Genetic Algorithm tolerance for minimum change in fitness function
% before terminating algorithm to 1e-4 and display each iteration's results.
opts = optimoptions('ga', 'FunctionTolerance', 1e-4, 'Display', 'iter');
[wbopt, fval] = ga(fun, Nw, opts);
totaltime = toc;
% Assign the weights to net
net = setwb(net, wbopt');
% Simulate the output
y = sim(net,x);
Where the function nmse is:
function nmse_calc = nmse(wb, net, input, target)
% 'wb' contains the weights and biases vector
% in row vector form as passed to it by the
% genetic algorithm. This must be transposed
% when being set as the weights and biases
% vector for the network.
% Reference MSE
var_t = mean(var(target,1,2));
% To set the weights and biases vector to the
% one given as input
net = setwb(net, wb');
% To evaluate the ouputs based on the given
% weights and biases vector
y = net(input);
% Calculating the Normalised Mean Squared Error (NMSE)
nmse_calc = mean((target(:) - y(:)).^2) / var_t;
My question is:
Is there any way to optimize the number of hidden nodes H in the same way? Maybe with multiobjective GA optimization... optimizing the weights set and the number of hidden nodes at once.
Thank you very much!

Accepted Answer

Greg Heath
Greg Heath on 20 Dec 2017
Edited: Greg Heath on 20 Dec 2017
If you search the NEWSGROUP and ANSWERS using
greg genetic
you should conclude that I don't recommend GA for NN design.
Nevertheless, if you insist, I recommend a double loop design:
j =0
for h = Hmin:dH:Hmax
standard stuff
for i = 1:Ntrials
Hope this helps,
Thank you for formally accepting my answer

  1 Comment

David Franco
David Franco on 28 Dec 2017
But in this way, there is no optimization of the number of Hidden nodes through a global optimization technique, just an exhaustive check of all Hidden nodes possibilities.
The number o Hidden nodes must be optimized too (as an input for GA, for instance)...

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!