MATLAB Answers

Optimization of hidden layer dimension in neural network (number of neurons)

40 views (last 30 days)
David Franco
David Franco on 14 Dec 2017
Edited: David Franco on 28 Dec 2017
I am using GA optimization to train my neural network (to try to find the best weights set):
% Neural Network trained by Genetic Algorithm (GA)
% Reference: http://www.mathworks.com/matlabcentral/answers/100323
% Thx Greg!
rng('default')
[x, t] = simplefit_dataset;
[I, ~] = size(x);
[O, N] = size(t);
% Reference MSE: Average Target Variance
var_t = mean(var(t,1,2));
% Hidden Node Choice
H = 4;
Nw = (I + 1)*H + (H + 1)*O;
% Create Regression/Curve-Fitting Neural Network:
net = feedforwardnet(H);
net.divideFcn = 'dividetrain';
% Configure the Net for the Simplefit Dataset
net = configure(net, x, t);
% Initial Weights and Errors
wb = getwb(net)';
% Create handle to the error function,
fun = @(wb) nmse(wb, net, x, t); % NMSE
% Set the Genetic Algorithm tolerance for minimum change in fitness function
% before terminating algorithm to 1e-4 and display each iteration's results.
opts = optimoptions('ga', 'FunctionTolerance', 1e-4, 'Display', 'iter');
tic
[wbopt, fval] = ga(fun, Nw, opts);
totaltime = toc;
% Assign the weights to net
net = setwb(net, wbopt');
% Simulate the output
y = sim(net,x);
Where the function nmse is:
function nmse_calc = nmse(wb, net, input, target)
% 'wb' contains the weights and biases vector
% in row vector form as passed to it by the
% genetic algorithm. This must be transposed
% when being set as the weights and biases
% vector for the network.
% Reference MSE
var_t = mean(var(target,1,2));
% To set the weights and biases vector to the
% one given as input
net = setwb(net, wb');
% To evaluate the ouputs based on the given
% weights and biases vector
y = net(input);
% Calculating the Normalised Mean Squared Error (NMSE)
nmse_calc = mean((target(:) - y(:)).^2) / var_t;
end
My question is:
Is there any way to optimize the number of hidden nodes H in the same way? Maybe with multiobjective GA optimization... optimizing the weights set and the number of hidden nodes at once.
Thank you very much!

  0 Comments

Sign in to comment.

Answers (1)

Greg Heath
Greg Heath on 20 Dec 2017
Edited: Greg Heath on 20 Dec 2017
If you search the NEWSGROUP and ANSWERS using
greg genetic
you should conclude that I don't recommend GA for NN design.
Nevertheless, if you insist, I recommend a double loop design:
j =0
for h = Hmin:dH:Hmax
j=j+1
standard stuff
for i = 1:Ntrials
GA
end
end
Hope this helps,
Thank you for formally accepting my answer
Greg

  1 Comment

David Franco
David Franco on 28 Dec 2017
But in this way, there is no optimization of the number of Hidden nodes through a global optimization technique, just an exhaustive check of all Hidden nodes possibilities.
The number o Hidden nodes must be optimized too (as an input for GA, for instance)...

Sign in to comment.

Sign in to answer this question.