LevenbergMarquardt backpropagation
net.trainFcn = 'trainlm'
[net,tr] = train(net,...)
trainlm
is a network training function that updates weight and bias
values according to LevenbergMarquardt optimization.
trainlm
is often the fastest backpropagation algorithm in the toolbox,
and is highly recommended as a firstchoice supervised algorithm, although it does require more
memory than other algorithms.
net.trainFcn = 'trainlm'
sets the network trainFcn
property.
[net,tr] = train(net,...)
trains the network with
trainlm
.
Training occurs according to trainlm
training parameters, shown here
with their default values:
net.trainParam.epochs  1000 
Maximum number of epochs to train 
net.trainParam.goal  0 
Performance goal 
net.trainParam.max_fail  6 
Maximum validation failures 
net.trainParam.min_grad  1e7 
Minimum performance gradient 
net.trainParam.mu  0.001 
Initial 
net.trainParam.mu_dec  0.1 

net.trainParam.mu_inc  10 

net.trainParam.mu_max  1e10 
Maximum 
net.trainParam.show  25 
Epochs between displays ( 
net.trainParam.showCommandLine  false 
Generate commandline output 
net.trainParam.showWindow  true 
Show training GUI 
net.trainParam.time  inf 
Maximum time to train in seconds 
Validation vectors are used to stop training early if the network performance on the
validation vectors fails to improve or remains the same for max_fail
epochs
in a row. Test vectors are used as a further check that the network is generalizing well, but do
not have any effect on training.
trainlm
is the default training function for several network creation
functions including newcf
, newdtdnn
,
newff
, and newnarx
.
You can create a standard network that uses trainlm
with
feedforwardnet
or cascadeforwardnet
.
To prepare a custom network to be trained with trainlm
,
Set net.trainFcn
to 'trainlm'
.
This sets net.trainParam
to trainlm
’s default
parameters.
Set net.trainParam
properties to desired
values.
In either case, calling train
with the resulting network trains the
network with trainlm
.
See help feedforwardnet
and help cascadeforwardnet
for examples.
This function uses the Jacobian for calculations, which assumes that performance is a mean
or sum of squared errors. Therefore, networks trained with this function must use either the
mse
or sse
performance function.
trainlm
supports training with validation and test vectors if the
network’s NET.divideFcn
property is set to a data division function.
Validation vectors are used to stop training early if the network performance on the validation
vectors fails to improve or remains the same for max_fail
epochs in a row.
Test vectors are used as a further check that the network is generalizing well, but do not have
any effect on training.
trainlm
can train any network as long as its weight, net input, and
transfer functions have derivative functions.
Backpropagation is used to calculate the Jacobian jX
of performance
perf
with respect to the weight and bias variables X
.
Each variable is adjusted according to LevenbergMarquardt,
jj = jX * jX je = jX * E dX = (jj+I*mu) \ je
where E
is all errors and I
is the identity
matrix.
The adaptive value mu
is increased by mu_inc
until
the change above results in a reduced performance value. The change is then made to the network
and mu
is decreased by mu_dec
.
Training stops when any of these conditions occurs:
The maximum number of epochs
(repetitions) is reached.
The maximum amount of time
is exceeded.
Performance is minimized to the goal
.
The performance gradient falls below min_grad
.
mu
exceeds mu_max
.
Validation performance has increased more than max_fail
times since
the last time it decreased (when using validation).