Clear Filters
Clear Filters

Marquardt adjustment parameter (trainbr)

6 views (last 30 days)
In "trainbr" of neural network toolbox, the following lines have been used
% net.trainParam.mu 0.005 Marquardt adjustment parameter % net.trainParam.mu_dec 0.1 Decrease factor for mu % net.trainParam.mu_inc 10 Increase factor for mu % net.trainParam.mu_max 1e-10 Maximum value for mu
Could you please explain the meaning of "mu" and "mu_max"? what is their significance here?
thank you

Accepted Answer

Greg Heath
Greg Heath on 31 Jul 2013

More Answers (2)

Giovanni
Giovanni on 30 Jul 2013
Hi Abul, the Levenberg-Marquardt algorithm uses an approximation to the Hessian matrix (using jacobian, so is not necessary to calculate the 2nd derivatives, only the 1rst, easier from computational point of view)
Backpropagation is used to calculate the Jacobian jX of performance perf with respect to the weight and bias variables X. Each variable is adjusted according to Levenberg-Marquardt,
jj = jX * jX je = jX * E dX = -(jj+I*mu) \ je where E is all errors and I is the identity matrix.
The adaptive value mu is increased by mu_inc until the change above results in a reduced performance value. The change is then made to the network and mu is decreased by mu_dec.
The parameter mem_reduc indicates how to use memory and speed to calculate the Jacobian jX. If mem_reduc is 1, then trainlm runs the fastest, but can require a lot of memory. Increasing mem_reduc to 2 cuts some of the memory required by a factor of two, but slows trainlm somewhat. Higher states continue to decrease the amount of memory needed and increase training times.
Training stops when any of these conditions occurs:
The maximum number of epochs (repetitions) is reached.
The maximum amount of time is exceeded.
Performance is minimized to the goal.
The performance gradient falls below min_grad.
mu exceeds mu_max.
Validation performance has increased more than max_fail times since the last time it decreased (when using validation).
  1 Comment
Abul Fujail
Abul Fujail on 25 Oct 2013
Thank you Giovanni ... I want to know what is "mu" ?

Sign in to comment.


Luis
Luis on 7 Apr 2015
Edited: Luis on 7 Apr 2015
Hi, anybody know how many percent use "trainbr" for training and how many for test, how change these percentages ??

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!