change learning rate trainlm
7 views (last 30 days)
Show older comments
How to change learning rate in trainlm? matlab allows changing mu and Gradient
tnx
1 Comment
Phuong Huynh
on 26 May 2022
I also wonder about this problem. I think trainlm function (optimizer) auto choose LR for us (http://matlab.izmiran.ru/help/toolbox/nnet/trainlm.html). Because other function allows us to change LR (for example :traingdx) when we change parameter net.trainParam.lr
Answers (1)
Krishna
on 25 Aug 2024
Hello,
From what I understand you want to change the learning rate of ‘trainlm’ optimiser. If you go through the documentation of ‘trainlm’ function you can see that there is no way to set a learning rate of this optimiser, but you can control the learning dynamics via mu, the adaptive parameter that adjusts the learning process in Levenberg-Marquardt as it acts similarly to the learning rate. A smaller mu allows smaller steps during learning, which can lead to finer adjustments but slower convergence. A larger mu takes bigger steps, which may speed up convergence but risks overshooting.
Set the hyper-paramter ‘net.trainParam.mu_dec’ to specify how much to decrease ‘mu’ when the error decreases after an iteration. Also set parameter ‘net.trainParam.mu_inc’ how much to increase ‘mu’ when the error increase after an iteration.
Please go through the following documentation to learn more regarding the hyper-parameters of Levenberg-Marquardt algorithm and also the mathematical update function being used in the following algorithm,
Also please go through the following documentation to learn more about how to ask question on MATLAB answer and get a fast response,
Hope this helps.
0 Comments
See Also
Categories
Find more on Deep Learning Toolbox in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!