MATLAB Answers

Avoid training certain neurons

20 views (last 30 days)
Hamid Moazed
Hamid Moazed on 22 Dec 2019
Commented: Hamid Moazed on 23 Dec 2019
Using the Deep Learning Toolbox, I wish to construct a simple feed-forward network for a simulation, however assume I have already trained one of the hidden neurons (out of several) with the correct weights and biases and I don't want them to change during training. How can I make this single specific neuron be "constant" and not get retrained with new wights and biases while the rest of the network is being trained?

Accepted Answer

Hiro
Hiro on 23 Dec 2019
There is an option to keep specific layers' learning rates low so you can fix them as they are.
for example
fullyConnectedLayer(<outputsize>, 'WeightLearnRateFactor', 0, 'BiasLearnRateFactor', 0)
This way, you would multiply zero to the global learning rate, which is set via trainingOptions function, and thus the learning rates of the weights in the fully-connected-layer are set as zero.
  1 Comment
Hamid Moazed
Hamid Moazed on 23 Dec 2019
Thanks for the quick and thorough response; yes, that makes total sense, why didn't I think of that! Thans again for getting me unstuck.

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!