MATLAB Answers

How Can I Modify Weights in a SeriesNetwork of a CNN Model ?

94 views (last 30 days)
Reza Akbari
Reza Akbari on 20 Feb 2018
Commented: Ali Al-Saegh on 10 Mar 2021
Hi, Everyone. I have run a CNN example of Matlab R2017b to classify images of digits 0-9 and the code of this example is shown below :
if true
% Loading Data and Splitting them to the Training and Testing Data Set
digitDatasetPath = fullfile(matlabroot,'toolbox','nnet','nndemos',...
'nndatasets','DigitDataset');
digitData = imageDatastore(digitDatasetPath,...
'IncludeSubfolders',true,'LabelSource','foldernames');
trainingNumFiles = 750;
rng(1) % For reproducibility
[trainDigitData,testDigitData] = splitEachLabel(digitData,...
trainingNumFiles,'randomize');
% Defining Layers
layers = [imageInputLayer([28 28 1]);
convolution2dLayer(5,20);
reluLayer();
maxPooling2dLayer(2,'Stride',2);
fullyConnectedLayer(10);
softmaxLayer();
classificationLayer()];
% Training the convnet
options = trainingOptions('sgdm','MaxEpochs',20,...
'InitialLearnRate',0.0001);
convnet = trainNetwork(trainDigitData,layers,options);
end
Actually, I want to modify weights of this Network, for example, I would like to Multiply weights of first convolutional layer by 0.5, but I have received this error:
"You cannot set the read-only property 'Layers' of SeriesNetwork"
Is there any solution for this problem?
Thank you for advising me on this topic.
Reza.

Answers (4)

Peter Gadfort
Peter Gadfort on 8 Jun 2018
If you save the network to a struct, you can edit it there and then load it back into the network (this works in 2018a, I have not tested in any other version yet)
tmp_net = convnet.saveobj;
tmp_net.Layers(2).Weights = 0.5 * tmp_net.Layers(2).Weights;
convnet = convnet.loadobj(tmp_net);
classify(convnet, testDigitData);
This will allow you to edit the Weights and Biases without retraining the network.
  1 Comment
EREZ MANOR
EREZ MANOR on 4 Jul 2018
This option of "saveobj" works, and seems as the best solution. Thank you.

Sign in to comment.


Carlo Tomasi
Carlo Tomasi on 21 Feb 2018
While the Layers property of a SeriesNetwork is read-only, the weights and biases of a net.cnn.layer.Layer can be set at will. Thus, one way to address your problem is to copy all the weights from convnet.Layers to layers, modify the weights of layers as you wish, and then make a new convnet with trainNetwork. For instance:
for l = 1:length(layers)
if isprop(layers(l), 'Weights') % Does layer l have weights?
layers(l).Weights = convnet.Layers(l).Weights;
end
if isprop(layers(l), 'Bias')% Does layer l have biases?
layers(l).Bias = convnet.Layers(l).Bias;
end
end
layers(2).Weights = 0.5 * layers(2).Weights; % Modify the weights in the second layer
convnet = trainNetwork(trainDigitData,layers,options); % Make a new network and keep training
Of course, this code assumes that the structures of layers and convnet.Layers match. I hope this helps.
  4 Comments
Theron FARRELL
Theron FARRELL on 3 Dec 2019
I think that Kirill may set WeightLearnRateFactor and BiasLearnRateFactor of corresponding layers 0 so that parameters will not be updated through training.
As for Jan Jaap van Assen 's issue, I suppoe that you may try a workaround by adjusting the traning epochs to a point at which you wish to manipulate those paramters, save and reload your network at that point, specify parameters manually, for example
% W and b can be user-specified
layer = convolution2dLayer(filterSize,numFilters, ...
'Weights',W, ...
'Bias',b)
then start training again...

Sign in to comment.


Jan Jaap van Assen
Jan Jaap van Assen on 23 Apr 2018
I was a bit annoyed by this very basic restriction where we can't manually set weights. Using Carlo Tomasi's answer as inspiration I wrote a function that allows you to replace weights for layers of DAGNetworks. Sadly, it is required to call trainNetwork, which massively slows down the process. It only changes the weights you specify, all the other weights stay unchanged in the new net object.
For example, to set the weights of layer 5 all to zero:
newNet = replaceWeights(oldNet,5,zeros(size(oldNet.Layers(5).Weights)));
See the code below, I also attached the function file. I hope somebody can correct me with faster solutions.
function newNet = replaceWeights(oldNet,layerID,newWeights)
%REPLACEWEIGHTS Replace layer weights of DAGNetwork
% newNet = replaceWeights(oldNet,layerID,newWeights)
% oldNet = the DAGnetwork you want to replace weights.
% layerID = the layer number of which you want to replace the weights.
% newWeights = the matrix with the replacement weights. This should be
% the original weights size.
% Split up layers and connections
oldLgraph = layerGraph(oldNet);
layers = oldLgraph.Layers;
connections = oldLgraph.Connections;
% Set new weights
layers(layerID).Weights = newWeights;
% Freeze weights, from the Matlab transfer learning example
for ii = 1:size(layers,1)
props = properties(layers(ii));
for p = 1:numel(props)
propName = props{p};
if ~isempty(regexp(propName, 'LearnRateFactor$', 'once'))
layers(ii).(propName) = 0;
end
end
end
% Build new lgraph, from the Matlab transfer learning example
newLgraph = layerGraph();
for i = 1:numel(layers)
newLgraph = addLayers(newLgraph,layers(i));
end
for c = 1:size(connections,1)
newLgraph = connectLayers(newLgraph,connections.Source{c},connections.Destination{c});
end
% Very basic options
options = trainingOptions('sgdm','MaxEpochs', 1);
% Note that you might need to change the label here depending on your
% network in my case '1' is a valid label.
newNet = trainNetwork(zeros(oldNet.Layers(1).InputSize),1,newLgraph,options);
end

Hakan
Hakan on 1 Dec 2019
I would like ask another question. I am thinking it is related to this subject. I would like to make int8 Quantization to the weights. I read this article:
I used Carlo Tomasi method (one way to address your problem is to copy all the weights from convnet.Layers to layers, modify the weights of layers as you wish) And used fi function to make the 8 bit Quantization.
and then tried to make changes as fallows:
layers(2).Weights(2) = fi(layers(2).Weights(2),1,8);
I tried just for one weight to see if it changes from single to int8. The simple code above works with no error but the weight type is still single. Is there any way to make the wieght int8?
  3 Comments
Ali Al-Saegh
Ali Al-Saegh on 10 Mar 2021
Hello David,
The purpose of using int8 quantization method is to reduce the memory requirement of the neural network, which can be up to 75% of its original size. However, the obtained network after quantization shows the same size as the original one. If the memory size is not reduced and more time is required for the inference process, so what is the benefit of quantization?
I really appreciate your help in giving clarification for that.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!