MATLAB Answers

How to obtain the relative importance of each input variable for a neural network?

134 views (last 30 days)
Jocelyn
Jocelyn on 15 Aug 2016
Commented: Reza Mazloum on 5 Oct 2021 at 12:00
Hi. I have build a regression neural network with 580 data points of 48 inputs and 5 outputs. The optimum network is 30 neurons for the first hidden layer and 17 neurons for the second hidden layer as shown in the figure below. I would like to know is there any methods to understand the relationship between inputs and outputs for the neural network (whether they have positive or negative significant relationship)? How to obtain the relative importance of each input to each output? Thanks.

Accepted Answer

Greg Heath
Greg Heath on 16 Aug 2016
The relative importance of an input variable depends on what other input variables are present.
For example, if you are not planning on removing unimportant variables, then you can rank inputs by the resulting performance when only that variable is replaced by its mean values and the net starts training from that point.
Hope this helps.
Greg
  3 Comments
Emran Alotaibi
Emran Alotaibi on 13 Sep 2021
Hey Greg!
I compeletly agree with you, However, could you provide any scientific paper with this procedure so it can be cited in our work? Regards

Sign in to comment.

More Answers (3)

Yago Veloso
Yago Veloso on 18 Jan 2017
Edited: Yago Veloso on 18 Jan 2017
You can use the relative importance method using the equation above, where Rij is the relative importance of the variable xi with respect to the output neuron j, H is the nunber of neurons in the hidden layer, Wik is the synaptic connection weight between the input neuron i and the hidden neuron k, and Wkj is the synaptic weight between the hidden neuron k and the output neuron j.
If it's not very clear for you take a look at this paper here: http://www.palisade.com/downloads/pdf/academic/DTSpaper110915.pdf
Good look! Regards Yago
  1 Comment
F S
F S on 23 Oct 2017
Hi Yago, this solution you present above applies to a Neural Network with only one Hidden Layer, right? And it doesn't really give the relative importance, in respect to the other inputs, does it? How does it deal with negative weights?

Sign in to comment.



Iman Jafari
Iman Jafari on 8 Sep 2021
Edited: Iman Jafari on 11 Sep 2021
The relative importance of input variables on selected outputs can be estimated using following equation and code. Where Ej, W and N are the relative importance of the jth input variable on, weighted coefficient, and the number of neurons, respectively. The subscripts ‘k’, ‘t’ and ‘n’ refer to input, hidden and output neurons, respectively and the superscripts ‘i’, ‘h’ and ‘o’ refer to input, hidden and output layers, respectively.
W1= net.IW{1,1}; % Weights between input and hidden layers
W2= net.LW{2,1}'; % Weights between hidden and output layers
k= size(W1,2); % number of input varables
g=1; % Select which output you are going to evaluate the relative importance of
% input variables on (No need to change if you have only one output)
for n=1:k
for i=1:k
a(:,i)=(abs(W1(:,n))./sum(abs(W1(:,1:i)),2)).*abs(W2(:,g));
end
b=sum(sum(a,2));
I(n,1)=sum(abs(W1(:,n))./sum(abs(W1),2))/b;
end
for i=1:k
R(i,1)=(I(i)/sum(I))*100; % Percentage
end
  1 Comment
Reza Mazloum
Reza Mazloum on 5 Oct 2021 at 12:00
Hi Iman, Tnx for this script. Just have a question. I used it for different dataset and don't understand why the first predictors for all outputs (classes) have the biggest value?
Cheers,
Reza

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!