Got Questions? Get Answers.
Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Manual Pruning of Weights in Feedforward Neural Network

Subject: Manual Pruning of Weights in Feedforward Neural Network

From: Titus Adelani

Date: 24 Feb, 2009 19:53:01

Message: 1 of 3

I am using neural network toolbox to approximate the relationship between some input and output samples. I need to manually exclude some input weights from my feedforward neural network. By this I want to severe connection between some inputs and hidden layer nodes to indicate that both (and consequently the output) do not have relationship. Can anyone how help me with the procedure and/or commands to do this in MATLAB?

Thank you.

Subject: Manual Pruning of Weights in Feedforward Neural Network

From: Greg Heath

Date: 2 Mar, 2009 17:07:33

Message: 2 of 3

On Feb 24, 2:53=A0pm, "Titus Adelani" <umade...@cc.umanitoba.ca> wrote:
> I am usingneuralnetwork toolbox to approximate the relationship between s=
ome input and output samples. I need to manually exclude some input weights=
 from my feedforwardneuralnetwork. By this I want to severe connection betw=
een some inputs and hidden layer nodes to indicate that both (and consequen=
tly the output) do not have relationship. Can anyone how help me with the p=
rocedure and/or commands to do this in MATLAB?
>
> Thank you.

Subject: Manual Pruning of Weights in Feedforward Neural Network

From: Greg Heath

Date: 4 Mar, 2009 04:56:05

Message: 3 of 3

On Mar 2, 12:07 pm, Greg Heath <he...@alumni.brown.edu> wrote:
> On Feb 24, 2:53 pm, "Titus Adelani" <umade...@cc.umanitoba.ca> wrote:
>
> > I am using neural network toolbox to approximate the relationship
> between some input and output samples. I need to manually exclude
> some input weights from my feed forward neural network.

WHY??

> By this I want to severe connection between some inputs and hidden
> layer nodes to indicate that both (and consequently the output) do not
> have relationship. Can anyone how help me with the procedure and/or
> commands to do this in MATLAB?

WHOOPS! Google Groups ate my response last night.

I'll try again:

1. Don't know if MATLAB allows fixing weight values.
2. One approach is to use batch updating for one epoch.
Then reset the selected weights to the desired fixed value
before training for the next epoch.
3. HOWEVER, you CANNOT achieve the above goal
because
    a. Each output depends on all H hidden node values
    b. Each hidden node value depends on all input values.
4. A measure of the strength of the relationship between
input i and output k is sum(j=1,H){W(k,j)*W(j,i)} for a MLP.
Weak relationships can be caused by low W values. However,
since the other outputs also depend on the same W(j,i) values,
it is more likely that weak dependence is the result of
algebraic cancellation.
5. I can't think of a similar measure for the RBF. However,
the nature of the difficulty is the same.

My advice is not to worry about it. A correctly trained
MLP or RBF is a universal approximator. Therefore,
post training, you can test for insignificant relationships
between any input and any output. Of course, linear
relationships tend to be easily detected, before training,
via the correlation coefficient matrix. However, the difficulty
increases as the inputs become more correlated.

Hope this helps.

Greg

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us