Thread Subject:

difference between numInputs and neurons in inputlayer?

I am going to use
net = network(numInputs,numLayers,biasConnect,inputConnect,layerConnect,outputConnect)
to create a custom neural network.

but I dont understand, what is the meaning of numInputs, and the difference between numInputs and neurons in the input layer.

does the numlayers include all layers (input layer+hidden layer+output layer)?
any one can explain these?

"preben" wrote in message <jilr6s$8kc$1@newscl01ah.mathworks.com>...
> I am going to use
> net = network(numInputs,numLayers,biasConnect,inputConnect,layerConnect,outputConnect)
> to create a custom neural network.
>
> but I dont understand, what is the meaning of numInputs, and the difference between >numInputs and neurons in the input layer.
>
> does the numlayers include all layers (input layer+hidden layer+output layer)?
> any one can explain these?

There is a difference between layers of nodes and layers of weights. The term "layer"
 in most neural network literature (including MATLAB's "numlayers") refers to weight layers.

For a typical FFMLP there are 3 node layers (input,hidden,output) but only 2 weight layers (input-hidden and hidden-output).

MATLAB's use of "numinputs" and "numoutputs" are interpreted in the vector sense:
There is one vector input and one vector output

Hidden and output nodes are associated with activation functions aka artificial neurons
whereas the input nodes are associated with applied signals and are characterized as
"fan-in units". To be perfectly clear, there are no neurons in the input layer.

Example:

clear all, close all, clc
p = randn(3,100);
t = exp(-p).*cos(p);
[ I N ] = size(p) % [ 3 100]
[ O N ] = size(t) % [3 100]
Neq = N*O % 300 No. of training equations
Hub = floor((Neq-O)/(I+O+1)) % 42 Neq >= Nw Upper bound of H
H =round(Hub/10) % 4 Neq ~ 10*Nw (want Neq >> Nw)
Nw = (I+1)*H+(H+1)*O % 31
% I-H-O = 3-4-3
net = newff(p,t,H) % No semicolon to display characteristics.

% Now investigate the contents of the net's dimensions, connections,
% subobjects, weight and bias values.

Hope this helps.

Greg

"Greg Heath" <heath@alumni.brown.edu> wrote in message <jiq2ev$99v$1@newscl01ah.mathworks.com>...
> "preben" wrote in message <jilr6s$8kc$1@newscl01ah.mathworks.com>...
> > I am going to use
> > net = network(numInputs,numLayers,biasConnect,inputConnect,layerConnect,outputConnect)
> > to create a custom neural network.
> >
> > but I dont understand, what is the meaning of numInputs, and the difference between >numInputs and neurons in the input layer.
> >
> > does the numlayers include all layers (input layer+hidden layer+output layer)?
> > any one can explain these?
>
> There is a difference between layers of nodes and layers of weights. The term "layer"
> in most neural network literature (including MATLAB's "numlayers") refers to weight layers.
>
> For a typical FFMLP there are 3 node layers (input,hidden,output) but only 2 weight layers (input-hidden and hidden-output).
>
> MATLAB's use of "numinputs" and "numoutputs" are interpreted in the vector sense:
> There is one vector input and one vector output
>
> Hidden and output nodes are associated with activation functions aka artificial neurons
> whereas the input nodes are associated with applied signals and are characterized as
> "fan-in units". To be perfectly clear, there are no neurons in the input layer.
>
> Example:
>
> clear all, close all, clc
> p = randn(3,100);
> t = exp(-p).*cos(p);
> [ I N ] = size(p) % [ 3 100]
> [ O N ] = size(t) % [3 100]
> Neq = N*O % 300 No. of training equations
> Hub = floor((Neq-O)/(I+O+1)) % 42 Neq >= Nw Upper bound of H
> H =round(Hub/10) % 4 Neq ~ 10*Nw (want Neq >> Nw)
> Nw = (I+1)*H+(H+1)*O % 31
> % I-H-O = 3-4-3
> net = newff(p,t,H) % No semicolon to display characteristics.
>
> % Now investigate the contents of the net's dimensions, connections,
> % subobjects, weight and bias values.
>
> Hope this helps.
>
> Greg

thanks for your reply.
I understand now.
I have a similar question with one guy who asked several years ago as following

"I am trying to design a 6-4-1 network. The first three input nodes
(i.e 1-3) are connected with the first two (i.e 1-2) nodes in the
hidden layer, while the last 3 input nodes (i.e 4-6) are connected
fully with the last two nodes in the hidden layer (3-4). . All the
four hidden nodes are connected to the output node. There is no
connection between input nodes (1-3) and hidden nodes (3-4) so also
there is no connection between input nodes (4-6) and hidden nodes
(1-2)."

how should I set the parameters of network function?
net = network(numInputs,numLayers,biasConnect,inputConnect,layerConnect,outputConnect)

I have another question.
can I train the net using [net,TR] = trainlm(net,TR,trainV,valV,testV)?
if so, how should I initialize the parameter of TR?

thanks in advance

"preben" wrote in message <jiqr5p$qnv$1@newscl01ah.mathworks.com>...
> "Greg Heath" <heath@alumni.brown.edu> wrote in message <jiq2ev$99v$1@newscl01ah.mathworks.com>...
> > "preben" wrote in message <jilr6s$8kc$1@newscl01ah.mathworks.com>...
> > > I am going to use
> > > net = network(numInputs,numLayers,biasConnect,inputConnect,layerConnect,outputConnect)
> > > to create a custom neural network.
> > >
> > > but I dont understand, what is the meaning of numInputs, and the difference between >numInputs and neurons in the input layer.
> > >
> > > does the numlayers include all layers (input layer+hidden layer+output layer)?
> > > any one can explain these?
> >
> > There is a difference between layers of nodes and layers of weights. The term "layer"
> > in most neural network literature (including MATLAB's "numlayers") refers to weight layers.
> >
> > For a typical FFMLP there are 3 node layers (input,hidden,output) but only 2 weight layers (input-hidden and hidden-output).
> >
> > MATLAB's use of "numinputs" and "numoutputs" are interpreted in the vector sense:
> > There is one vector input and one vector output
> >
> > Hidden and output nodes are associated with activation functions aka artificial neurons
> > whereas the input nodes are associated with applied signals and are characterized as
> > "fan-in units". To be perfectly clear, there are no neurons in the input layer.
> >
> > Example:
> >
> > clear all, close all, clc
> > p = randn(3,100);
> > t = exp(-p).*cos(p);
> > [ I N ] = size(p) % [ 3 100]
> > [ O N ] = size(t) % [3 100]
> > Neq = N*O % 300 No. of training equations
> > Hub = floor((Neq-O)/(I+O+1)) % 42 Neq >= Nw Upper bound of H
> > H =round(Hub/10) % 4 Neq ~ 10*Nw (want Neq >> Nw)
> > Nw = (I+1)*H+(H+1)*O % 31
> > % I-H-O = 3-4-3
> > net = newff(p,t,H) % No semicolon to display characteristics.
> >
> > % Now investigate the contents of the net's dimensions, connections,
> > % subobjects, weight and bias values.
> >
> > Hope this helps.
> >
> > Greg
>
> thanks for your reply.
> I understand now.
> I have a similar question with one guy who asked several years ago as following
>
> "I am trying to design a 6-4-1 network. The first three input nodes
> (i.e 1-3) are connected with the first two (i.e 1-2) nodes in the
> hidden layer, while the last 3 input nodes (i.e 4-6) are connected
> fully with the last two nodes in the hidden layer (3-4). . All the
> four hidden nodes are connected to the output node. There is no
> connection between input nodes (1-3) and hidden nodes (3-4) so also
> there is no connection between input nodes (4-6) and hidden nodes
> (1-2)."
>
> how should I set the parameters of network function?
> net = network(numInputs,numLayers,biasConnect,inputConnect,layerConnect,outputConnect)

Not exactly sure. I always start with full connections. Then if needed, I
SEQUENTIALLY delete ineffective input nodes that are ranked last by the
decrease in performance when the inputs to that node are scrambled.

Can probably figure this out by looking at the properties of my 3-4-3
example.

Will respond later.

> I have another question.
> can I train the net using [net,TR] = trainlm(net,TR,trainV,valV,testV)?
> if so, how should I initialize the parameter of TR?
>
> thanks in advance

No. If you would read the documentation

help trainlm
doc trainlm

you will clearly see that trainlm is called by train which automatically initializes
all of the inputs.

Hope this helps.

Greg

Thanks Greg.
if I cannot use trainlm directly, is it possible to use different data to train the net? I mean, use different data for train, validation and test to get the performance (plotperform).

liu

"Greg Heath" <heath@alumni.brown.edu> wrote in message <jirm8m$ss0$1@newscl01ah.mathworks.com>...
> "preben" wrote in message <jiqr5p$qnv$1@newscl01ah.mathworks.com>...
> > "Greg Heath" <heath@alumni.brown.edu> wrote in message <jiq2ev$99v$1@newscl01ah.mathworks.com>...
> > > "preben" wrote in message <jilr6s$8kc$1@newscl01ah.mathworks.com>...
> > > > I am going to use
> > > > net = network(numInputs,numLayers,biasConnect,inputConnect,layerConnect,outputConnect)
> > > > to create a custom neural network.
> > > >
> > > > but I dont understand, what is the meaning of numInputs, and the difference between >numInputs and neurons in the input layer.
> > > >
> > > > does the numlayers include all layers (input layer+hidden layer+output layer)?
> > > > any one can explain these?
> > >
> > > There is a difference between layers of nodes and layers of weights. The term "layer"
> > > in most neural network literature (including MATLAB's "numlayers") refers to weight layers.
> > >
> > > For a typical FFMLP there are 3 node layers (input,hidden,output) but only 2 weight layers (input-hidden and hidden-output).
> > >
> > > MATLAB's use of "numinputs" and "numoutputs" are interpreted in the vector sense:
> > > There is one vector input and one vector output
> > >
> > > Hidden and output nodes are associated with activation functions aka artificial neurons
> > > whereas the input nodes are associated with applied signals and are characterized as
> > > "fan-in units". To be perfectly clear, there are no neurons in the input layer.
> > >
> > > Example:
> > >
> > > clear all, close all, clc
> > > p = randn(3,100);
> > > t = exp(-p).*cos(p);
> > > [ I N ] = size(p) % [ 3 100]
> > > [ O N ] = size(t) % [3 100]
> > > Neq = N*O % 300 No. of training equations
> > > Hub = floor((Neq-O)/(I+O+1)) % 42 Neq >= Nw Upper bound of H
> > > H =round(Hub/10) % 4 Neq ~ 10*Nw (want Neq >> Nw)
> > > Nw = (I+1)*H+(H+1)*O % 31
> > > % I-H-O = 3-4-3
> > > net = newff(p,t,H) % No semicolon to display characteristics.
> > >
> > > % Now investigate the contents of the net's dimensions, connections,
> > > % subobjects, weight and bias values.
> > >
> > > Hope this helps.
> > >
> > > Greg
> >
> > thanks for your reply.
> > I understand now.
> > I have a similar question with one guy who asked several years ago as following
> >
> > "I am trying to design a 6-4-1 network. The first three input nodes
> > (i.e 1-3) are connected with the first two (i.e 1-2) nodes in the
> > hidden layer, while the last 3 input nodes (i.e 4-6) are connected
> > fully with the last two nodes in the hidden layer (3-4). . All the
> > four hidden nodes are connected to the output node. There is no
> > connection between input nodes (1-3) and hidden nodes (3-4) so also
> > there is no connection between input nodes (4-6) and hidden nodes
> > (1-2)."
> >
> > how should I set the parameters of network function?
> > net = network(numInputs,numLayers,biasConnect,inputConnect,layerConnect,outputConnect)
>
> Not exactly sure. I always start with full connections. Then if needed, I
> SEQUENTIALLY delete ineffective input nodes that are ranked last by the
> decrease in performance when the inputs to that node are scrambled.
>
> Can probably figure this out by looking at the properties of my 3-4-3
> example.
>
> Will respond later.
>
> > I have another question.
> > can I train the net using [net,TR] = trainlm(net,TR,trainV,valV,testV)?
> > if so, how should I initialize the parameter of TR?
> >
> > thanks in advance
>
> No. If you would read the documentation
>
> help trainlm
> doc trainlm
>
> you will clearly see that trainlm is called by train which automatically initializes
> all of the inputs.
>
> Hope this helps.
>
> Greg

CORRECTED FOR THE HEINOUS SIN OF TOP-POSTING!

On Mar 5, 6:23 am, "preben " <lzs19971...@163.com> wrote:
> "Greg Heath" <he...@alumni.brown.edu> wrote in message <jirm8m$ss...@newscl01ah.mathworks.com>...
> > "preben" wrote in message <jiqr5p$qn...@newscl01ah.mathworks.com>...
> > > "Greg Heath" <he...@alumni.brown.edu> wrote in message <jiq2ev$99...@newscl01ah.mathworks.com>...
> > > > "preben" wrote in message <jilr6s$8k...@newscl01ah.mathworks.com>...
> > > > > I am going to use
> > > > > net = network(numInputs,numLayers,biasConnect,inputConnect,layerConnect,outputCon­nect)
> > > > > to create a custom neural network.
>
> > > > > but I dont understand, what is the meaning of numInputs, and the difference between >numInputs and neurons in the input layer.
>
> > > > > does the numlayers include all layers (input layer+hidden layer+output layer)?
> > > > > any one can explain these?
>
> > > > There is a difference between layers of nodes and layers of weights. The term "layer"
> > > > in most neural network literature (including MATLAB's "numlayers") refers to weight layers.
>
> > > > For a typical FFMLP there are 3 node layers (input,hidden,output) but only 2 weight layers (input-hidden and hidden-output).
>
> > > > MATLAB's use of "numinputs" and "numoutputs" are interpreted in the vector sense:
> > > > There is one vector input and one vector output
>
> > > > Hidden and output nodes are associated with activation functions aka artificial neurons
> > > > whereas the input nodes are associated with applied signals and are characterized as
> > > > "fan-in units". To be perfectly clear, there are no neurons in the input layer.
>
> > > > Example:
>
> > > > clear all, close all, clc
> > > > p = randn(3,100);
> > > > t = exp(-p).*cos(p);
> > > > [ I N ] = size(p) % [ 3 100]
> > > > [ O N ] = size(t) % [3 100]
> > > > Neq = N*O % 300 No. of training equations
> > > > Hub = floor((Neq-O)/(I+O+1)) % 42 Neq >= Nw Upper bound of H
> > > > H =round(Hub/10) % 4 Neq ~ 10*Nw (want Neq >> Nw)
> > > > Nw = (I+1)*H+(H+1)*O % 31
> > > > % I-H-O = 3-4-3
> > > > net = newff(p,t,H) % No semicolon to display characteristics.
>
> > > > % Now investigate the contents of the net's dimensions, connections,
> > > > % subobjects, weight and bias values.
>
>
> > > thanks for your reply.
> > > I understand now.
> > > I have a similar question with one guy who asked several years ago as following
>
> > > "I am trying to design a 6-4-1 network. The first three input nodes
> > > (i.e 1-3) are connected with the first two (i.e 1-2) nodes in the
> > > hidden layer, while the last 3 input nodes (i.e 4-6) are connected
> > > fully with the last two nodes in the hidden layer (3-4). . All the
> > > four hidden nodes are connected to the output node. There is no
> > > connection between input nodes (1-3) and hidden nodes (3-4) so also
> > > there is no connection between input nodes (4-6) and hidden nodes
> > > (1-2)."
>
> > > how should I set the parameters of network function?
> > > net = network(numInputs,numLayers,biasConnect,inputConnect,layerConnect,outputCon­nect)
>
> > Not exactly sure. I always start with full connections. Then if needed, I
> > SEQUENTIALLY delete ineffective input nodes that are ranked last by the
> > decrease in performance when the inputs to that node are scrambled.
>
> > Can probably figure this out by looking at the properties of my 3-4-3
> > example.
>
> > Will respond later.

I guess the only way to do this is to define 2 inputs and 3 weight
layers. The
first two weight layers are in parallel and each is connected to one
of the inputs..

> > > I have another question.
> > > can I train the net using [net,TR] = trainlm(net,TR,trainV,valV,testV)?
> > > if so, how should I initialize the parameter of TR?
>
> > No. If you would read the documentation
>
> > help trainlm
> > doc trainlm
>
> > you will clearly see that trainlm is called by train which automatically initializes
> > all of the inputs.
>
> Thanks Greg.
> if I cannot use trainlm directly,

Then, like everyone else use it indirectly via train.

> is it possible to use different data to train the net? I mean, use different data for train, validation and test to get the performance (plotperform).

Possible? That is the default: randomly selected with a 70/15/15
division ratio. See the documentation
for a different selection and/or ratio.

Hope this helps.

Greg