Thread Subject: Training NN with outputs not with vectors but matrices

Hello,

I would appreciate if anyone could give me an answer to this problem. I was wondering if a MLP neural network could be trained giving at the output layer not a vector (e.g. 4x1, which means 4 output nodes) but a matrix (e.g. 4x4). If this is possible, how many output nodes should I declare at the training of the network. For example I want the output of the network to be four pairs of x,y, namely, x1,y1 - x2,y2 - x3,y3 - x4,y4. I should note that x,y are independent variables.

Thank you in advance,

Takis

On Nov 9, 8:23 am, "Takis Pan" <k_g...@hotmail.com> wrote:
> Hello,
>
> I would appreciate if anyone could give me an answer to this problem. I was wondering if a MLP neural network could be trained giving at the output layer not a vector (e.g. 4x1, which means 4 output nodes) but a matrix (e.g. 4x4). If this is possible, how many output nodes should I declare at the training of the network. For example I want the output of the network to be four pairs of x,y, namely, x1,y1 - x2,y2 - x3,y3 - x4,y4. I should note that x,y are independent variables.

Your question is not clearly stated. So I will just state:

There is one output node for each output signal. If your
output, y, is a time series of an mXn matrix for N observation
times and you want to predict the signal for time t(N+1).

Unfold the training targets if size(t) = [m n N], use the
matrix unfolding operator (:) to reduce the matrices to
vectors then size(t1) = [p N] for p = m*n and the rest
folloes as usual.

Therefore the net output will have p = m*n nodes and
the matrix structure of the output y can be recovered
from y1 by using the reshape function.

Hope this helps.

Greg

Dear Greg, thank you for your quick response. Sorry if my question was misleading. Let me try be more precise about what I need.

Generally, a MLP has an output layer with N nodes, with each node correspond to a single value. Thus, one can represent these values with a vector, let say T. If the corresponding input vector is P, then the training pairs are [P,T] and the training could be:
net=newff(minmax(P),[30,N],{'tansig','purelin'},'trainlm');
net=init(net);
[pn,minp,maxp,tn,mint,maxt] = premnmx(P,T);
[net,tr]=train(net,pn,tn);
where 30 is the number of hidden layer's nodes and N the number of output nodes.

Now, I do not have the case of a time series. I want to examine the case where each output node gives a point in the xy plane, corresponding to a coordinate (e.g. x1, y1). How could I train a NN, given the corresponding input vector P, to find N such points (xn, yn), namely N pairs of x,y? What number of output nodes should I give in the line:
net=newff(minmax(P),[30,?],{'tansig','purelin'},'trainlm'); ? The output would be a matrix (Nx2), and if so is it possible to use as an output a matrix instead of a vector?

Probably the information you gave about unfolding the targets is useful, but I am not familiar with this procedure, and I am not sure I comprehend it properly.

Thank you for your patience and I hope I made myself more clear this time.

Takis


Greg Heath <heath@alumni.brown.edu> wrote in message <30f3ffdd-9a01-4897-b605-37d1b9544fd4@p23g2000vbl.googlegroups.com>...
> On Nov 9, 8:23?am, "Takis Pan" <k_g...@hotmail.com> wrote:
> > Hello,
> >
> > I would appreciate if anyone could give me an answer to this problem. I was wondering if a MLP neural network could be trained giving at the output layer not a vector (e.g. 4x1, which means 4 output nodes) but a matrix (e.g. 4x4). If this is possible, how many output nodes should I declare at the training of the network. For example I want the output of the network to be four pairs of x,y, namely, x1,y1 - x2,y2 - x3,y3 - x4,y4. I should note that x,y are independent variables.
>
> Your question is not clearly stated. So I will just state:
>
> There is one output node for each output signal. If your
> output, y, is a time series of an mXn matrix for N observation
> times and you want to predict the signal for time t(N+1).
>
> Unfold the training targets if size(t) = [m n N], use the
> matrix unfolding operator (:) to reduce the matrices to
> vectors then size(t1) = [p N] for p = m*n and the rest
> folloes as usual.
>
> Therefore the net output will have p = m*n nodes and
> the matrix structure of the output y can be recovered
> from y1 by using the reshape function.
>
> Hope this helps.
>
> Greg

PLEASE DO NOT TOP POST. PLACE YOUR REPLY AFTER OR
INTERSPERSED WITHIN THE PREVIOUS POST

On Nov 9, 2:33 pm, "Takis Pan" <k_g...@hotmail.com> wrote:
> Dear Greg, thank you for your quick response. Sorry if my question was misleading. Let me try be more precise about what I need.
>
> Generally, a MLP has an output layer with N nodes, with each node correspond to a single value. Thus, one can represent these values with a vector, let say T. If the corresponding input vector is P, then the training pairs are [P,T] and the training could be:
> net=newff(minmax(P),[30,N],{'tansig','purelin'},'trainlm');
> net=init(net);
> [pn,minp,maxp,tn,mint,maxt] = premnmx(P,T);
> [net,tr]=train(net,pn,tn);
> where 30 is the number of hidden layer's nodes and N the number of output nodes.
>
> Now, I do not have the case of a time series. I want to examine the case where each output node gives a point in the xy plane, corresponding to a coordinate (e.g. x1, y1). How could I train a NN, given the corresponding input vector P, to find N such points (xn, yn), namely N pairs of x,y? What number of output nodes should I give in the line:
> net=newff(minmax(P),[30,?],{'tansig','purelin'},'trainlm'); ?

size(t,1) = 2*N

>The output would be a matrix (Nx2),

No. The output for a single input must be a vector of size [2*N 1]

and if so is it possible to use as an output a matrix instead of a
vector?

No. Unfold the matrix into a vector using (:).

> Probably the information you gave about unfolding the targets is useful, but I am not familiar with this procedure, and I am not sure I comprehend it properly.

A = rand(2,3)
B = A(:)

Hope this helps.

Greg

> Greg Heath <he...@alumni.brown.edu> wrote in message <30f3ffdd-9a01-4897-b605-37d1b9544...@p23g2000vbl.googlegroups.com>...
> > On Nov 9, 8:23?am, "Takis Pan" <k_g...@hotmail.com> wrote:
> > > Hello,
>
> > > I would appreciate if anyone could give me an answer to this problem. I was wondering if a MLP neural network could be trained giving at the output layer not a vector (e.g. 4x1, which means 4 output nodes) but a matrix (e.g. 4x4). If this is possible, how many output nodes should I declare at the training of the network. For example I want the output of the network to be four pairs of x,y, namely, x1,y1 - x2,y2 - x3,y3 - x4,y4. I should note that x,y are independent variables.
>
> > Your question is not clearly stated. So I will just state:
>
> > There is one output node for each output signal. If your
> > output, y, is a time series of an mXn matrix for N observation
> > times and you want to predict the signal for time t(N+1).
>
> > Unfold the training targets if size(t) = [m n N], use the
> > matrix unfolding operator (:) to reduce the matrices to
> > vectors then size(t1) = [p N] for p = m*n and the rest
> > folloes as usual.
>
> > Therefore the net output will have p = m*n nodes and
> > the matrix structure of the output y can be recovered
> > from y1 by using the reshape function.
>
> > Hope this helps.
>
> > Greg- Hide quoted text -
>
> - Show quoted text -

Greg Heath <heath@alumni.brown.edu> wrote in message <a8d9417e-3150-40c9-8ec2-9de855c65ad6@l35g2000vba.googlegroups.com>...
> PLEASE DO NOT TOP POST. PLACE YOUR REPLY AFTER OR
> INTERSPERSED WITHIN THE PREVIOUS POST
>
> On Nov 9, 2:33?pm, "Takis Pan" <k_g...@hotmail.com> wrote:
> > Dear Greg, thank you for your quick response. Sorry if my question was misleading. Let me try be more precise about what I need.
> >
> > Generally, a MLP has an output layer with N nodes, with each node correspond to a single value. Thus, one can represent these values with a vector, let say T. If the corresponding input vector is P, then the training pairs are [P,T] and the training could be:
> > net=newff(minmax(P),[30,N],{'tansig','purelin'},'trainlm');
> > net=init(net);
> > [pn,minp,maxp,tn,mint,maxt] = premnmx(P,T);
> > [net,tr]=train(net,pn,tn);
> > where 30 is the number of hidden layer's nodes and N the number of output nodes.
> >
> > Now, I do not have the case of a time series. I want to examine the case where each output node gives a point in the xy plane, corresponding to a coordinate (e.g. x1, y1). How could I train a NN, given the corresponding input vector P, to find N such points (xn, yn), namely N pairs of x,y? What number of output nodes should I give in the line:
> > net=newff(minmax(P),[30,?],{'tansig','purelin'},'trainlm'); ?
>
> size(t,1) = 2*N
>
> >The output would be a matrix (Nx2),
>
> No. The output for a single input must be a vector of size [2*N 1]
>
> and if so is it possible to use as an output a matrix instead of a
> vector?
>
> No. Unfold the matrix into a vector using (:).
>
> > Probably the information you gave about unfolding the targets is useful, but I am not familiar with this procedure, and I am not sure I comprehend it properly.
>
> A = rand(2,3)
> B = A(:)
>
> Hope this helps.
>
> Greg
>

-------------------
Thank you again Greg,

I understand what you are proposing, unfortunately I had already tried with a vector of size[2*N,1], with no good results. I expected that probably there would not be any other alternative for the network output (a matrix instead of a vector), but I asked just in the case I was wrong.
I believe the results are not good because in the case of a vector you have a row of successive xs and ys, so the network does not learn to create pairs of x and y. It thinks that the two successive nodes correspond to independent values of the same variable. Probably it does not distinguishes between x and y. That's why I wanted to assign to each output node a pair of numbers (x and y) and not just one.

Thank you for your time and if you have another idea I will be happy to hear it.

Regards,

Takis











> > Greg Heath <he...@alumni.brown.edu> wrote in message <30f3ffdd-9a01-4897-b605-37d1b9544...@p23g2000vbl.googlegroups.com>...
> > > On Nov 9, 8:23?am, "Takis Pan" <k_g...@hotmail.com> wrote:
> > > > Hello,
> >
> > > > I would appreciate if anyone could give me an answer to this problem. I was wondering if a MLP neural network could be trained giving at the output layer not a vector (e.g. 4x1, which means 4 output nodes) but a matrix (e.g. 4x4). If this is possible, how many output nodes should I declare at the training of the network. For example I want the output of the network to be four pairs of x,y, namely, x1,y1 - x2,y2 - x3,y3 - x4,y4. I should note that x,y are independent variables.
> >
> > > Your question is not clearly stated. So I will just state:
> >
> > > There is one output node for each output signal. If your
> > > output, y, is a time series of an mXn matrix for N observation
> > > times and you want to predict the signal for time t(N+1).
> >
> > > Unfold the training targets if size(t) = [m n N], use the
> > > matrix unfolding operator (:) to reduce the matrices to
> > > vectors then size(t1) = [p N] for p = m*n and the rest
> > > folloes as usual.
> >
> > > Therefore the net output will have p = m*n nodes and
> > > the matrix structure of the output y can be recovered
> > > from y1 by using the reshape function.
> >
> > > Hope this helps.
> >
> > > Greg- Hide quoted text -
> >
> > - Show quoted text -

On Nov 11, 2:16 pm, "Takis Pan" <k_g...@hotmail.com> wrote:
> Greg Heath <he...@alumni.brown.edu> wrote in message <a8d9417e-3150-40c9-8ec2-9de855c65...@l35g2000vba.googlegroups.com>...
> > PLEASE DO NOT TOP POST. PLACE YOUR REPLY AFTER OR
> > INTERSPERSED WITHIN THE PREVIOUS POST
>
> > On Nov 9, 2:33?pm, "Takis Pan" <k_g...@hotmail.com> wrote:
> > > Dear Greg, thank you for your quick response. Sorry if my question was misleading. Let me try be more precise about what I need.
>
> > > Generally, a MLP has an output layer with N nodes, with each node correspond to a single value. Thus, one can represent these values with a vector, let say T. If the corresponding input vector is P, then the training pairs are [P,T] and the training could be:
> > > net=newff(minmax(P),[30,N],{'tansig','purelin'},'trainlm');
> > > net=init(net);
> > > [pn,minp,maxp,tn,mint,maxt] = premnmx(P,T);
> > > [net,tr]=train(net,pn,tn);
> > > where 30 is the number of hidden layer's nodes and N the number of output nodes.
>
> > > Now, I do not have the case of a time series. I want to examine the case where each output node gives a point in the xy plane, corresponding to a coordinate (e.g. x1, y1). How could I train a NN, given the corresponding input vector P, to find N such points (xn, yn), namely N pairs of x,y? What number of output nodes should I give in the line:
> > > net=newff(minmax(P),[30,?],{'tansig','purelin'},'trainlm'); ?
>
> > size(t,1) = 2*N
>
> > >The output would be a matrix (Nx2),
>
> > No. The output for a single input must be a vector of size [2*N 1]
>
> > and if so is it possible to use as an output a matrix instead of a
> > vector?
>
> > No. Unfold the matrix into a vector using (:).
>
> > > Probably the information you gave about unfolding the targets is useful, but I am not familiar with this procedure, and I am not sure I comprehend it properly.
>
> > A = rand(2,3)
> > B = A(:)
>
> > Hope this helps.
>
> > Greg
>
> -------------------
> Thank you again Greg,
>
> I understand what you are proposing, unfortunately I had already tried with a
> vector of size[2*N,1], with no good results. I expected that probably there would
> not be any other alternative for the network output (a matrix instead of a vector),
> but I asked just in the case I was wrong.
> I believe the results are not good because in the case of a vector you have a row > of successive xs and ys, so the network does not learn to create pairs of x and y.
> It thinks that the two successive nodes correspond to independent values of the
> same variable.

No, it does not.

It assumes whatever correlation structure is contained in the target
matrix.

> Probably it does not distinguishes between x and y.

It doesn't and it doesn't have to.

> That's why I
> wanted to assign to each output node a pair of numbers (x and y) and not just .
> one. Thank you for your time and if you have another idea I will be happy to
> hear it.  

The cause of failure is not output vector formatting.
Does your training data adequately characterize the complete data set?
size(Ptrn) = ?
size(Ttrn) = ?
size(Ptst) = ?
size(Ttst) = ?
Are you overfitting with H = 30?

etc.

Greg

Greg Heath <heath@alumni.brown.edu> wrote in message <c77481c4-1fd6-4c19-a9b7-20178f67f84f@p36g2000vbn.googlegroups.com>...
> On Nov 11, 2:16?pm, "Takis Pan" <k_g...@hotmail.com> wrote:
> > Greg Heath <he...@alumni.brown.edu> wrote in message <a8d9417e-3150-40c9-8ec2-9de855c65...@l35g2000vba.googlegroups.com>...
> > > PLEASE DO NOT TOP POST. PLACE YOUR REPLY AFTER OR
> > > INTERSPERSED WITHIN THE PREVIOUS POST
> >
> > > On Nov 9, 2:33?pm, "Takis Pan" <k_g...@hotmail.com> wrote:
> > > > Dear Greg, thank you for your quick response. Sorry if my question was misleading. Let me try be more precise about what I need.
> >
> > > > Generally, a MLP has an output layer with N nodes, with each node correspond to a single value. Thus, one can represent these values with a vector, let say T. If the corresponding input vector is P, then the training pairs are [P,T] and the training could be:
> > > > net=newff(minmax(P),[30,N],{'tansig','purelin'},'trainlm');
> > > > net=init(net);
> > > > [pn,minp,maxp,tn,mint,maxt] = premnmx(P,T);
> > > > [net,tr]=train(net,pn,tn);
> > > > where 30 is the number of hidden layer's nodes and N the number of output nodes.
> >
> > > > Now, I do not have the case of a time series. I want to examine the case where each output node gives a point in the xy plane, corresponding to a coordinate (e.g. x1, y1). How could I train a NN, given the corresponding input vector P, to find N such points (xn, yn), namely N pairs of x,y? What number of output nodes should I give in the line:
> > > > net=newff(minmax(P),[30,?],{'tansig','purelin'},'trainlm'); ?
> >
> > > size(t,1) = 2*N
> >
> > > >The output would be a matrix (Nx2),
> >
> > > No. The output for a single input must be a vector of size [2*N 1]
> >
> > > and if so is it possible to use as an output a matrix instead of a
> > > vector?
> >
> > > No. Unfold the matrix into a vector using (:).
> >
> > > > Probably the information you gave about unfolding the targets is useful, but I am not familiar with this procedure, and I am not sure I comprehend it properly.
> >
> > > A = rand(2,3)
> > > B = A(:)
> >
> > > Hope this helps.
> >
> > > Greg
> >
> > -------------------
> > Thank you again Greg,
> >
> > I understand what you are proposing, unfortunately I had already tried with a
> > vector of size[2*N,1], with no good results. I expected that probably there would
> > not be any other alternative for the network output (a matrix instead of a vector),
> > but I asked just in the case I was wrong.
> > I believe the results are not good because in the case of a vector you have a row > of successive xs and ys, so the network does not learn to create pairs of x and y.
> > It thinks that the two successive nodes correspond to independent values of the
> > same variable.
>
> No, it does not.
>
> It assumes whatever correlation structure is contained in the target
> matrix.
>
> > Probably it does not distinguishes between x and y.
>
> It doesn't and it doesn't have to.
>
> > That's why I
> > wanted to assign to each output node a pair of numbers (x and y) and not just .
> > one. Thank you for your time and if you have another idea I will be happy to
> > hear it. ?
>
> The cause of failure is not output vector formatting.
> Does your training data adequately characterize the complete data set?
> size(Ptrn) = ?
> size(Ttrn) = ?
> size(Ptst) = ?
> size(Ttst) = ?
> Are you overfitting with H = 30?
>
> etc.
>
> Greg

Dear Greg,

It is indeed possible that the number of training pairs is not adequately big. Till now I have only worked with typical feedforward networks. Do you think Self Organizing Networks could help me more with this particular problem?

Thank you,

Takis

On Nov 20, 7:19 am, "Takis Pan" <k_g...@hotmail.com> wrote:
> Greg Heath <he...@alumni.brown.edu> wrote in message <c77481c4-1fd6-4c19-a9b7-20178f67f...@p36g2000vbn.googlegroups.com>...
> > On Nov 11, 2:16?pm, "Takis Pan" <k_g...@hotmail.com> wrote:
> > > Greg Heath <he...@alumni.brown.edu> wrote in message <a8d9417e-3150-40c9-8ec2-9de855c65...@l35g2000vba.googlegroups.com>...
> > > > PLEASE DO NOT TOP POST. PLACE YOUR REPLY AFTER OR
> > > > INTERSPERSED WITHIN THE PREVIOUS POST
>
> > > > On Nov 9, 2:33?pm, "Takis Pan" <k_g...@hotmail.com> wrote:
> > > > > Dear Greg, thank you for your quick response. Sorry if my question was misleading. Let me try be more precise about what I need.
>
> > > > > Generally, a MLP has an output layer with N nodes, with each node correspond to a single value. Thus, one can represent these values with a vector, let say T. If the corresponding input vector is P, then the training pairs are [P,T] and the training could be:
> > > > > net=newff(minmax(P),[30,N],{'tansig','purelin'},'trainlm');
> > > > > net=init(net);
> > > > > [pn,minp,maxp,tn,mint,maxt] = premnmx(P,T);
> > > > > [net,tr]=train(net,pn,tn);
> > > > > where 30 is the number of hidden layer's nodes and N the number of output nodes.
>
> > > > > Now, I do not have the case of a time series. I want to examine the case where each output node gives a point in the xy plane, corresponding to a coordinate (e.g. x1, y1). How could I train a NN, given the corresponding input vector P, to find N such points (xn, yn), namely N pairs of x,y? What number of output nodes should I give in the line:
> > > > > net=newff(minmax(P),[30,?],{'tansig','purelin'},'trainlm'); ?
>
> > > > size(t,1) = 2*N
>
> > > > >The output would be a matrix (Nx2),
>
> > > > No. The output for a single input must be a vector of size [2*N 1]
>
> > > > and if so is it possible to use as an output a matrix instead of a
> > > > vector?
>
> > > > No. Unfold the matrix into a vector using (:).
>
> > > > > Probably the information you gave about unfolding the targets is useful, but I am not familiar with this procedure, and I am not sure I comprehend it properly.
>
> > > > A = rand(2,3)
> > > > B = A(:)
>
> > > > Hope this helps.
>
> > > > Greg
>
> > > -------------------
> > > Thank you again Greg,
>
> > > I understand what you are proposing, unfortunately I had already tried with a
> > > vector of size[2*N,1], with no good results. I expected that probably there would
> > > not be any other alternative for the network output (a matrix instead of a vector),
> > > but I asked just in the case I was wrong.
> > > I believe the results are not good because in the case of a vector you have a row > of successive xs and ys, so the network does not learn to create pairs of x and y.
> > > It thinks that the two successive nodes correspond to independent values of the
> > > same variable.
>
> > No, it does not.
>
> > It assumes whatever correlation structure is contained in the target
> > matrix.
>
> > > Probably it does not distinguishes between x and y.
>
> > It doesn't and it doesn't have to.
>
> > > That's why I
> > > wanted to assign to each output node a pair of numbers (x and y) and not just .
> > > one. Thank you for your time and if you have another idea I will be happy to
> > > hear it. ?
>
> > The cause of failure is not output vector formatting.
> > Does your training data adequately characterize the complete data set?
> > size(Ptrn) = ?
> > size(Ttrn) = ?
> > size(Ptst) = ?
> > size(Ttst) = ?
> > Are you overfitting with H = 30?
>
> > etc.
>
> > Greg
>
> Dear Greg,
>
> It is indeed possible that the number of training pairs is not adequately big.

I see you didn't answer my questions. What are the sizes?

Till now I have only worked with typical feedforward networks. Do you
think Self Organizing Networks could help me more with this particular
problem?

This particular problem looks pretty ordinary.
A self organizing nets is unsupervised.
Why would you think that would help?

Greg

Greg Heath <heath@alumni.brown.edu> wrote in message <70d526b2-26c4-4258-bd8a-e7aab174a0ba@l2g2000yqd.googlegroups.com>...
> On Nov 20, 7:19?am, "Takis Pan" <k_g...@hotmail.com> wrote:
> > Greg Heath <he...@alumni.brown.edu> wrote in message <c77481c4-1fd6-4c19-a9b7-20178f67f...@p36g2000vbn.googlegroups.com>...
> > > On Nov 11, 2:16?pm, "Takis Pan" <k_g...@hotmail.com> wrote:
> > > > Greg Heath <he...@alumni.brown.edu> wrote in message <a8d9417e-3150-40c9-8ec2-9de855c65...@l35g2000vba.googlegroups.com>...
> > > > > PLEASE DO NOT TOP POST. PLACE YOUR REPLY AFTER OR
> > > > > INTERSPERSED WITHIN THE PREVIOUS POST
> >
> > > > > On Nov 9, 2:33?pm, "Takis Pan" <k_g...@hotmail.com> wrote:
> > > > > > Dear Greg, thank you for your quick response. Sorry if my question was misleading. Let me try be more precise about what I need.
> >
> > > > > > Generally, a MLP has an output layer with N nodes, with each node correspond to a single value. Thus, one can represent these values with a vector, let say T. If the corresponding input vector is P, then the training pairs are [P,T] and the training could be:
> > > > > > net=newff(minmax(P),[30,N],{'tansig','purelin'},'trainlm');
> > > > > > net=init(net);
> > > > > > [pn,minp,maxp,tn,mint,maxt] = premnmx(P,T);
> > > > > > [net,tr]=train(net,pn,tn);
> > > > > > where 30 is the number of hidden layer's nodes and N the number of output nodes.
> >
> > > > > > Now, I do not have the case of a time series. I want to examine the case where each output node gives a point in the xy plane, corresponding to a coordinate (e.g. x1, y1). How could I train a NN, given the corresponding input vector P, to find N such points (xn, yn), namely N pairs of x,y? What number of output nodes should I give in the line:
> > > > > > net=newff(minmax(P),[30,?],{'tansig','purelin'},'trainlm'); ?
> >
> > > > > size(t,1) = 2*N
> >
> > > > > >The output would be a matrix (Nx2),
> >
> > > > > No. The output for a single input must be a vector of size [2*N 1]
> >
> > > > > and if so is it possible to use as an output a matrix instead of a
> > > > > vector?
> >
> > > > > No. Unfold the matrix into a vector using (:).
> >
> > > > > > Probably the information you gave about unfolding the targets is useful, but I am not familiar with this procedure, and I am not sure I comprehend it properly.
> >
> > > > > A = rand(2,3)
> > > > > B = A(:)
> >
> > > > > Hope this helps.
> >
> > > > > Greg
> >
> > > > -------------------
> > > > Thank you again Greg,
> >
> > > > I understand what you are proposing, unfortunately I had already tried with a
> > > > vector of size[2*N,1], with no good results. I expected that probably there would
> > > > not be any other alternative for the network output (a matrix instead of a vector),
> > > > but I asked just in the case I was wrong.
> > > > I believe the results are not good because in the case of a vector you have a row > of successive xs and ys, so the network does not learn to create pairs of x and y.
> > > > It thinks that the two successive nodes correspond to independent values of the
> > > > same variable.
> >
> > > No, it does not.
> >
> > > It assumes whatever correlation structure is contained in the target
> > > matrix.
> >
> > > > Probably it does not distinguishes between x and y.
> >
> > > It doesn't and it doesn't have to.
> >
> > > > That's why I
> > > > wanted to assign to each output node a pair of numbers (x and y) and not just .
> > > > one. Thank you for your time and if you have another idea I will be happy to
> > > > hear it. ?
> >
> > > The cause of failure is not output vector formatting.
> > > Does your training data adequately characterize the complete data set?
> > > size(Ptrn) = ?
> > > size(Ttrn) = ?
> > > size(Ptst) = ?
> > > size(Ttst) = ?
> > > Are you overfitting with H = 30?
 
I do not quite understand the "H=30".
> >
> > > etc.
> >
> > > Greg
> >
> > Dear Greg,
> >
> > It is indeed possible that the number of training pairs is not adequately big.
>
> I see you didn't answer my questions. What are the sizes?
>
I did not answer your questions beacause I do not know what number would properly describe the problem. I have used a number of some thousands training pairs, however there are so many combinations of (x,y) pairs in the plane, that I do not know what number would be sufficient. Additionally there may be ambiguities in the since an output of the net (e.g. [x1 y1 x2 y2]) is equivalent for my problem with [x2 y2 x1 y1]. Namely, the order does not matter.

> Till now I have only worked with typical feedforward networks. Do you
> think Self Organizing Networks could help me more with this particular
> problem?
>
> This particular problem looks pretty ordinary.
> A self organizing nets is unsupervised.
> Why would you think that would help?

The problem is ordinary, however the great number of combinations and ambiguities make it difficult. I do not know much about self organizing nets, but maybe they could give a more clear picture and put in an order, at least the inputs of the problem.

Thank you,

Regards,

Takis
>
> Greg
>

On Nov 22, 12:35 pm, "Takis Pan" <k_g...@hotmail.com> wrote:
> Greg Heath <he...@alumni.brown.edu> wrote in message <70d526b2-26c4-4258-bd8a-e7aab174a...@l2g2000yqd.googlegroups.com>...
> > On Nov 20, 7:19?am, "Takis Pan" <k_g...@hotmail.com> wrote:
> > > Greg Heath <he...@alumni.brown.edu> wrote in message <c77481c4-1fd6-4c19-a9b7-20178f67f...@p36g2000vbn.googlegroups.com>...
> > > > On Nov 11, 2:16?pm, "Takis Pan" <k_g...@hotmail.com> wrote:
> > > > > Greg Heath <he...@alumni.brown.edu> wrote in message <a8d9417e-3150-40c9-8ec2-9de855c65...@l35g2000vba.googlegroups.com>...
> > > > > > PLEASE DO NOT TOP POST. PLACE YOUR REPLY AFTER OR
> > > > > > INTERSPERSED WITHIN THE PREVIOUS POST
>
> > > > > > On Nov 9, 2:33?pm, "Takis Pan" <k_g...@hotmail.com> wrote:
> > > > > > > Dear Greg, thank you for your quick response. Sorry if my question was misleading. Let me try be more precise about what I need.
>
> > > > > > > Generally, a MLP has an output layer with N nodes, with each node correspond to a single value. Thus, one can represent these values with a vector, let say T. If the corresponding input vector is P, then the training pairs are [P,T] and the training could be:
> > > > > > > net=newff(minmax(P),[30,N],{'tansig','purelin'},'trainlm');
> > > > > > > net=init(net);
> > > > > > > [pn,minp,maxp,tn,mint,maxt] = premnmx(P,T);
> > > > > > > [net,tr]=train(net,pn,tn);
> > > > > > > where 30 is the number of hidden layer's nodes and N the number of output nodes.
>
> > > > > > > Now, I do not have the case of a time series. I want to examine the case where each output node gives a point in the xy plane, corresponding to a coordinate (e.g. x1, y1). How could I train a NN, given the corresponding input vector P, to find N such points (xn, yn), namely N pairs of x,y? What number of output nodes should I give in the line:
> > > > > > > net=newff(minmax(P),[30,?],{'tansig','purelin'},'trainlm'); ?
>
> > > > > > size(t,1) = 2*N
>
> > > > > > >The output would be a matrix (Nx2),
>
> > > > > > No. The output for a single input must be a vector of size [2*N 1]
>
> > > > > > and if so is it possible to use as an output a matrix instead of a
> > > > > > vector?
>
> > > > > > No. Unfold the matrix into a vector using (:).
>
> > > > > > > Probably the information you gave about unfolding the targets is useful, but I am not familiar with this procedure, and I am not sure I comprehend it properly.
>
> > > > > > A = rand(2,3)
> > > > > > B = A(:)
>
> > > > > > Hope this helps.
>
> > > > > > Greg
>
> > > > > -------------------
> > > > > Thank you again Greg,
>
> > > > > I understand what you are proposing, unfortunately I had already tried with a
> > > > > vector of size[2*N,1], with no good results. I expected that probably there would
> > > > > not be any other alternative for the network output (a matrix instead of a vector),
> > > > > but I asked just in the case I was wrong.
> > > > > I believe the results are not good because in the case of a vector you have a row > of successive xs and ys, so the network does not learn to create pairs of x and y.
> > > > > It thinks that the two successive nodes correspond to independent values of the
> > > > > same variable.
>
> > > > No, it does not.
>
> > > > It assumes whatever correlation structure is contained in the target
> > > > matrix.
>
> > > > > Probably it does not distinguishes between x and y.
>
> > > > It doesn't and it doesn't have to.
>
> > > > > That's why I
> > > > > wanted to assign to each output node a pair of numbers (x and y) and not just .
> > > > > one. Thank you for your time and if you have another idea I will be happy to
> > > > > hear it. ?
>
> > > > The cause of failure is not output vector formatting.
> > > > Does your training data adequately characterize the complete data set?
> > > > size(Ptrn) = ?
> > > > size(Ttrn) = ?
> > > > size(Ptst) = ?
> > > > size(Ttst) = ?
> > > > Are you overfitting with H = 30?
>
> I do not quite understand the "H=30".
>
> > > > etc.
>
> > > > Greg
>
> > > Dear Greg,
>
> > > It is indeed possible that the number of training pairs is not adequately big.
>
> > I see you didn't answer my questions. What are the sizes?
>
> I did not answer your questions beacause I do not know what number would properly describe the problem. I have used a number of some thousands training pairs, however there are so many combinations of (x,y) pairs in the plane, that I do not know what number would be sufficient. Additionally there may be ambiguities in the since an output of the net (e.g. [x1 y1 x2 y2]) is equivalent for my problem with [x2 y2 x1 y1]. Namely, the order does not matter.
>
> > Till now I have only worked with typical feedforward networks. Do you
> > think Self Organizing Networks could help me more with this particular
> > problem?
>
> > This particular problem looks pretty ordinary.
> > A self organizing nets is unsupervised.
> > Why would you think that would help?
>
> The problem is ordinary, however the great number of combinations and ambiguities make it difficult. I do not know much about self organizing nets, but maybe they could give a more clear picture and put in an order, at least the inputs of the problem.

What does a single input vector represent?
What does the corresponding output vector represent?

Greg