|
On Sep 24, 7:04 am, "Amal " <Amal_khassaw...@hotmail.com> wrote:
> Greg Heath <he...@alumni.brown.edu> wrote in message <a085d3b2-8d28-4973-9bdd-e99573524...@c58g2000hsc.googlegroups.com>...
> > On Sep 21, 3:15 pm, Greg Heath <he...@alumni.brown.edu> wrote:
> > > On Sep 19, 9:22 am, "Amal " <Amal_khassaw...@hotmail.com> wrote:
> > > > "Amal " <Amal_khassaw...@hotmail.com> wrote in message <gai6s6$qp...@fred.mathworks.com>...
> > > > > I send a sample of data to your email.
> > > > > Greg Heath <he...@alumni.brown.edu> wrote in message <9656fffa-1437-4938-badc-d4ddf6fd1...@s50g2000hsb.googlegroups.com>...
> > > > > > On Sep 11, 6:12=A0am, "Amal " <Amal_khassaw...@hotmail.com> wrote:
> > > > > > > Hello,
>
> > > > > > > I think it is need improving becuase neural network cannot find a good fi=
> > > > > > tting function when the number of output is greater than the number of inpu=
> > > > > > t, actually i try it on my data and i got very bad result, so im trying to =
> > > > > > improve my network by changing the training algorithm.
> > > > > > > when the number of output is too large comparing with the number of input=
> > > > > > the neural network can not find good fitting for these data.
>
> > > > > > Can you email or post your data? Some of us might like to see exactly
> > > > > > what your problem is.
>
> > > > > > If you do, don't expect a rapid turn around time.
>
> > > > > > Greg
>
> > > > Thank you for your posts, and am posting from down as you ask..
> > > > back to subject, the data was just a sample, I am a researcher in bioinformatics field, and the data i send is about proteins, the input represent the protein sequence and the output represent its corrosponding torsion angle, my work try to analyze if there is a relationship between protein amino acids and its angles.
>
> > > > i am ready for any anymore expalnation ..
>
> > > The data you sent appears to have serious problems.
>
> Thank you for deep explanation, i know that my data have a lot of problems.
>
> > > 1. The number of input and output columns is not the same:
>
> > > size(X0) = [ 18 46 ] % Must delete the last column!
> > > size(Y0) = [ 64 45 ]
>
> > > X0 = X0(:,1:end-1);
>
> thanks.
>
> > > 2. Only 35 columns of X0 are unique
>
> > > % Valid duplicate observations? Check the corresponding columns in Y0
>
> Yes, it is a valid duplication.
>
> > OK. There are no duplicate columns in Y0. However, that means
> > there are 10 observations for which the X-->Y transformation
> > is not unique, This scatter can be considered as noise and the
> > effects should be revisited.
>
> I need these becuase it mean a lot in my research.
>
> > > 3. rank(X0) = 13 and rank(Y0) = 31 % Consider variable
> > > reductions
>
> > > 4. Only 17 rows of X0 are unique and only 61 rows of Y0 are unique.
> > > a. Rows 16 and 17 of X0 are identical; delete Row 17
> > > b. Rows 1, 62, 63 and 64 of Y0 are identical; delete Row
> > > 62-64
> > > c. Rows 1, 62, 63 and 64 of Y0 are identicallly
> > > zero. Therefore they have zero variance; delete Row 1
>
> > > X1 = unique(X0,'rows'); % Row 17
> > > Y1 = unique(Y0,'rows'); % Rows 62-64
> > > deleterowsX1 = find(~var(X1')) % []
> > > deleterowsY1 = find(~var(Y1')) % Row 34
>
> > > size(Y1) = [ 17 45 ]
> > > size(X1) = [ 60 45 ]
>
> > > 5. CX = corrcoef(X1') shows that all of the cross correlation
> > > coefficients among input variables 3 to 7 exceed 0.98. This
> > > accounts for rank(X1) = 13. In addition, CX(1,2) = 0.97
>
> > > % I J CX(I,J)
> > > % 1 2 0.97204
> > > % 3 4 1
> > > % 3 5 0.98835
> > > % 4 5 0.98835
> > > % 3 6 1
> > > % 4 6 1
> > > % 5 6 0.98835
> > > % 3 7 0.98129
> > > % 4 7 0.98129
> > > % 5 7 0.99916
> > > % 6 7 0.98129
>
> > > At this stage I would feel confident removing input variables,
> > > 2 t0 6 resulting in
>
> > > X2(2:6) = [];
> > > size(X2) = [12 45]
> > > rank(X2) = 12
>
> I have a question for you here, when you delete these rows that is mean you delete part of the sequences in my research
No. It doesn't. See below.
>and i want the complete sequence.My idea in this work is to look to the whole sequnce and try to find the relation between the element of the sequence and it is corrosponding angle values, and try predicting the effect of neighbors.
If you have inputs x1, x2 with correlation coefficient
CX12 > 0.98, then, with small error
x2 ~ x20+C12*(x1-x10)*(sx1/sx2)) .
where (xi0,sxi) are the mean and standard deviation of xi.
Therefore x1 and x2 are, for all practical purposes, linearly
dependent. Consequently, it is sufficient to use only one of
them to predict y(x1,x2) ~g1(x1) ~ g2(x2).
In fact when predictors are linearly dependent (or nearly
so), models relating input and output tend to be nonunique
and, therefore, unstable. Therefore, it is impossible to
deduce the influence of an individual predictor.
Consequently, it is usually wise to remove near linear
dependencies among inputs by either removing
"redundant" inputs or using orthogonal linear input
combinations (e.g., PCA).
Similarly, if you have outputs y1(x), y2(x) with correlation
coefficient CY12 > 0.98, then, with small error
y2 ~ y20+C12*(y1-y10)*(sy1/sy2)) .
Therefore y1(x) and y2(x) are, for all practical purposes,
linearly dependent. Consequently, it is sufficient to use
x to predict only one of them, say y1. Then y2 can be
obtained from y1. Unlike inputs, keeping linearly
dependent outputs should not significantly degrade the
model. However, the additional unnecessary calulations
are a waste of resources.
> > > 6. Take a look at CY = corrcoef(Y1') to see which Y rows are
> > > obviously redundant and can be readily removed from training
> > > the model. You should find 22 output variables are
> > > highly correlated with abs(CY(i,j)) > 0.95. An alternative
> > > is to use PCA and obtain 31 or less linearly independent
> > > output variables.
>
> > > 7. Before designing a 12-H-31 MLP, take a look at a linear
> > > model..
>
> > With 12 inputs, 45 observations and O outputs,there are
> > Neq = 45*O equations to estimate Nw = (12+1)*O
> > = weights. The ratio Neq/Nw = 3.48 is marginal for
> > insuring weight estimate accuracy but is independent
> > of the number of outputs. Nevertheless, smaller numbers
> > of outputs improve computational efficiency and precision.
>
> > Using the unreduced 60 outputs, the linear model yields
>
> > NMSE = 0.366, R^2 = 0.634
>
> > and with the degree of freedom adjustment,
>
> > NMSEadj = 0.497, Radj^2 = 0.503
>
> > 8. Using a NN to effect a factor of 10 reduction in NMSEadj
> > should be satisfactory. However, the increase of Nw caused
> > by hidden nodes might require regularization. N = 45 may
> > not be large enough to support Stopped Training.
>
>
> Hope you see my notice in between, but in general you don't think that you deal with input in behalf of output, that is mean you treat them independently.Thank you so much for your notice, they was a great help, but i have to analyze my data more to check if i can apply your ideas.
> Again thank you..
Let me emphasize the following:
If inputs are linearly dependent, their individual
influence on individual outputs cannot be deduced. In
addition, the models produced tend to be unstable
because they are not unique.
Therefore input reduction via eliminating redundant
input variables or using linearly independent linear
combinations (e.g., PCA) is warranted.
Hope this helps.
Greg
|
Feed for this Thread
E-mail this page to a colleague