neural networks, MSE goal

Respected colleague

Would we perhaps be able to explain the term linked to neural networks MSEgoal = 0.01*Ndof*MSE00a/Ntrneq.

This term is quite often seen in training neural networks , but it unfortunately is not completely clear for me.

I need a more detailed explanation of the term , maybe some references, example or just explanations.

Also on this occasion, I wish to thank Mr. Greg Heath for excellent explanations and tutorials . Mr Greg Heath explanations are much better than many that can be found in the available literature . Thank you very much for that .

Answer 1

[Greg Heath]

You could have searched the NEWSGROUP and ANSWERS.

Anyway,

See: WIKIPEDIA: Degrees of Freedom

 Ntrn    = number of training examples
 O       = dimension of target examples
 Ntrneq  = Ntrn*O  number of training equations
 Nw      = number of unknown weights
 Ntrndof = Ntrneq-Nw number of estimation degrees of freedom

REFERENCE: THE NAIVE CONSTANT OUTPUT MODEL

 output = repmat(mean(target,2),1,Ntrn)
 MSE00  = MSE of the naive constant output model
 error  = target - output;
 MSE00  = mean(error.^2)
        = var(target,1)  % biased 
 MSE00a = var(target,0)  % unbiased
% "a" ==> "adjusted" for loss of degrees of freedom

 Unbiased MSEtrn of a NN with Nw weights
 Ntrndof = Ntrn - Nw
 MSEtrn  = SSEtrn/Ntrneq        % biased
 MSEtrna = SSEtrn/Ntrndof       % unbiased

NN training goal

 MSEtrna = MSE00a/100
 SSEtrn  = Ntrndof*MSE00a/100
 MSEtrn  = 0.01*Ntrndof*MSE00a/Ntrneq

Hope this helps.

Thank you for formally accepting my answer

Greg