MATLAB Answers

0

fitnet Why test results are unbiased?

Asked by Emiliano Rosso on 2 Apr 2015
Latest activity Answered by Greg Heath
on 22 May 2015
I've read form Greg Heath:
Data = Design + Non-design
Design = Train(estimate weights) + Validation(Stop training when MSEval goes thru a minimum).
Non-design = Test(Obtain UNBIASED generalization estimate of performance on unseen non-design data).
I like to know why and how test results are calculated on unbiased net despite the fact that training is made on biased network and how to obtain the same results using :
output=net(input)
I must use :
net.biasConnect = [0; 0];
after training and before using net for new outputs?
There is a contradiction between training net and the changing made after?
What do you mean when you say in:
"The test set error is unbiased because it is completely independent of design (training and validation)." ?

  0 Comments

Sign in to comment.

1 Answer

Answer by Greg Heath
on 22 May 2015
 Accepted Answer

This discussion has absolutely nothing to do with bias weights.
Theoretically, training, validation and test data are all assumed to be random samples from the same probability distribution function.
Both training and validation data are used to design the net with the goal of minimizing a performance function (such as mse (fitnet) or crossentropy (patternnet) for net performance on ANY data (seen and unseen) obtained from the same probability distribution.
Therefore, their performance cannot be used to obtain an unbiased (i.e., honest) estimate of performance on unseen data.
On the other hand, the test data is in no way involved in the design of the net. Therefore it is a valid representative of "unseen" data and it's performance is considered an unbiased estimate of net performance on unseen data.
Hope this helps.
Greg
PS Not sure why I never saw this post before.

  0 Comments

Sign in to comment.