Neural network validation checks net.TrainParam.max_fail <- is a bigger or a smaller number better?

While trying to improve my neural network I wondered, whether I should increase or decrease

TrainParam.max_fail

(default value is 6)

This documentary (https://de.mathworks.com/help/deeplearning/ref/trainlm.html) says:

Training stops when any of these conditions occurs:

• "Validation performance has increased more than max_fail times since the last time it decreased (when using validation)."

which I interpret as: if validation error decreases more than 6 times -> early stopping

This documentary (https://de.mathworks.com/help/deeplearning/ug/improve-neural-network-generalization-and-avoid-overfitting.html) says:

When the validation error increases for a specified number of iterations (net.trainParam.max_fail), the training is stopped, and the

weights and biases at the minimum of the validation error are returned.

which I interpret as: if validation error increases more than 6 times -> early stopping

So what is the purpose of the net.TrainParam.max_fail?

____________________________________________________________________________________

Second question in the same post:

When my Trainratio/Validationratio/Testratio is 70/25/5.

After how many Train-epochs is there an Validation-Epoch?

Thank you very much in advance!

Answer 1

[Anshika Chaurasia]

Hi,

1. Training stops when any of these conditions occurs:

• "Validation performance has increased more than max_fail times since the last time it decreased (when using validation)."

In above lines, "Validation Performance" means validation error. Hence, the interpretation of above line will be:

if validation error increases more than 6 times -> early stopping

To understand the terminology refer to following documents:

Calculate network performance - MATLAB perform (mathworks.com)

https://www.mathworks.com/help/deeplearning/ug/neural-network-object-properties.html#bss4hk6-52

2. After each training epoch validation will occur. Or, in an epoch first training will be done then validation.

[NightStalker]

Hi Anshika Chaurasia,

this was the very solution to my first question. Thank you!

Your answer to my second question lead to more questions, which I really hope you can answer me:

2.a) What will happen, if the testset has 1000 samples but the validationset only 500? will the validation-procedures be evenly distributed over the test-procedures or will some validation samples be used more than 1 time and force the validation-procedure after every test-procedure?

2.b) After one test-procedure, how does MATLAB pick/choose the sample within the validationset to validate this test-procedure?

Thanks in advance!

Answer 2

[pathakunta]

1. Training stops when any of these conditions occurs: "Validation performance has increased more than max_fail times since the last time it decreased (when using validation)." In above lines, "Validation Performance" means validation error. Hence, the interpretation of above line will be: if validation error increases more than 6 times -> early stopping To understand the terminology refer to following documents: Calculate network performance - MATLAB perform (mathworks.com) https://www.mathworks.com/help/deeplearning/ug/neural-network-object-properties.html#bss4hk6-52 2. After each training epoch validation will occur. Or, in an epoch first training will be done then validation.

Answer 3

[pathakunta]

1. Training stops when any of these conditions occurs: "Validation performance has increased more than max_fail times since the last time it decreased (when using validation)." In above lines, "Validation Performance" means validation error. Hence, the interpretation of above line will be: if validation error increases more than 6 times -> early stopping To understand the terminology refer to following documents: Calculate network performance - MATLAB perform (mathworks.com) https://www.mathworks.com/help/deeplearning/ug/neural-network-object-properties.html#bss4hk6-52 2. After each training epoch validation will occur. Or, in an epoch first training will be done then validation.

Answer 4

[pathakunta]

1. Training stops when any of these conditions occurs: "Validation performance has increased more than max_fail times since the last time it decreased (when using validation)." In above lines, "Validation Performance" means validation error. Hence, the interpretation of above line will be: if validation error increases more than 6 times -> early stopping To understand the terminology refer to following documents: Calculate network performance - MATLAB perform (mathworks.com) https://www.mathworks.com/help/deeplearning/ug/neural-network-object-properties.html#bss4hk6-52 2. After each training epoch validation will occur. Or, in an epoch first training will be done then validation.