Very different performance in Neural Network Toolbox 2015a Vs 2014a

I recently upgraded to 2015a and immediately noticed my neural networks were not performing correctly. I have a fresh install of both 2015a and 2014a, and I'm running the exact same code in each version. Bellow is the output of the nntraintool, performance plot and the confusion plot. 2014a is on the left and 2015a is on the right. The 2014a version runs for 263 iterations but the 2015a version stops after just 15. From the performance plot you can see that the 2015a version has stopped early because it very quickly achieves a great performance but if you look at the confusion plots you can see the output of the 2015a version is always 1. The 2015a version ends up with an error rate of about 40%, almost double the 22% error rate from the 2014a version. Once again these are both fresh installs running the exact same m-file. Does anyone know what has changed between these versions? I had a look at the change log but all it seems to say is "bug fixes" which is not terribly useful in determining why I'm seeing this behaviour. The code I'm using to train is:

 ClassificationNet = patternnet;
 ClassificationNet = train(ClassificationNet,x,y);

where x is 33 by 120826 double and y is 1 by 120826 double.

Answer 1

[TG Downes]

Ok so apparently 2015a no longer lets you use a single row output for binary classification. I think it was always documented that each class needed its own row but in 2014a you could get away with just on row if you only had two classes. This no longer seems to be the case in 2015a. I added the line of code:

 y = [y; ~y];

So now y a 2 by 120826 double, and now the performance between the two versions is comparable.

[Greg Heath]

For c mutually exclusive classes you only need c-1 rows because the estimated input-conditional probabilities should sum to 1.

However, that MATLAB option was only valid for c=2. NOW it is not valid at all?

In addition, MATLAB code makes no distinction between the cases of mutually exclusive classes (unity sum outputs) and non mutually exclusive outputs (outputs not constrained to unity sum) which require DIFFERENT FORMS of the cross-entropy function or logsig vs softmax for mean-square-error.