fit function not iterating

I am attempting to fit a set of data with a custom function. The fit function uses my fitoptions.startpoint data, but then stops and spits out bad fit statistics.

datax = [-7.00100000000000
-6.95600000000000
-6.90600000000000
-6.85600000000000
-6.80600000000000
-6.75600000000000
-6.70700000000000
-6.65700000000000
-6.60700000000000
-6.55700000000000
-6.50700000000000
-6.45700000000000
-6.40700000000000
-6.35700000000000
-6.30700000000000
-6.25700000000000
-6.20700000000000
-6.15800000000000
-6.10800000000000
-6.05800000000000
-6.00900000000000
-5.96300000000000
-5.91400000000000
-5.86400000000000
-5.81300000000000
-5.76400000000000
-5.71300000000000
-5.66300000000000
-5.61400000000000
-5.56300000000000
-5.51400000000000
-5.46400000000000
-5.41400000000000
-5.36400000000000
-5.31400000000000
-5.26400000000000
-5.21500000000000
-5.16400000000000
-5.11000000000000
-5.06000000000000
-5.00900000000000
-4.96500000000000
-4.91600000000000
-4.86500000000000
-4.81600000000000
-4.76500000000000
-4.71600000000000
-4.66600000000000
-4.61600000000000
-4.56500000000000
-4.51600000000000
-4.46500000000000
-4.41600000000000
-4.36700000000000
-4.31700000000000
-4.26700000000000
-4.21700000000000
-4.16600000000000
-4.11700000000000
-4.06700000000000
-4.01700000000000
-3.97200000000000
-3.92200000000000
-3.87200000000000
-3.82300000000000
-3.77300000000000
-3.72300000000000
-3.67300000000000
-3.62300000000000
-3.57300000000000
-3.52300000000000
-3.47300000000000
-3.42300000000000
-3.37300000000000
-3.32300000000000
-3.27400000000000
-3.22400000000000
-3.17400000000000
-3.12400000000000
-3.07400000000000
-3.02500000000000
-2.97900000000000
-2.93000000000000
-2.88000000000000
-2.82900000000000
-2.77900000000000
-2.73000000000000
-2.67900000000000
-2.63000000000000
-2.58000000000000
-2.53100000000000
-2.48100000000000
-2.43100000000000
-2.38100000000000
-2.33100000000000
-2.28100000000000
-2.23100000000000
-2.18200000000000
-2.13200000000000
-2.08200000000000
-2.03200000000000
-1.98700000000000
-1.93700000000000
-1.88700000000000
-1.83700000000000
-1.78700000000000
-1.73700000000000
-1.68700000000000
-1.63700000000000
-1.58700000000000
-1.53700000000000
-1.48700000000000
-1.43700000000000
-1.38700000000000
-1.33700000000000
-1.28700000000000
-1.23800000000000
-1.18900000000000
-1.13900000000000
-1.08900000000000
-1.03900000000000
-0.994000000000000
-0.944000000000000
-0.894000000000000
-0.845000000000000
-0.795000000000000
-0.745000000000000
-0.695000000000000
-0.645000000000000
-0.595000000000000
-0.545000000000000
-0.495000000000000
-0.445000000000000
-0.395000000000000
-0.345000000000000
-0.295000000000000
-0.245000000000000
-0.196000000000000
-0.146000000000000
-0.0960000000000000
-0.0460000000000000]
datay = [4.75700000000000e-12
4.71700000000000e-12
4.72400000000000e-12
4.72900000000000e-12
4.78700000000000e-12
4.79900000000000e-12
4.80800000000000e-12
4.82000000000000e-12
4.83300000000000e-12
4.83900000000000e-12
4.85700000000000e-12
4.86400000000000e-12
4.87500000000000e-12
4.88800000000000e-12
4.89700000000000e-12
4.91300000000000e-12
4.92600000000000e-12
4.93600000000000e-12
4.94500000000000e-12
4.96200000000000e-12
4.96700000000000e-12
4.95200000000000e-12
4.96200000000000e-12
4.97800000000000e-12
5.02100000000000e-12
5.03100000000000e-12
5.05500000000000e-12
5.06200000000000e-12
5.07600000000000e-12
5.09100000000000e-12
5.10200000000000e-12
5.11400000000000e-12
5.13300000000000e-12
5.15000000000000e-12
5.15900000000000e-12
5.18000000000000e-12
5.19300000000000e-12
5.20600000000000e-12
5.22500000000000e-12
5.19200000000000e-12
5.23500000000000e-12
5.25500000000000e-12
5.26900000000000e-12
5.29000000000000e-12
5.30400000000000e-12
5.31900000000000e-12
5.33900000000000e-12
5.35300000000000e-12
5.36700000000000e-12
5.38800000000000e-12
5.42200000000000e-12
5.42200000000000e-12
5.44800000000000e-12
5.46600000000000e-12
5.47800000000000e-12
5.50300000000000e-12
5.52000000000000e-12
5.54000000000000e-12
5.56300000000000e-12
5.58500000000000e-12
5.60400000000000e-12
5.63700000000000e-12
5.65400000000000e-12
5.67600000000000e-12
5.70200000000000e-12
5.72600000000000e-12
5.74200000000000e-12
5.77600000000000e-12
5.79700000000000e-12
5.81600000000000e-12
5.84700000000000e-12
5.86500000000000e-12
5.88800000000000e-12
5.91300000000000e-12
5.93900000000000e-12
5.96700000000000e-12
5.99600000000000e-12
6.02300000000000e-12
6.05200000000000e-12
6.08100000000000e-12
6.10800000000000e-12
6.09100000000000e-12
6.12100000000000e-12
6.14700000000000e-12
6.18200000000000e-12
6.21000000000000e-12
6.23500000000000e-12
6.27800000000000e-12
6.30500000000000e-12
6.34700000000000e-12
6.37900000000000e-12
6.41800000000000e-12
6.46000000000000e-12
6.49300000000000e-12
6.53100000000000e-12
6.60700000000000e-12
6.65300000000000e-12
6.69100000000000e-12
6.73700000000000e-12
6.78200000000000e-12
6.82900000000000e-12
6.86500000000000e-12
6.91400000000000e-12
6.96000000000000e-12
7.01300000000000e-12
7.06300000000000e-12
7.11500000000000e-12
7.17300000000000e-12
7.22400000000000e-12
7.28000000000000e-12
7.34100000000000e-12
7.40100000000000e-12
7.46600000000000e-12
7.53200000000000e-12
7.59800000000000e-12
7.67100000000000e-12
7.74500000000000e-12
7.81400000000000e-12
7.89500000000000e-12
7.97900000000000e-12
8.05900000000000e-12
8.11800000000000e-12
8.20900000000000e-12
8.30600000000000e-12
8.41100000000000e-12
8.53900000000000e-12
8.62900000000000e-12
8.76400000000000e-12
8.88700000000000e-12
8.99100000000000e-12
9.13700000000000e-12
9.28600000000000e-12
9.45000000000000e-12
9.62000000000000e-12
9.79700000000000e-12
9.99300000000000e-12
1.02000000000000e-11
1.04290000000000e-11
1.06980000000000e-11
1.09840000000000e-11
1.12850000000000e-11]

CJ_fittype = fittype('11.8.*8.85e-12.*area./((((m+2).*11.8.*8.85e-12)./(1.602e-19.*beta).*(Vbi-Va)).^(1./(m+2)))',...
'coefficients',{'Vbi','area','beta','m'},'independent',{'Va'})

CJ_fitoptions = fitoptions(CJ_fittype)
CJ_fitoptions.StartPoint = [0.7 5.2e-5 1e19 1];
CJ_fitoptions.Lower = [0.2 5.2e-9 1e10 0];
CJ_fitoptions.Upper = [1.2 5.2e-3 1e24 3];
[curveFit1,curveGOF1] = fit(datax,datay,CJ_fittype,CJ_fitoptions)

As I stated earlier, the fit function runs once with the initial guess, and provides a 95% confidence bound for m, but does not for Vbi, area, or beta. I have attempted to change the other fitoptions tolerances and robust settings. I have verified my equation is input correctly. I believe the remaining "coefficients" should be unique. But I need to run through this process about 100 more times, and not having to manually iterate each fit would be nice. Any help or suggestions would be appreciated.

Thank you,

George

[Nestor Hernandez]

If I understand correctly, you want to try 100 configurations for fitpotions and run fit with each one and then see the results? If you have all the possible configurations, i.e., startpoint, lower, upper, we could try creating vectors for each parameter, or a matlab table and iterate over it using a for loop and calling fit every time.

[George Peterson]

Nestor,

Not quite. I would like the fit function to compare a set of coefficients of the function to the data and evaluate the fitting function, attempt a 2nd set of coefficients of the function, and again evalute the fitting function... did the fit get better or worse (i.e. R_squared etc from the goodness of fit output) and then adjust the coefficients again accordingly to arrive at a set of coefficients that minimizes the deviation from the data.

I have used the fit function for this before, but for this particular fit, it only performs one calculation, and then exits the function.Previously, I have had to increase the maximum itterations to 10000+. I am just at a loss for what is causing the function to exit right now.

Thank you,

George

[Alex Sha]

The result is not bad:

Root of Mean Square Error (RMSE): 1.63940860716371E-14
Sum of Squared Residual: 3.78960141955188E-26
Correlation Coef. (R): 0.999942423834309
R-Square: 0.999884850983632

Parameter	Best Estimate
----------	-------------
vbi	0.533035313889232
area	1185.68913752864
beta	0.00136089634095781
m	0.9485721118

[George Peterson]

Alex,

I agree that your fit is not bad. But for some reason, your attempted fit has gone through multiple iterations instead of spitting out the calculation from the fitoptions.Startpoint. So my question is what did you do to get the fit function to run multiple times where my attempt was a single iteration? What were your exact commands to Matlab?

Thank you,

George

Answer 1

[Matt J]

You should adjust your units for both the x,y data and for the parameters so that the datay are not so uniformly close to 0 and so you don't have such disparate orders of magnitudes among your parameters like 0.2 versus 1e24. Note that double floating point arithemetic can only keep track of 16 different orders of mangitude.

It would be best if all the data and all the parameters were within 6 orders of magnitude of each other or so.

[George Peterson]

Matt,

The parameters for upper and lower are just the physics of the problem I am examining. Vbi is the built in voltage, and will vary at most from 0.4 to 1.8 eV at the extremes, beta is essentially the carrier concentration and will vary from 1e10 to 1e24 at the extremes. I appreciate you are attempting to help, but that is not something I can alter and maintain a realistic model.

Thanks,

George

[Matt J]

You don't have to measure carrier concentration in number of particles per cubic meter. You could measure it in megaparticles per cubic nanometer, which would result in much smaller numbers. Similarly, you don't have to measure area in square meters, but instead square microns.

I'm making these units up, but the point is that the choice of units is something that you control, and the fitting process is sensitive to it.

[Matt J]

Also, it looks like your mode might be over-parametrized. From what I can see, your model can be simplified to,

CJ_fittype = fittype('A.*(Vbi-Va)).^(-1./(m+2))  )',...
'coefficients',{'Vbi','A','m'},'independent',{'Va'});

where A is just some multiplicative coefficient. If so, there are only 3 parameters. The combinations of area and beta that give rise to a particular A will be infinitely non-unique.

[George Peterson]

Matt,

I appreciate your input, but beta and area should be unique. CJ should be linearly dependent on Area, and should be inversely proportional to beta as 1/(m+2).

But this does not address my real issue, which is why is the fit function not iterating. I can make my initial guess quite close, and get an r-squared value from the GOF of 0.993 or better, or far away, getting an r-squared of 0.4 or less, but something is triggering an exit from the function such that the coefficients never alter from the fitoptions.Startpoint values. And if I do not include the fitoptions.Startpoint values, fit makes a random guess, exits, and provides the calculated GOF at the initial random guess.

Thank you,

George

[Matt J]

The reason it is not iterating is because, over a very broad neighborhood of your StartPoint, your rmse=2.2836e-13 is miniscule and also changes in your parameters result in similarly miniscule changes in rmse. With, default stopping tolerances like CJ_fitoptions.TolFun=1e-6, the code interprets this to mean that you are already at an optimum point and no iterations need be done.

The reason your rmse and its gradients are (artificially) miniscule is, in part, because they are scaled to be on the order of 1e-12. If you take even the simplest step of changing the scale of your datay, as I have been suggesting, you will see the iterations start to move. Below, all I have done is scale both your datay and your model function by 1e11, and as you can see 17 iterations are executed:

load data


CJ_fittype = fittype('1e11*11.8.*8.85e-12.*area./((((m+2).*11.8.*8.85e-12)./(1.602e-19.*beta).*(Vbi-Va)).^(1./(m+2)))',...
'coefficients',{'Vbi','area','beta','m'},'independent',{'Va'});
CJ_fitoptions = fitoptions(CJ_fittype);
CJ_fitoptions.StartPoint = [0.7 5.2e-5 1e19 1];
CJ_fitoptions.Lower = [0.2 5.2e-9 1e10 0];
CJ_fitoptions.Upper = [1.2 5.2e-3 1e24 3];

[curveFit1,curveGOF1,output] = fit(datax,1e11*datay,CJ_fittype,CJ_fitoptions)

curveFit1 = 
     General model:
     curveFit1(Va) = 1e11*11.8.*8.85e-12.*area./((((m+2).*11.8.*8.85e-12)./(1.602e-
                    19.*beta).*(Vbi-Va)).^(1./(m+2)))
     Coefficients (with 95% confidence bounds):
       Vbi =       0.533
       area =   4.551e-05
       beta =       1e+19
       m =      0.9486  (0.9329, 0.9642)

curveGOF1 = struct with fields:
           sse: 3.7896e-04
       rsquare: 0.9999
           dfe: 137
    adjrsquare: 0.9999
          rmse: 0.0017

output = struct with fields:
           numobs: 141
         numparam: 4
        residuals: [141×1 double]
         Jacobian: [141×4 double]
         exitflag: 1
    firstorderopt: 1.9974e-07
       iterations: 17
        funcCount: 90
     cgiterations: 0
        algorithm: 'trust-region-reflective'
         stepsize: 3.4011e-06
          message: 'Success. Fitting converged to a solution.'