Use integral of function in bin instead of value at bin centre for histogram fit

Hi,

My question is about curve fitting to histograms. Is there a way to use the integral of the fitting curve in the bin instead of the value at the centre of the bin during fitting? I think this option is useful when trying to fit a relatively sharply peaked curve to coarsely binned data. I know some analysis toolkits do offer this fit option(1), how about Matlab?

Cheers

(1) E.g. ROOT: see option "I" in https://root.cern.ch/root/htmldoc/guides/users-guide/FittingHistograms.html

[Jeff Miller]

Are you using the distribution fitter app ? It looks like that will let you set bin widths to whatever coarse level you have, and it should fit based on the integrated bin probability.

[Laurent Kelleter]

Hi Jeff,

I'm currently not using any specific tool at all: I'm trying to find out if there is a built-in solution to my problem in Matlab.

The integrated bin probability is the value of the bin, right? I do not expect the fitting to do anything fancier with the binned data apart from using its value in a bin.

The standard way of fitting a curve to binned data is minimising the difference between the value in a bin and the value of the fitting curve at the bin centre. What I want to do is to minimise the difference between the bin value and the integral of the fitting curve in the bin.

Is there any other (built-in) way than coding this manually?

[Jeff Miller]

"The standard way of fitting a curve to binned data is minimising..." That's not my impression, at least if you are fitting a standard parametric distribution.

A minimum chi-square fit is one pretty standard option, and it minimizes the difference between observed and predicted bin probabilities, with predicted bin probabilities based on integrated bin area, which sounds like what you want. You can do that with Cupid, for example.

[Laurent Kelleter]

I think I'm with you for the case when I fit a discrete distribution to discrete (binned) data. But what if I try to fit a continuous disctribution to binned data? Assuming I have a custom (continuous) distribution that cannot be found in the standard libraries. Would I need to convert it to a discrete distribution myself? Or are there some Matlab packages (e.g. CUPID) that do that work for me, taking a continuous distribution as input and fitting the binned data as if the distribution would follow the same binning?

[Jeff Miller]

Cupid can fit continuous distributions to binned data. It's pretty easy if you have a continuous distribution that is already built-in to Cupid. If you want to fit a custom distribution that isn't already there, then you have to make a new distribution class to represent your custom distribution. This is easier than it probably sounds, as long as you have a function to compute either the PDF or CDF of your custom distribution (it runs faster if you have both functions, but either can be computed numerically from the other if you only have one).

What custom distribution do you want to fit?

[Laurent Kelleter]

When you say that CUPID can fit continuous distributions to binned data, does that mean CUPID integrates the continuous distribution in the bin (according to the binned data) and uses that value for fitting? Or does CUPID evaluate the continuous distribution at the bin centre position? This is the crucial point for me.

My background is medical/nuclear physics and I want to fit a depth-dose distribution (Bragg curve) of proton beams with an analytical description.

[Jeff Miller]

Yes, Cupid integrates to get the probability in each bin.

More specifically, each probability distribution object inherits a method called "EstChiSq" that you call with one vector indicating the upper bound of each bin and another vector indicating the probability in each bin. The method adjusts the distribution's parameters to minimize a chi-square statistic reflecting the difference between observed and predicted bin probabilities.

Looking quickly at the analytical description link, I don't immediately see a probability distribution, but this is way out of my area. But if you can write a function to compute the simple or cumulative probability density for the distribution you want to estimate, then I believe it will be possible to estimate parameters using integrated bin probabilities as you say.

[Laurent Kelleter]

Ok, thank you Jeff!

Answer 1

[Star Strider]

See if the fitdist function will do what you want.

[Laurent Kelleter]

Hey,

Thank you for the suggestion, but I don't think so.

I can choose from all sorts of distributions to be fitted to the data but I cannot change the function that is being minimised. It says that it is using maximum likelihood estimation for fitting. I assume it takes the value of the distribution at the bin centre. I want it to use the integral of the distribution between the bin edges instead. I can't see a way of implementing this with fitdist.

[Star Strider]

That is the only built-in function that will automatically fit a histogram. You will need to write your own objective function and use one of the nonlinear parameter estimation routines to estimate the parameters for an arbitrary function.

[Laurent Kelleter]

Alright, thank you for your help!

[Star Strider]

My pleasure!