Got Questions? Get Answers.
Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Noise from integer grayscale image

Subject: Noise from integer grayscale image

From: Eric Diaz

Date: 24 Mar, 2011 19:23:05

Message: 1 of 15

I've copied my original post into this one, because the data on the original thread was a bit much to scroll through every time. Please see data on original thread though, so you can see the population of noise I am dealing with.
===================================================
(original thread with data pasted in)
http://www.mathworks.com/matlabcentral/newsreader/view_thread/304988
===================================================
On Mar 23, 11:03 pm, "Eric Diaz" <eric.d...@gmail.com> wrote:
> Given: UINT8 image, Areas of signal and areas of no signal, noise
>
> Background: If one thresholds an image to separate signal from background, then proceeds to only use pixels from the background for analysis, then the noise from those pixels is discrete.
>
> Question #1: If one attempts to fit a distribution to the discrete and positive noise population (think histogram fit), then which distribution should be used? A continuous one or a discrete one?
>
> Question #2: If one knows that a particular continuous distribution should fit the data well, then is there a way to interpolate between discrete density bins?
>
> Thanks in advance,
>
> Eric

-------------------------------------------------
1. "Fits" are generally continuous because they're based on some
model, or assumption you've made. I didn't plot your data so I don't
know what kind of distribution should be used. Common ones are
normal, log normal, Poisson, uniform, and speckle, or others. Depends
on the cause of the noise.
2. Yes.

Subject: Noise from integer grayscale image

From: Eric Diaz

Date: 24 Mar, 2011 19:44:20

Message: 2 of 15

> -------------------------------------------------
> 1. "Fits" are generally continuous because they're based on some
> model, or assumption you've made. I didn't plot your data so I don't
> know what kind of distribution should be used. Common ones are
> normal, log normal, Poisson, uniform, and speckle, or others. Depends
> on the cause of the noise.
> 2. Yes.

Hi Image Analyst,

Your expertise is clearly something I would like to leverage.

Given:
1) Yes, fits are generally continuous
2) dfittool, Distribution Fitting Tool, provided with Matlab
3) Data, provided in the original post, which is a background noise squared distribution for uint8 image.
4) Cause of noise is difficult to explain, however, I'm willing to give it a try if you would understand it. Suffice it to say, the distribution of squared background noise is known to be modeled best by a central chi squared distribution.
5) dfittool, when used to try and fit the data, provides a selection of models to choose from (i.e., normal, log normal, Poisson, gamma, etc.), however, none of them are able to fit the data even within a reasonable goodness of fit.
6) The parameters of interest for a central chi squared distribution can be computed after some algebraic manipulation of parameters found from gamma distribution fitting. This can be easily shown.

Problem:
1) Is the reason the models are not working to fit the data because the data is discrete? I mean is the issue of analog to digital conversion and strictly discrete integer values for noise confounding the modeling problems?
2) Is filling empty bins of integers using interpolation the approach to take to fix the issue?

Thanks in advance,

Eric

Subject: Noise from integer grayscale image

From: ImageAnalyst

Date: 24 Mar, 2011 21:30:10

Message: 3 of 15

What empty bins? From the histogram? Why are there empty bins? Did
you scale the image or something?

Subject: Noise from integer grayscale image

From: Eric Diaz

Date: 25 Mar, 2011 18:36:04

Message: 4 of 15

What empty bins? From the histogram? Why are there empty bins? Did
you scale the image or something?
====================================================
====================================================

Hi ImageAnalyst, it is clear that you need answers to this question before moving on to the other parts of the problem. So, let's try a thought experiment.

Given:
1) uint16 image with areas of signal and areas of no signal
2) the uint16 image also has areas of noise
3) the image is thresholded and the areas of signal are separated from the areas of no signal
4) the areas of no signal are used to characterize the noise
5) this noise is discrete and positive, i.e., with intensity values like 1,2,3,4,5,6,7,8,...
6) when this population of noise is population of noise is plotted on a histogram, there are several different outcomes, depending on how binning is determined:
a) just use 'hist' default binning
b) calculate the "optimal number" of bins using various algorithms out there, such as friedman-draconis, scott's, or a least squares method
c) set the binning to be centered on integers. This is what I did and is evident from the previous post, where I included the command:
"
figure;hist(data ,1:4500)
===================
data is below
==================
"
7) Now, depending on how binning is determined, there will be a different shapes to the distribution, however, given that I am using each integer as its own bin, the shape of the distribution is exactly how I want it to be, pure representation of each discrete noise value.
8) Now, take this noise, as described in #5, and square it. This will take intensity values, like 1,2,3,4,5,6,7,8..., and turn them into values, like, 1,4,9,16,25,36,49,64,...
9) If we repeat step #6, there will also be different outcomes depending on how binning is determined:
a) just use 'hist' default binning
b) calculate the "optimal number" of bins using various algorithms out there, such as friedman-draconis, scott's, or a least squares method
c) set the binning to be centered on integers
d) set the binning to be centered on squared integers. This is what I did...figure;hist(data ,[1,4,9,16,25,36,49,64,...])

10) If one uses the binning choices 9d, then there are no "empty bins" per se, however, clearly when one thinks about the population from a discrete non-squared integer perspective, there are "empty bins".

Answer to your question = #10. The reason the noise population is squared is because it is known what the distribution of that squared population is, as mentioned previously.

Problem: the difficulty lies in the fact that all attempts to used the known distribution fail when using fitdist or dfittool, provided by matlab. Why is it failing? Is it because of these "empty bins" or the discreteness of the population? Please try plotting the data, so that you can see for yourself. dfittool, is a great way to visualize the data and do fitting at the same time, if you have it. You can also control the type of binning, however I feel that it may be irrelevant to the actual MLE fitting technique used.

====================================================
repeat of last post is below...
====================================================

Given:
1) Yes, fits are generally continuous
2) dfittool, Distribution Fitting Tool, provided with Matlab
3) Data, provided in the original post, which is a background noise squared distribution for uint8 image.
4) Cause of noise is difficult to explain, however, I'm willing to give it a try if you would understand it. Suffice it to say, the distribution of squared background noise is known to be modeled best by a central chi squared distribution.
5) dfittool, when used to try and fit the data, provides a selection of models to choose from (i.e., normal, log normal, Poisson, gamma, etc.), however, none of them are able to fit the data even within a reasonable goodness of fit.
6) The parameters of interest for a central chi squared distribution can be computed after some algebraic manipulation of parameters found from gamma distribution fitting. This can be easily shown.

Problem:
1) Is the reason the models are not working to fit the data because the data is discrete? I mean is the issue of analog to digital conversion and strictly discrete integer values for noise confounding the modeling problems?
2) Is filling empty bins of integers using interpolation the approach to take to fix the issue?

Subject: Noise from integer grayscale image

From: ImageAnalyst

Date: 25 Mar, 2011 21:15:15

Message: 5 of 15

Eric:
Sorry, I do not have difftool - it is not supplied with base MATLAB.
It comes with the Statistics Toolbox. But I would think a curve
fitting tool should be able to operate on x and y coordinate pairs and
not require that the x be evenly spaced. Maybe call the Mathworks for
help if no one here can. Sorry i can't, but like I said, I don't have
that tool.
ImageAnalyst

Subject: Noise from integer grayscale image

From: Eric Diaz

Date: 25 Mar, 2011 22:11:05

Message: 6 of 15

Image Analyst:

Well, I wouldn't have thought so either, however both dfittool & fitdist are failing pretty miserably. It's unfortunate that dfittool is not a base part of MATLAB, as fitting a statistical distribution is such a basic yet powerful concept. Perhaps someone should make a way to use/call R through MATLAB, so that the incredible statistical power of R, could be used without having to spend lots of $$ on extra toolboxes.

While I know that this is a user forum, if ANYONE at MATLAB is reading this, please help.

Eric

Subject: Noise from integer grayscale image

From: Eric Diaz

Date: 26 Mar, 2011 21:55:04

Message: 7 of 15

If anyone can help with this challenging problem, I would really appreciate it.

Subject: Noise from integer grayscale image

From: Eric Diaz

Date: 29 Mar, 2011 01:33:03

Message: 8 of 15

"Eric Diaz" wrote in message <imlnbo$7p1$1@fred.mathworks.com>...
> If anyone can help with this challenging problem, I would really appreciate it.

Are there any experts on distribution fitting or noise fitting or maximum likelihood estimation of distribution parameters that can help with this problem? My data is posted if it needs to be looked at. I was using the statistics toolbox programs dfittool for a good visual representation of the distribution and the poor job of fitting done by the available distributions of choice. Somebody please help.

Subject: Noise from integer grayscale image

From: Eric Diaz

Date: 29 Mar, 2011 19:41:04

Message: 9 of 15

If there is anyone that can help with this, please help.

Subject: Noise from integer grayscale image

From: Eric Diaz

Date: 30 Mar, 2011 01:45:21

Message: 10 of 15

Anyone???

Subject: Noise from integer grayscale image

From: ImageAnalyst

Date: 30 Mar, 2011 01:55:04

Message: 11 of 15

On Mar 29, 9:45 pm, "Eric Diaz" <eric.d...@gmail.com> wrote:
> Anyone???

--------------------------------------------------------------------------------
No one is answering you (probably) because of the subject line. I
know there are fitting experts here. Why don't you start a new thread
with a subject line like "Need help fitting chi-square distribution to
non-uniformly spaced data" or something like that? And keep it simple
by eliminating other stuff, like the image noise stuff, that is not
essential to the problem.
ImageAnalyst

Subject: Noise from integer grayscale image

From: Eric Diaz

Date: 31 Mar, 2011 21:14:05

Message: 12 of 15

ImageAnalyst <imageanalyst@mailinator.com> wrote in message <2c0e2c05-b8a8-4f2e-938b-b444c80cdeb8@k22g2000yqh.googlegroups.com>...
> On Mar 29, 9:45 pm, "Eric Diaz" <eric.d...@gmail.com> wrote:
> > Anyone???
>
> --------------------------------------------------------------------------------
> No one is answering you (probably) because of the subject line. I
> know there are fitting experts here. Why don't you start a new thread
> with a subject line like "Need help fitting chi-square distribution to
> non-uniformly spaced data" or something like that? And keep it simple
> by eliminating other stuff, like the image noise stuff, that is not
> essential to the problem.
> ImageAnalyst

Well, that didn't really work. So I'm back to my original post. Isn't there anyone that can help?

Subject: Noise from integer grayscale image

From: Eric Diaz

Date: 1 Apr, 2011 18:30:22

Message: 13 of 15

I can't believe there is no one that can help.

Subject: Noise from integer grayscale image

From: ImageAnalyst

Date: 1 Apr, 2011 18:35:12

Message: 14 of 15

On Apr 1, 2:30 pm, "Eric Diaz" <eric.d...@gmail.com> wrote:
> I can't believe there is no one that can help.
----------------------------------------------------------------------------
There is. Call the Mathworks. That's what you pay your maintenance
fees for. I call them all the time - might as well get my money's
worth.

Subject: Noise from integer grayscale image

From: Eric Diaz

Date: 4 Apr, 2011 00:55:14

Message: 15 of 15

oh, great, let me just call Mathworks. see if they can help. Um...no thanks. I think I'm already disappointed enough with how poorly the statistics toolbox software works.

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us