Find local minima in x,y array. Need for careful elimination of Johnston noise in images.

I am trying to eliminate noise in images similar to the following so I may use them for VERY sensitive and precise pixel analysis later:

Note that the real image is 15MB at a much higher resolution.

My current image processing algorithm:

• reading all the images into a cell array

• converting each image into 3264x4928x3 doubles 

• summing their components into a 3264x4928x1 double (this is because I only care about RGB not saturation or whatever like in im2gray) ~~~ 'im3{count}=im2{count}(:,:,1)+im2{count}(:,:,2)+im2{count}(:,:,1)'

• adding each image in quadrature into a single 3264x4928x1 double

From here I want to eliminate JUST the Johnston Noise (red haze) in the image, but leave the noise from the dead pixels so I can eliminate them by comparing two images and subtracting the similarities to make a third. The dead pixel noise tends to be only blue, green, or red, but NOT all three so their intensity using my algorithm will be around 1/3 of the maximum.

I then took my 3264x4928 matrix and uniformly scaled it so the highest intensity pixel was at intensity level 1000, and then rounded using ceil(x).

I then used a loop to scan through and count the number of pixels at each intensity value. I expected two roughly bell curve shapes. One for the Johnston noise and the other for the dead pixel noise, which is what I found:

The x-axis is the rounded intensity level of each pixel The y-axis is a count of the pixels in the image closest to each intensity level.

Exactly as expected. a peak at the low end for the Johnston noise and another peak at about 1/3 max intensity.

I am wondering if I should:

a) Scan the slopes of the linear fits of the 50 points on each side of each point between 200 and 450 and find the point where the left slope is negative, the right slope is positive AND the slope change is the greatest.

b) Use some function (which I don't know about) to do it for me (whose precision and accuracy I worry about).

I'm not sure which would give me better results so I'm hesitant at this point. Then from here I can go back to the original image, eliminate everything under the threshold that I just found, and compare two images to find the dead pixels.

Does anybody have any suggestions on finding this minimum where I should cut everything before it off? Or should I have an entire different approach to this whole problem in general.

Answer 1

[Kyle Bryenton]

I resolved this question myself and I figured I would answer my own question for anybody else that came across this at any point.

I realized the data I was using was incorrect, thus why there is no bell curve for the thermal noise above. With new data, I arrived with the following intensity histograms:

My first note is that, Johnston noise wasn't really what that haze should be called, honestly I should really be calling it thermal noise, which has been tested multiple times and shows very strong correlation.

Second, I thought about only getting rid of the noise on the red layer, but I found out the noise came in as ABOUT 60% red, 35% blue, and 5% green. I also noticed around the edge of the image, and especially in the corners the thermal noise got gradually stronger, for now I decided to crop the outer 10% of the image, later I will revise this with a filter so I don't have to crop it.

Third, my efforts to somehow spread the peaks out so I would have more data to work with, never really got very far.

• for n ϵ {2.0, 3.0, 4.0...}, I tried creating a new matrix as: Data.*n
• for n ϵ {1.5, 2.0, 2.5...}, I tried creating a new matrix as: Data.^n
• for n ϵ {1.5, 1.7, 2.0...}, I tried creating a new matrix as: n.^(Data)

The problem was as I spread the peaks apart I was also spreading the second peak from the max value by the same scale. This was disastrous when using the data as an exponential or even raising the data to an exponent. Eventually I ended up with a graph that peaked at 3/1000 and only had about 50 pixels between 5 and 1000 on the histogram. The best solution I came up with was to add the colour intensities in quadrature, and average each picture with the rest normally. The layers should be added in quadrature because the thermal noise curve spreads to about 30% before it is about the same scale as the bad pixel noise. When you add in quadrature the thermal noise stays in the first part of the graph but it groups the bad pixels together more and it moves the second peak from about 33% of the max intensity to about 50%. Theoretically it looks like this:

Fourth, with the new pictures the histogram seemed to really fit the standard distribution curve nicely. One very high bell curve for the thermal noise, overlapping a second very small bell curve for the bad pixels. Rather than finding the minimum in my data as I mentioned above, I realized there is only a single correct answer of how to go about finding the point where I should place my threshold.

• If you have two overlapping bell curves: f(x) & g(x)
• To remove as many volume elements of the first as possible: ∫ f(x) dx
• But leave as many from the second as possible: ∫ g(x) dx,
• A good approximation is: x ϵ (-∞, f(x) ∩ g(x)]

I am currently working on this approach so I don't have an overlap of the curves I found to show you, I'm not sure how close I can get the match to be because of all the noise but I'm trying a few peak preserving smoothing algorithms before I try to fit the bell curves. It's only an approximation because the higher curve will normally be decaying much slower at the intersection than the smaller one, and it also varies with the standard deviation and the distance between the means as well. I assume some perfect mathematical solution exists or some loop could be created to actually find the optimal solution but I couldn't find one with a brief search so I will revisit this later.

The curves above were generated with the following algorithm:

• The images were imported
• The outer 10% was cropped out of the image
• The images were converted to doubles
• The red, green, and blue layers were added in quadrature. This is the most important part.
• The the mean of each individual pixel intensity was taken for all the images
• The intensities were scaled to 1000 and rounded up

If this isn't clear enough or if my logic has failed me in any way comment on this answer or send me an email and I will be happy to help out or revise my solution, however I shall mark this solution as the answer to the question for the time being.

Answer 2

[Image Analyst]

If you want to "Find local minima in x,y array" then use imregionalmin() in the Image Processing Toolbox. It finds all regional mins, no matter how bright or dark. If you only want ones less than some value, then you can AND it with a thresholded image that gets mins in the intensity range you want.

[Kyle Bryenton]

Sorry this isn't what I meant by minimum. I'm looking for a minimum between intensities. For example an image such as this

The imregionalmin() function would say the black hole is the minimum and eliminate it (I think... right?).

What I am trying to find the minimum of is the pixel count at various intensities.

So for this image black would be at a maximum, because the most of the pixels in this image are black (as in my image above) the minimum in this image would be on the red line. that particular shade has the least amount of pixels that correspond to it, or the pixel count for that shade is less than any of the other shades in this image.

In my image above I'm trying to find the intensity value that best describes the "transition" from the brighter dead pixel "sploches", and the red haze that is all throughout the image due to Johnston noise, then I would just say every pixel below that intensity is set to black.

[Image Analyst]

Get the histogram

[pixelCount, grayLevels] = imhist(grayImage, 256);

This gives you the count at every intensity, not just "various" intensities. Then find the max count(s):

[maxCount, indexOfMaxCount] = max(pixelCounts);

I'm not clear on what you want to do after that.

[Kyle Bryenton]

I don't like the way MATLAB does its greyscale conversion. It doesn't keep true to the colors so I did it my own way.

I also added the intensities of 10+ images together in quadrature after converting them to a double format.

So those graphs above are exactly that, the histograms. Only with more precision.

The way I ended up setting up a loop that started at 200, and went to 350

It made an interval of the 20 points to the left of the current one, and another interval of the 20 points to the right. I then found the slope of the linear fit of each and found the difference.

When the loop exited I had 151 differences between the left and right slopes for each point; the largest difference was taken as the minimum.

.

So I used that as the threshold, and eliminated everything below it. The following are two images of a before and after zoomed into the noisiest spot on the original image.

Before:

After:

as you can see the noise below that threshold is all gone, no more red haze.

I'm just wondering if there is a better way to find that minima than to scan the slopes on each side of each point and take the maximum difference... or if I should be taking an entire different solution to this problem.