How to choose the parameters of vision.For​eGroundDet​ector ?

Hi everyone,

I am trying to use the ForegroundDetector of the computer vision toolbox in order to detect moving objects in a grayscale image.

However, I encounter some trouble to choose the right parameters of the function, that is to say: - number of gaussians - number of training frames - learning rate - initial variance.

Is there any tip to determine the best parameters, or is it only empirical depending on the images ?

I feel like the crucial parameter is the initial variance for me. But I am not really familiar with statistics and image processing, so that I don't really understand well what these parameters are...

Thanks for any help.

Answer 1

[Dima Lisin]

Hi Benjamin,

Initial variance is indeed crucial, and it depends on the range of pixel values in your video. If the data type of your frame is 'double' or 'single', with the pixel values ranging between 0 and 1, then you should use the default value of InitialVariance, which is (30/255)^2. If your frame is of type 'uint8', with the values ranging between 0 and 255, then you should set InitialVariance to 30^2.

The other parameters depend on the content of your video. You can reduce NumGaussians if you have nice static background, as in an indoor environment. On the other hand, more Gaussians help when your backround is non-stationary. Examples would be outdoor scenes with rustling leaves or the surface of a sea or a lake.

LearningRate controls how quickly your background model adapts to changes in the background. If the LearningRate is too high, then slow moving objects may become part of the background. If the LearningRate is too low, then your background model will not be able to adopt to lighting changes.

[Benjamin]

Thank you Dima for your answer.

The videos I am working on are showing the movement of a closed shape into a tube under a microscope, so that there is little light variations. The contrast is only visible on the contour of the shape which is darker, and I want to extract it, with high accuracy. Thus, I want the detection to find a closed object as much as possible. I use GMM because the tube involves high contrast that is part of background, and classical methods don't work.

I have 'uint8' data, so that I chose InitialVariance to 900. If I understand well, the more the variance is, the easier pixels will belong to background ? I noticed that with a low variance, there is a lot of noise, but my image is better detected (more pixels are detected). This is crucial for me as my images are closed contours I want to extract. If pixels from my contour (foreground) are detected as background, there are holes in my contour, that I cannot fill (or have to interpolate, but this does not work well when high curvatures). Then I am not sure if I should keep the default value and find the right values for the other parameters, then post-processing, or if I should choose the variance that seems the best (about 45 to 90 in my case), then choose the other parameters and post-process ?

I also find that 25 to 30 gaussians give better results. So, the more background variations, the more gaussians we input in the model, is it ?

My videos are between 100 and 500 images long. What training number should I choose ? I thought of about one third of the images. Is it ok to take the whole video as training ? What is the difference ?

The default value of the learning rate is 0.005. What is a correct range ? From what you explained, I would like to raise it.

What about the minimum background ratio ? If I want to keep the most pixels possible in the foreground, I should use a high background ratio, am I right (meaning I will have more noise) ?

Thanks again.

[Stephan Zimmer]

Hey Benjamin,

have you found a solution to all of your questions? I am writing right now my master thesis and have the same problems as you had. I would be very happy if you could share your experience!

Best regards,

Stephan

Answer 2

[Ahsan Malik]

Hello everyone, I want to ask that if there is any way to train foregrounddetector on some other video containing background and then test on the video containing same background with foreground objects. Or can we take 50 training samples other than the start .

[Mr.Alb]

Yes, I'm interested too. Thank you for asking.

[Jonathan Meerholz]

I know this is a late response, however I belive it is still useful for others.

As far as I understand , say you setup your detector as follows:

detector = vision.ForegroundDetector(...
            'NumTrainingFrames', 150, ...
            'InitialVariance', 30*30);

Then the next 150 calls of the detector will be used to train the detector/ background model. Thus, you could use 150 frames of a training video to train then use it on actual footage after:

video_training = VideoReader("video_training_path"); %Path to training video
video_input = VideoReader("video_input_path"); %Path to input video

% Training Loop
for i = 1:150
    frame_current = readFrame( video_training ); %Read Training frame
    bw_training = detector(frame_current); %Train detector
end

% Detection Loop 
while hasFrame(video_input)
    frame_current = readFrame( video_input ); %Read actual video input
    bw_mask = detector(frame_current); %Use trained detector
end