How to use image background subtraction in video

I want to background subtraction in the video.

The problem is that the first frame of some videos is not empty. I mean, there is a person in the first frame of video.

So when I use background subtraction, the result is not as good as below picture.

Do you have a great idea to solve it?

I think that I can use background image.

I am using this dataset http://www.wisdom.weizmann.ac.il/~vision/SpaceTimeActions.html

(Bend dataset)

I want to get this kind result. (the dataset is different)

(Side dataset)

clear all
close all
%// read the video:
reader = VideoReader('shahar_bend.avi');
vid = {};
while hasFrame(reader)
    vid{end+1} = im2single(readFrame(reader));
end
%// simple background estimation using mean:
bg = mean( cat(4, vid{:}), 4);
%// estimate foreground as deviation from estimated background:

fIdx = 40; %// do it for frame 43
fg1 = sum( abs( vid{fIdx} - bg ), 3 ) > 0.25;
fg2 = imresize(fg1, 0.5);

figure;
subplot(141); imshow( bg ); 
subplot(142); imshow( vid{fIdx} );
subplot(143); imshow( fg2 );

Answer 1

[Raynier Suresh]

To do background subtraction but without knowing the background you could try image segmentation to remove the background. Many different image segmentation algorithms are available in MATLAB.

For Image Segmentation in MATLAB you could refer this link :

https://www.mathworks.com/help/images/image-segmentation.html

The “vision.ForegroundDetector” might help you to get the foreground object using the “Gaussian Mixture Models” .

For “vision.ForegroundDetector” in MATLAB you could refer this link:

https://www.mathworks.com/help/vision/ref/vision.foregrounddetector-system-object.html

Referring to the following links might be helpful:

Image Segmentation using K-Means Clustering:

https://www.mathworks.com/help/images/color-based-segmentation-using-k-means-clustering.html

Gray Image Thresholding : https://www.mathworks.com/help/images/ref/graythresh.html

[Kong]

Hello. Thank you so much!

vision.ForegroundDetector is so amazing!

Could you let me know how to get several images inbox as a matrix?

I want to save these images as matrix.

I want to get this image.

videoSource = VideoReader('shahar_run.avi');
detector = vision.ForegroundDetector(...
       'NumTrainingFrames', 5, ...
       'InitialVariance', 30*30);
blob = vision.BlobAnalysis(...
       'CentroidOutputPort', false, 'AreaOutputPort', false, ...
       'BoundingBoxOutputPort', true, ...
       'MinimumBlobAreaSource', 'Property', 'MinimumBlobArea', 250);
shapeInserter = vision.ShapeInserter('BorderColor','White');
videoPlayer = vision.VideoPlayer();

while hasFrame(videoSource)
     frame  = readFrame(videoSource);
     fgMask = detector(frame);
     bbox   = blob(fgMask);
     out    = shapeInserter(fgMask,bbox);
     videoPlayer(out);
     pause(0.1);
end