Image Segmentation Bounding Box

I have a image that contains the topview of an animal. The objective is to isolate the animal from the image. The original image can be seen in the left upper picture form the following figure:

I do some preprocessing steps that lead to the image (inverseBW_filtered) shown in the right lower corner. Using this image, I run the following line of code:

stats = regionprops(inverseBW_filtered, 'BoundingBox', 'Centroid');

Sadly, just one bounding box is found (plotted as the red box). I would have expected to get a few boxes at least with one of them surrounding the animal. Can I get some suggestions how to make sure that the animal gets segmented using regionprops (or any other technique)?

[Image Analyst]

That does not look like an original image in the upper left. At the very least it's pseudocolored. Do you have an image of hte background without the animal in it that we can subtract using imabsdiff?

Don't invert your mask. You want the animal/foreground to be white like in the lower left image. Because you inverted the image, that's why your bounding box is the whole entire image.

[Henk-Jan Ramaker]

The image is originally from a 3D-depth camera, so you are right about the pseudocoloring. They represent heights.

I like your conceptual idea a lot about the image without an animal. If I get you right:

Picture2 (scenery with an animal) - Picture1 (scenery without animal) = Picture3 (isolated animal).

Picture3 provides good means for further analysis. Unfortunately, I do not have such a Picture1 at this moment.

I now filter Image | BW with a combination of "imclearborder" and "bwareaopen". This results in Image | BW Filtered. After applying the blob analysis (or regionprops) I get the following:

[Pratham Shah]

Great!

I think this is what you wanted, right?

Answer 1

[Pratham Shah]

Hi!

As @Image Analyst rightly mentioned, don't use inverted image for bounding box. Use Image | BW.

To get the bounding box around the animal; If you know the area animal the animal will cover use blobAnalysis function. To remove the white pixels from bottom-right corner you can use 'imclearborder' morphological function.

newBW=imclearborder(ImgBW,8);  %You can change '8' depending on your application
blob = vision.BlobAnalysis('BoundingBoxOutputPort', true,...
        'AreaOutputPort', false, 'CentroidOutputPort', false, ...
        'MinimumBlobArea', 50);
box= step(blob, newBW);
Output = insertShape(Image, 'Rectangle', box, 'Color', 'red','Linewidth',2);

[Henk-Jan Ramaker]

Thanks Pratham, "imclearborder" certainly helps to clean up the picture!

Answer 2

[Image Analyst]

If you don't have a background image, you can create one by taking the mode of every pixel over all frames in the video (or as many of them as you can fit in memory).

You can even get an estimate for the background if you can hand trace the animal in one frame. Then you can use regionfill to estimate the background underneath the animal. Demos for hand tracing regions are attached.\

It's kind of weird that the height of the animal is both higher and lower than the background at the left of the scene. Do you have a visible light image of the scene you can share? The animal looks like it's shaped like a trough.

[Henk-Jan Ramaker]

Suppose I create these "empty" scenes for a number of pictures where the animals have been removed. How can I use the distrubution of colors from these "empty" pictures wisely to preprocess a new picture (that shows this scenery + animal)?

By the way, the animal is surrounded by a fence. This also shows as a "height". The 3D-depth camera is based on IR technology. As you can see, it's not always delivering up to expectations.

[Image Analyst]

If you have an empty/no-animal image, then you can find the animal by using imabsdiff followed by thresholding.

diffImage = imabsdiff(emptyFrame, currentFrame);
mask = diffImage > someValue; % Threshold to find "tall" things in the scene.
% Take largest blob only.
mask = bwareafilt(mask, 1);