Measuring the width and hight for smallest Bounding Boxes of detected objects

Hi guys,

Kindly looking for Measuring the width and highth for the smallest Bounding Boxex of detected object in the following code

load('Detector.mat');
vidReader = VideoReader('vs_002_00.avi');
vidPlayer = vision.DeployableVideoPlayer;
i = 1;
results = struct('Boxes',[],'Scores',[]);
while(hasFrame(vidReader)) 
I = readFrame(vidReader); 
 % PROCESS
    [bboxes, scores, label] = detect(detector,I,'MiniBatchSize', 128);
    
 % Select strongest detection 

% New - Find those bounding boxes that surpassed a threshold
T = 0.5; % Define threshold here
idx = scores >= T;

% Retrieve those scores that surpassed the threshold
s = scores(idx);

% Do the same for the labels as well
lbl = label(idx);

bboxes = bboxes(idx, :); % This logic doesn't change


for ii = 1 : size(bboxes, 1)
    annotation = sprintf('%s: (Confidence = %f)', lbl(ii), s(ii)); % Change    
    I = insertObjectAnnotation(I, 'rectangle', bboxes(ii,:), annotation); % New - Choose the right box
end
boundingBoxArea = prod(boundingBox(3:4));

step(vidPlayer,I);
 i = i+1;
end
 results = struct2table(results);
 release(vidPlayer);

[Guillaume]

Is the variable boundingBox defined somewhere in your code? It's not clear what its relationship is to bboxes.

You haven't actually asked a question, so I'm not exactly sure what you want.

[Abdussalam Elhanashi]

Guillaume yes what i mean by bboxes is the boundingBox that are used for object detection in fact there are many objects detected in the video and i want to measure the hight and width for the smallest bboxes of these detected objects in this code

Answer 1

[Guillaume]

For the area of Bounding Boxes what is the unit used?

pixel squared. If you want to convert to physical unit (e.g. ) you need to know the scale of your images.

You still haven't explained where boundingBox come from. As far as I can tell, your code uses the variable before defining it.

Using the bboxes variable, this is how you would find the bounding with the smallest area:

bboxes = bboxes(idx, :); % This line from your code. Unchanged

bboxesarea = prod(bboxes(:, 3:4), 2);  %calculate the area of all the bounding boxes by multiplying height by width. produces a column vector
[smallestarea, boxindex] = min(bboxarea);  %get index of smallest bounding box (and area if you need it).
%optional: label the bounding box in red
I = insertObjectAnnotation(I, 'rectangle', bboxes(boxindex,:), sprintf('%s: (Confidence = %f), Area (pixel^2) = %d', lbl(boxindex), s(boxindex), smallestarea), 'Color', 'red');

[Abdussalam Elhanashi]

Guillaume

Thanks for your answer Iam using R-CNN detector and ground truth labeller app so durin training i used bboxes for lebling ROI of features in training data

just what lastly i need how to make record for all bounding boxes detected with values of pixel square during the whole run of this code

[Guillaume]

bboxesarea contain the area of all the bounding boxes that you've kept.

[Abdussalam Elhanashi]

Hi Guillaume

It gives only the last detected bounding box from the video but not history for all bounding boxes detected with pixel squared measurment

I need the whole history of all detected boxes with pixel squared

i attached image captured from Matlab

Answer 2

[Image Analyst]

If you want boxes aligned with the image edges, then use regionprops() and ask for 'BoundingBox'.

If you want boxes at any angle, use bwferet().

Or you could use John's program. https://www.mathworks.com/matlabcentral/fileexchange/34767-a-suite-of-minimal-bounding-objects

[Abdussalam Elhanashi]

Hi Image Analyst

I want to measure box of detected object ( width and hight ) in pixel

image as example attached to this comment

so How can be implemented in my code above ?

[Image Analyst]

I appreciate getting the original RGB image but where is your code and your segmented (binary) image?

Somehow you must have segmented that image to get a pixel map of the "detected object" pixels. I have no idea what object in that photo you detected.

Or you might have used Deep Learning to find something in the image, like orange blobs or rectangular regions or something. I have no idea how you found your "detected object" or even what the "detected" object is.

[Abdussalam Elhanashi]

Hi Image Analyst

Thanks for your reply

Herein the code Iam using

load('Detector.mat');
vidReader = VideoReader('vs_002_00.avi');
vidPlayer = vision.DeployableVideoPlayer;
i = 1;
results = struct('Boxes',[],'Scores',[]);
while(hasFrame(vidReader)) 
I = readFrame(vidReader); 
 % PROCESS
    [bboxes, scores, label] = detect(detector,I,'MiniBatchSize', 128);
    
 % Select strongest detection 
% New - Find those bounding boxes that surpassed a threshold
T = 0.5; % Define threshold here
idx = scores >= T;
% Retrieve those scores that surpassed the threshold
s = scores(idx);
% Do the same for the labels as well
lbl = label(idx);
bboxes = bboxes(idx, :); % This logic doesn't change
for ii = 1 : size(bboxes, 1)
    annotation = sprintf('%s: (Confidence = %f)', lbl(ii), s(ii)); % Change    
    I = insertObjectAnnotation(I, 'rectangle', bboxes(ii,:), annotation); % New - Choose the right box
end
boundingBoxArea = prod(boundingBox(3:4));
step(vidPlayer,I);
 i = i+1;
end
 results = struct2table(results);
 release(vidPlayer);

I am using deep learning for detecting smoke

in fact regardless what is the object is to be detected what i need is to measure the width and hight for the bounding box appeared in the video even in milimeters

[Guillaume]

@ImageAnalyst, in Abdulassam code, the segmentation is performed by the detect function of the computer vision toolbox. This function directly returns the bounding boxes of the detected objects (according to a preconfigured detector).

[Image Analyst]

That is not always the smallest, though I'm not convinced he actually needs the smallest. Using one aligned with image edges may be good enough.

But you're right about him not defining boundingBox (maybe put that as a separate answer). Is that the overall bounding box of all the smaller, interior bounding boxes?

Does he want the total area of all bounding boxes? So maybe this:

boundingBoxArea = 0;
for ii = 1 : size(bboxes, 1)
    annotation = sprintf('%s: (Confidence = %f)', lbl(ii), s(ii)); % Change    
    I = insertObjectAnnotation(I, 'rectangle', bboxes(ii,:), annotation); % New - Choose the right box
    boundingBoxArea = boundingBoxArea + bboxes(ii, 3) * bboxes(ii, 4);
end
% Don't use this line boundingBoxArea = prod(boundingBox(3:4));

No idea what he wants.

[Abdussalam Elhanashi]

hI Image Analyst Thanks For your reply

What i mean i attached an other image as example if during my detection many bounding boxes appeared i want to measure the smallest bounding box area in the video

For the area of Bounding Boxes what is the unit used ?

[Image Analyst]

So index it and take the min

boundingBoxArea(ii) = bboxes(ii, 3) * bboxes(ii, 4);

then after the loop

minBoxArea = min(boundingBoxArea)

[Abdussalam Elhanashi]

Image Analyst

Hi Again Is it possible to get values of width and hight detected Bounding Box separatly

Best,

[Guillaume]

As documented bboxes(:, 3) is the width of the bounding boxes, and bboxes(:, 4) is the height.

[Abdussalam Elhanashi]

Hi Guillaume

For this code how can i know them separatly

and appeared in image Width x Height near the Bounding box

bboxes = bboxes(idx, :); % This line from your code. Unchanged
 
bboxesarea = prod(bboxes(:, 3:4), 2);  %calculate the area of all the bounding boxes by multiplying height by width. produces a column vector
[smallestarea, boxindex] = min(bboxesarea);  %get index of smallest bounding box (and area if you need it).
%optional: label the bounding box in red
I = insertObjectAnnotation(I, 'rectangle', bboxes(boxindex,:), sprintf('%s(Confidence=%0.02f,region size Area=%d', lbl(boxindex), s(boxindex), smallestarea));

[Image Analyst]

Maybe try

bboxes = bboxes(idx, :); % This line from your code. Unchanged
bboxesarea = prod(bboxes(:, 3:4), 2);  %calculate the area of all the bounding boxes by multiplying height by width. produces a column vector
[smallestarea, boxindex] = min(bboxesarea);  %get index of smallest bounding box (and area if you need it).
%optional: label the bounding box in red
textLabel = sprintf('%s(Confidence = %0.02f, Area=%d pixels = %d wide by %d tall', ...
    lbl(boxindex), s(boxindex), smallestarea, bboxes(boxindex, 3), bboxes(boxindex, 4))
I = insertObjectAnnotation(I, 'rectangle', bboxes(boxindex,:), textLabel);

[Abdussalam Elhanashi]

Dear Sirs

Guillaume Image Analyst

Thanks for your help just i have last question so in case i want to use width and height like for example measuring buonding box , we can then say (width 27 x Height 42 ) and the unit is pixel

Best,

[Image Analyst]

Yes, but you have to say for width and height if you're using whole pixels, or the distance from pixel center to pixel center. For example, in this vector [0 1 1 1 0] what would you say the "width" of the 1 region is? Is it 3 or 2? It's 3 if you're considering whole pixels, but 2 if you go from the center of the left pixel to the center of the right pixel. Either could be "right" -- it really depends on the physics and optics of your image capture situation. And MATLAB functions aren't always consistent on which they use. For example regionprops() will compute the area based on whole pixels, while bwarea() does not (it uses a more complicated algorithm).

[Abdussalam Elhanashi]

Image Analyst I want to meansure dimension of whole detected Bounding Box. Meaning the Hight and width so i can say for example 22 width pixel x 47 Height pixel?

[Image Analyst]

Like I said, yes. Make sure you understand my last comment though.

[Abdussalam Elhanashi]

@ImageAnalyst

I am using yolov2 for object detection and i want to implement the following issue for my code

i want to make a condition in the time when two or more bounding boxes are near each other for instance 6 feet or less , the color of bounding box need to change its color from green to red. And once these bounding boxes of detected objects get away from each other more than 6 feet the color of these bounding boxes change back from red to green color

6 feet from bounding box to another

Herein my code

close all
clc
load('detectorYolo2.mat');
vidReader = VideoReader('vn_045_00.avi');
vidPlayer = vision.DeployableVideoPlayer;
i = 1;
results = struct('Boxes',[],'Scores',[]);
while(hasFrame(vidReader))    
    % GET DATA
    I = readFrame(vidReader);    
     fps = 0;
     avgfps = [];
     tic;
    % PROCESS
    [bboxes, scores,label] = detect(detectorYolo2,I,'Threshold',0.4);
     newt = toc;
% fps
fps = .9*fps + .1*(1/newt);
avgfps = [avgfps, fps]; %#ok<AGROW>
    
    % Select strongest detection 
    T = 0.0; % Define threshold here
idx = scores >= T;
% Retrieve those scores that surpassed the threshold
s = scores(idx);
% Do the same for the labels as well
lbl = label(idx);
bboxes = bboxes(idx, :); % This logic doesn't change
for ii = 1 : size(bboxes, 1)
    annotation = sprintf('%s: (Confidence = %f)', lbl(ii), s(ii)); % Change    
    I = insertObjectAnnotation(I, 'rectangle', bboxes(ii,:), annotation); % New - Choose the right box
    I  = insertText(I , [1, 1],  sprintf('FPS %2.2f', fps));    
end
step(vidPlayer,I);
 i = i+1;
end
 results = struct2table(results);
 release(vidPlayer);