MATLAB Examples

evaluateObjectDetector

Evaluate the input object detection algorithm by comparing output to ground truth data

Syntax

  • SAS = evaluateObjectDetector(GTS,detector,label)
  • SAS = evaluateObjectDetector(GTS,detector,label,Name,Value)

Description

  • SAS = evaluateObjectDetector(GTS,detector,label) Takes as an input the user specified Ground Truth Data (GTS) from the Ground Truth Labeler App, the name of the user created object detector and the label corresponding to the object which needs to be compared.
  • SAS = evaluateObjectDetector(GTS,detector,label,Name,Value) Specifies additional options with one or more Name,Value pair arguments, using any of the previous syntaxes.

Input Arguments

GTS - Ground Truth Session

Data Type: struct or string

Input structure or name of .MAT file containing structure. Created by Ground Truth Labeler app.

detector - Object Detection Algorithm

Data Type: string or function handle

Name of object detection function or handle to function. Created by user. Output of this function must be an Mx4 array of bounding boxes corresponding to the object the user wants to detect.

label - Label

Data Type: string

Name of object in Ground Truth Session that needs to be detected by the input detector object detection algorithm.

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside single quotes (' '). You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

'Verbosity' - Verbosity

Data Type: logical {true (default) | false}

Specify verbosity of evaluateObjectDetector. If true, a status message indicating what media is being analyzed will be returned. If false, nothing will output to the Command Window.

'AreaPercentage' - Area Comparison Percentage

Data Type: string or double {'Non-Zero'(default) | value in the range of (0 100]}

Specify the percentage of overlap area between the output bounding box of an object detector and the recorded bounding box from the ground truth session. By default, the algorithm determines if an object is found if a non-zero area of overlap is found.

Detection occurs if the following comparison is met: (AreaObectDetector/AreaGroundTruth) > (AreaPercentage/100)

Output Arguments

SAS - Sensitivity and Specificity results

Output structure. Contains the sensitivity and specificity data used to evaluate the input object detection algorithm.

Examples

Basic Example

Load the Ground Truth Session

load GroundTruthSessionFull
detector = 'ThresholdingObjectDetector';
label = 'red buoy';

Calculate the Sensitivity and Specificity data

SAS = evaluateObjectDetector(GTS,detector,label);
Analyzing Media 1 out of 4: buoys.jpg
Analyzing Media 2 out of 4: stop.jpg
Analyzing Media 3 out of 4: BuoyRun.avi
Analyzing Media 4 out of 4: vipwarnsigns.avi
Ground Truth Analysis Finished

Visualize the results

viewSAS(SAS);
SessionTotal = 

    TP    FN     P     FP    TN      N 
    __    __    ___    __    ___    ___

    90    22    112    2     104    104


SessionIndividual = 

                        TP    FN     P     FP    TN      N                                      Location                                 
                        __    __    ___    __    ___    ___    __________________________________________________________________________

    buoys.jpg            1     1      2    0       1      1    'C:\MyMatlab\Projects\GroundTruthLabelerAppProject\GroundTruthLabelerApp…'
    stop.jpg             0     0      0    0       1      1    'C:\MyMatlab\Projects\GroundTruthLabelerAppProject\GroundTruthLabelerApp…'
    BuoyRun.avi         89    21    110    2     102    102    'C:\MyMatlab\Projects\GroundTruthLabelerAppProject\GroundTruthLabelerApp…'
    vipwarnsigns.avi     0     0      0    0       0      0    'C:\Program Files\MATLAB\R2015b\toolbox\vision\visiondata'                

Specify Area of Overlap Percentage

Load the Ground Truth Session

load GroundTruthSessionFull
detector = 'ThresholdingObjectDetector';
label = 'red buoy';

Calculate the Sensitivity and Specificity data

SAS = evaluateObjectDetector(GTS,detector,label,'AreaPercentage',50);
Analyzing Media 1 out of 4: buoys.jpg
Analyzing Media 2 out of 4: stop.jpg
Analyzing Media 3 out of 4: BuoyRun.avi
Analyzing Media 4 out of 4: vipwarnsigns.avi
Ground Truth Analysis Finished

View the results

viewSAS(SAS);
SessionTotal = 

    TP    FN     P     FP    TN      N 
    __    __    ___    __    ___    ___

    19    93    112    74    104    104


SessionIndividual = 

                        TP    FN     P     FP    TN      N                                      Location                                 
                        __    __    ___    __    ___    ___    __________________________________________________________________________

    buoys.jpg            0     2      2     1      1      1    'C:\MyMatlab\Projects\GroundTruthLabelerAppProject\GroundTruthLabelerApp…'
    stop.jpg             0     0      0     0      1      1    'C:\MyMatlab\Projects\GroundTruthLabelerAppProject\GroundTruthLabelerApp…'
    BuoyRun.avi         19    91    110    73    102    102    'C:\MyMatlab\Projects\GroundTruthLabelerAppProject\GroundTruthLabelerApp…'
    vipwarnsigns.avi     0     0      0     0      0      0    'C:\Program Files\MATLAB\R2015b\toolbox\vision\visiondata'                

See Also

groundTruthLabeler, viewSAS, visualizeSAS