groundTruth

Ground truth label data

Description

The groundTruth object contains information about the data source, label definitions, and marked label annotations for a set of ground truth labels. You can export or import a groundTruth object from the Image Labeler and Video Labeler apps.

• To create training data for an object detector from arrays of groundTruth objects, use the objectDetectorTrainingData function.

• To create training data for a semantic segmentation network from arrays of groundTruth objects, use the pixelLabelTrainingData function.

Creation

To export a groundTruth object from a labeling app, on the app toolstrip, select Export Labels > To Workspace. The app exports the object to the MATLAB^® workspace. To create a groundTruth object programmatically, use the groundTruth function (described here).

Syntax

gTruth = groundTruth(dataSource,labelDefs,labelData)

Description

gTruth = groundTruth(dataSource,labelDefs,labelData) returns an object containing ground truth labels that can be imported into the Image Labeler and Video Labeler apps.

• dataSource specifies the source of the ground truth data and sets the DataSource property.

• labelDefs specifies the label, sublabel, and attribute definitions of the ground truth data and sets the LabelDefinitions property.

• labelData specifies the identifying information, position, and timestamps for marked labels and sets the LabelData property.

Properties

DataSource — Source of ground truth data

groundTruthDataSource object

Source of ground truth data, specified as a groundTruthDataSource object. The object contains information that describes the video, image sequence, or custom data source from which ground truth data was labeled.

To access images from the original data source, use VideoReader or imageDatastore. You can also use a custom read function. For more details, see Use Custom Image Source Reader for Labeling.

LabelDefinitions — Label definitions

table

This property is read-only.

Label definitions, specified as a table. To create this table, use one of these options.

• In one of the labeling apps, create label definitions, and then export them as part of a groundTruth object.

• Use a labelDefinitionCreator object to generate a label definitions table. If you save this table to a MAT-file, you can then load the label definitions into a labeling app session by selecting Load > Label Definitions from the app toolstrip.

• Create the label definitions table at the MATLAB command line.

This table describes the required and optional columns of the table specified in the LabelDefinitions property.

Column	Description	Required or Optional
Name	Strings or character vectors specifying the name of each label definition.	Required
Type	labelType enumerations that specify the type of each label definition, such as Rectangle or Scene.	Required
LabelColor	1-by-3 row vectors of RGB triplets that specify the colors of the label definitions. Values are in the range [0, 1]. The color yellow (RGB triplet [1 1 0]) is reserved for the color of selected labels in the labeling apps.	Optional When you define labels in a labeling app, you must specify a color. Therefore, an exported label definitions table always includes this column. When you create label definitions using the labelDefinitionCreator object without specifying colors, the returned label definitions table includes this column, but all column values are empty.
PixelLabelID	Scalars, column vectors, or M-by-3 matrices of integer-valued label IDs. PixelLabelID specifies the pixel label values used to represent a label definition. Pixel label ID values must be between 0 and 255.	Optional When you define pixel labels in a labeling app or the labelDefinitionCreator object, the generated label definitions table includes this column. When creating a label definitions table at the MATLAB command line, if you set Type to labelType.PixelLabel for any label, then this column is required.
Group	Strings or character vectors specifying the group to which each label definition belongs.	Optional If you create the label definitions table at the MATLAB command line, you do not need to include a Group column. If you export label definitions from a labeling app or create them using a labelDefinitionCreator object, the label definitions table includes this column, even if you did not specify groups. The app assigns each label definition a Group value of 'None'.
Description	Strings or character vectors that describe each label definition.	Optional If you create the label definitions table at the MATLAB command line, you do not need to include a Description column. If you export label definitions from a labeling app or create them using a labelDefinitionCreator object, the label definitions table includes this column, even if you did not specify descriptions. The Description for these label definitions is an empty character vector.
Hierarchy	Structures containing sublabel and attribute data for each label definition. For an example of the Hierarchy format, see Get Started with the Image Labeler or Get Started with the Video Labeler.	Optional In labeling apps, when you define sublabels or attributes, the exported groundTruth object includes this column.

For example, consider a table with label definitions named Sky, Vegetation, Lanes, StopSign, and Vehicle, and that was exported from the Video Labeler app.

• The label definitions include pixel labels, so the table includes a PixelLabelID column.

• Two of the labels contain attributes, so the app created a Hierarchy column that applies across all label definitions.

• The label definitions do not have assigned groups, so the Group column is 'None' for all label definitions.

LabelData — Label data for each ROI and scene label

table | timetable

This property is read-only.

Label data for each ROI and scene label, specified as a table for image collections or a timetable for videos or image sequences. Each column of LabelData holds labels for a single label definition and corresponds to the Name value for each row in LabelDefinitions. These LabelData describes the elements of the table. The label categories are specified as labelType enumerations.

Alternatively, for ROI label data that is grouped by label type, a single column labeled ROILabelData, can be used and specified as a structure containing at least one label type, RectangleData, PolygonData, LineData, or ProjCuboidData.

The storage format for each label type is described in the table.

Label Type	Storage Format for Labels at Each Timestamp
labelType.Rectangle	M-by-4 numeric matrix of the form [x, y, w, h], where: M is the number of labels in the frame. x and y specify the upper-left corner of the rectangle. w specifies the width of the rectangle, which is its length along the x-axis. h specifies the height of the rectangle, which is its length along the y-axis.
labelType.Cuboid	M-by-9 numeric matrix with rows of the form [xctr, yctr, zctr, xlen, ylen, zlen, xrot, yrot, zrot], where: M is the number of labels in the frame. xctr, yctr, and zctr specify the center of the cuboid. xlen, ylen, and zlen specify the length of the cuboid along the x-axis, y-axis, and z-axis, respectively, before rotation has been applied. xrot, yrot, and zrot specify the rotation angles for the cuboid along the x-axis, y-axis, and z-axis, respectively. These angles are clockwise-positive when looking in the forward direction of their corresponding axes. The figure shows how these values determine the position of a cuboid.
labelType.ProjectedCuboid	M-by-8 vector of the form [x1, y1, w1, h1, x2, y2, w2, h2], where: M is the number of labels in the frame. x1, y1 specifies the x,y coordinates for the upper-left location of the front-face of the projected cuboid w1 specifies the width for the front-face of the projected cuboid. h1 specifies the height for the front-face of the projected cuboid. x2, y2 specifies the x,y coordinates for the upper-left location of the back-face of the projected cuboid. w2 specifies the width for the back-face of the projected cuboid. h2 specifies the height for the back-face of the projected cuboid. The figure shows how these values determine the position of a cuboid.
labelType.Line	M-by-1 vector of cell arrays, where M is the number of labels in the frame. Each cell array contains an N-by-2 numeric matrix of the form [x1 y1; x2 y2; ... ; xN yN] for N points in the polyline.
labelType.PixelLabel	Label data for all pixel label definitions is stored in a single M-by-1 PixelLabelData column for M images or frames. Each element contains a filename for a pixel label image. A pixel label image describes the label or labels contained in the corresponding image. The labels can be described as a 1- or 3- channel label matrix. To use PixelLabelData with any of the labeler apps, you must use a single-channel label matrix, where the values are of type uint8. You can convert a 3-channel pixel label data matrix to a single-channel label matrix programmatically to use with the labeler apps.
labelType.Polygon	M-by-1 vector of cell arrays, where M is the number of labels. Each cell array contains an N-by-2 numeric matrix of the form [x1 y1; x2 y2; ... ; xN yN] for N points in the polygon.
labelType.Custom	Labels are stored exactly as they are specified in the timetable. If you import a groundTruthMultisignal object containing custom label data into the Ground Truth Labeler app, this data is not imported into the app. Use custom data when gathering label data for training and combining it with data labeled in the app.

Supported GroundTruth Objects

	Video Labeler App	Image Labeler App
Data source	Video file, image sequence folder, custom reader	Image files
Label definitions	Rectangle, Line, PixelLabel, or Scene label types	Rectangle, Line, PixelLabel, or Scene label types
Label data	Timetable of Rectangle, Line, PixelLabel, or Scene label types	Table (no timetable) for Rectangle, Line, PixelLabel, or Scene label types

To add ground truth data that is not an ROI (Rectangle, Line, PixelLabel) or Scene label category to a groundTruth object, provide a label definition with a labelType that is Custom. The custom data is not visible when you load it into the labeling app.

Object Functions

selectLabelsByGroup	Select ground truth labels by label group
selectLabelsByType	Select ground truth labels by label type
selectLabelsByName	Select ground truth labels by label name
changeFilePaths	Change file paths in ground truth data
gatherLabelData	Gather label data from ground truth

Examples

Create Ground Truth for Stop Signs and Cars

Create a data source from a collection of images.

data = load('stopSignsAndCars.mat');
imageFilenames = data.stopSignsAndCars.imageFilename(1:2)

imageFilenames = 2x1 cell
    {'stopSignImages/image001.jpg'}
    {'stopSignImages/image002.jpg'}

imageFilenames = fullfile(toolboxdir('vision'),'visiondata',imageFilenames);
dataSource = groundTruthDataSource(imageFilenames);

Define labels used to specify the ground truth. Use labelDefinitionCreator to create the label definitions table.

ldc = labelDefinitionCreator();
addLabel(ldc,'stopSign',labelType.Rectangle);
addLabel(ldc,'carRear',labelType.Rectangle);
labelDefs = create(ldc)

labelDefs=2×5 table
        Name          Type       LabelColor     Group      Description
    ____________    _________    __________    ________    ___________

    {'stopSign'}    Rectangle    {0x0 char}    {'None'}       {' '}   
    {'carRear' }    Rectangle    {0x0 char}    {'None'}       {' '}   

Initialize label data for rectangle ROIs.

stopSignTruth = {[856   318    39    41];[445   523    52    54]};
carRearTruth = {[398   378   315   210];[332   633   691   287]};

Construct a table of label data.

labelNames = {'stopSign';'carRear'};
labelData = table(stopSignTruth,carRearTruth,'VariableNames',labelNames)

labelData=2×2 table
        stopSign               carRear      
    _________________    ___________________

    {[856 318 39 41]}    {[398 378 315 210]}
    {[445 523 52 54]}    {[332 633 691 287]}

Create a ground truth object.

gTruth = groundTruth(dataSource,labelDefs,labelData)

gTruth = 
  groundTruth with properties:

          DataSource: [1x1 groundTruthDataSource]
    LabelDefinitions: [2x5 table]
           LabelData: [2x2 table]

Create Ground Truth Data of Road Lanes

Create a groundTruth object to store data representing marked road lanes.

Create a data source from an image.

dataSource = groundTruthDataSource({'stopSignTest.jpg'});

Define labels used to specify ground truth. Use labelDefinitionCreator to create label definitions table.

ldc = labelDefinitionCreator();
addLabel(ldc,'Lane',labelType.Line);
labelDefs = create(ldc);

Assign two lane markers in the image.

laneMarkerTruth = {[257 254;311 180] [327 183;338 205;374 250]};

Construct a table of label data.

labelNames = {'Lane'};
labelData = table(laneMarkerTruth,'VariableNames',labelNames)

labelData=table
                Lane            
    ____________________________

    {2x2 double}    {3x2 double}

Create a groundTruth object.

gTruth = groundTruth(dataSource,labelDefs,labelData)

gTruth = 
  groundTruth with properties:

          DataSource: [1x1 groundTruthDataSource]
    LabelDefinitions: [1x5 table]
           LabelData: [1x1 table]

Create Ground Truth Data For Pixel Labels

Create a groundTruth object to store data representing parts of a scene.

Create a data source.

dataSource = groundTruthDataSource({'visionteam.jpg'});

Use labelDefinitionCreator to create the label definitions table. Define labels, 'Person' and 'Background'. Assign their corresponding label type as PixelLabel.

ldc =labelDefinitionCreator();
addLabel(ldc,'Person',labelType.PixelLabel);
addLabel(ldc,'Background',labelType.PixelLabel);
labelDefs = create(ldc)             

labelDefs=2×6 table
         Name            Type       LabelColor    PixelLabelID     Group      Description
    ______________    __________    __________    ____________    ________    ___________

    {'Person'    }    PixelLabel    {0x0 char}       {[1]}        {'None'}       {' '}   
    {'Background'}    PixelLabel    {0x0 char}       {[2]}        {'None'}       {' '}   

Specify the location of the pixel label data for the image.

dataFile = {'visionteamPixelLabels.png'}    

dataFile = 1x1 cell array
    {'visionteamPixelLabels.png'}

Construct a table of label data for the pixel label data.

labelData = table(dataFile,'VariableNames',{'PixelLabelData'})

labelData=table
           PixelLabelData        
    _____________________________

    {'visionteamPixelLabels.png'}

Create a groundTruth object.

gTruth = groundTruth(dataSource,labelDefs,labelData)

gTruth = 
  groundTruth with properties:

          DataSource: [1x1 groundTruthDataSource]
    LabelDefinitions: [2x6 table]
           LabelData: [1x1 table]

Create Ground Truth for Car and Lane Markers

Create a data source from a video.

videoName = 'caltech_cordova1.avi';
dataSource = groundTruthDataSource(videoName);

Define labels used to specify the ground truth. Use a labelDefinitionCreator object to create the label definitions table.

ldc = labelDefinitionCreator();
addLabel(ldc,'Cars',labelType.Rectangle);
addLabel(ldc,'LaneMarkers',labelType.Line);
labelDefs = create(ldc)

labelDefs=2×5 table
         Name            Type       LabelColor     Group      Description
    _______________    _________    __________    ________    ___________

    {'Cars'       }    Rectangle    {0x0 char}    {'None'}       {' '}   
    {'LaneMarkers'}    Line         {0x0 char}    {'None'}       {' '}   

Create label data for cars and lane markers.

numRows = numel(dataSource.TimeStamps);
carsTruth = cell(numRows,1);
laneMarkerTruth = cell(numRows,1);

Add two car labels and two lane markers to the first frame.

carsTruth{1} = [182 186 31 22;404 191 53 34];
laneMarkerTruth{1} = {[257 254;311 180] [327 183;338 205;374 250]};

Create a table of label data.

labelNames = {'Cars','LaneMarkers'};
labelData = table(carsTruth,laneMarkerTruth,'VariableNames',labelNames);

Create a groundTruth object. To import this object into a labeling app, select an option from the Open > Import Labels menu.

gTruth = groundTruth(dataSource,labelDefs,labelData)

gTruth = 
  groundTruth with properties:

          DataSource: [1x1 groundTruthDataSource]
    LabelDefinitions: [2x5 table]
           LabelData: [250x2 timetable]

Tips

• groundTruth objects for video-based data sources rely on the video reading capabilities of your operating system. A groundTruth object created using a video data source remains consistent only for the same platform that was used to create it. To create a platform-independent groundTruth object, convert the video into a sequence of images and include the associated timestamps with the image sequence.