crossentropy

Categorical cross-entropy loss

Description

The cross-entropy operation computes the categorical cross-entropy loss between network predictions and target values for multiclass classification problems.

The loss is calculated using the following formula

loss=1Ni=1MTilog(Xi)

where Xi is the network response, Ti is the target value, M is the total number of responses in X (across all observations), and N is the total number of observations in X.

Note

This function computes the cross-entropy loss between predictions and targets stored as dlarray data. If you want to calculate the cross-entropy loss within a layerGraph object or Layer array for use with trainNetwork, use the following layer:

example

dlY = crossentropy(dlX,targets) computes the categorical cross-entropy loss between the predictions dlX and the target values targets for multiclass classification problems. The input dlX is a formatted dlarray with dimension labels. The output dlY is an unformatted scalar dlarray with no dimension labels.

dlY = crossentropy(dlX,targets,'DataFormat',FMT) also specifies the dimension labels FMT when dlX is not a formatted dlarray.

Examples

collapse all

The cross-entropy loss evaluates how well the network predictions correspond to the target classification.

Create the input classification data as a matrix of random variables. The data can be any of 10 categories, and there are 12 observations.

numCategories = 10;
observations = 12;

X = rand(numCategories,observations);
dlX = dlarray(X,'CB');

Convert the category values in the data to probability scores for each category.

dlX = softmax(dlX);

Create the target data that holds the correct category for each observation in dlX.

targetsIdx = randi(10,1,12);
targets = zeros(10,12);
for i = 1:numel(targetsIdx)
    targets(targetsIdx(i),i) = 1;
end

Compute the cross-entropy loss between the predictions and the targets

dlY = crossentropy(dlX,targets)
dlY = 
  1x1 dlarray

    2.3343

Input Arguments

collapse all

Predictions, specified as a dlarray with or without dimension labels or a numeric array. When dlX is not a formatted dlarray, you must specify the dimension label format using 'DataFormat',FMT. If dlX is a numeric array, targets must be a dlarray.

Data Types: single | double

Target classification labels, specified as a dlarray with or without dimension labels or a numeric array. If targets is an unformatted dlarray or a numeric array, it must have the same dimension order as the input dlX. The size of each dimension of targets must match the size of each corresponding dimension in dlX.

Data Types: single | double

Dimension order of unformatted input data, specified as the comma-separated pair consisting of 'DataFormat' and a character array or string that provides a label for each dimension of the data. Each character in FMT must be one of the following:

  • 'S' — Spatial

  • 'C' — Channel

  • 'B' — Batch (for example, samples and observations)

  • 'T' — Time (for example, sequences)

  • 'U' — Unspecified

You can specify multiple dimensions labeled 'S' or 'U'. You can use the labels 'C', 'B', or 'T' at most once.

You must specify 'DataFormat' when the input data dlX is an unformatted dlarray.

Example: 'DataFormat','SSCB'

Data Types: char | string

Output Arguments

collapse all

Cross-entropy loss, returned as a dlarray scalar without dimension labels. The output dlY has the same underlying data type as the input dlX.

The cross-entropy loss dlY is the average logarithmic loss across the 'B' batch dimension of dlX.

More About

collapse all

Cross-Entropy Loss

The crossentropy function computes the cross-entropy loss for classification problems. For more information, see the definition of Classification Output Layer on the ClassificationOutputLayer reference page.

Extended Capabilities

Introduced in R2019b