Main Content

If Deep Learning Toolbox™ does not provide the layer you require for your classification or regression problem, then you can define your own custom layer. For a list of built-in layers, see List of Deep Learning Layers.

The example Define Custom Weighted Classification Layer shows how to define and create a custom weighted classification output layer with weighted cross entropy loss and goes through the following steps:

Name the layer – Give the layer a name so it can be used in MATLAB

^{®}.Declare the layer properties – Specify the properties of the layer.

Create a constructor function (optional) – Specify how to construct the layer and initialize its properties. If you do not specify a constructor function, then the software initializes the properties with

`''`

at creation.Create a forward loss function – Specify the loss between the predictions and the training targets.

Create a backward loss function (optional) – Specify the derivative of the loss with respect to the predictions. If you do not specify a backward loss function, then the forward loss function must support

`dlarray`

objects.

Creating a backward loss function is optional. If the forward loss function only uses
functions that support `dlarray`

objects, then software determines the
derivatives automatically using automatic differentiation. For a list of functions that
support `dlarray`

objects, see List of Functions with dlarray Support. If you want to use
functions that do not support `dlarray`

objects, or want to use a specific
algorithm for the backward loss function, then you can define a custom backward function
using this example as a guide.

The example Define Custom Weighted Classification Layer shows how to create a weighted classification layer.

A weighted classification layer computes the weighted cross
entropy loss for classification problems. Weighted cross entropy is an error measure between two continuous random
variables. For prediction scores *Y* and training targets *T*,
the weighted cross entropy loss between *Y* and *T* is given by

$$L=-\frac{1}{N}{\displaystyle \sum _{n=1}^{N}{\displaystyle \sum}_{i=1}^{K}}\text{}{w}_{i}{T}_{ni}\mathrm{log}({Y}_{ni}),$$

where *N* is the number of observations,
*K* is the number of classes, and *w* is a vector of
weights for each class.

View the layer created in the example Define Custom Weighted Classification Layer. This layer does not
have a `backwardLoss`

function.

classdef weightedClassificationLayer < nnet.layer.ClassificationLayer properties % Vector of weights corresponding to the classes in the training % data ClassWeights end methods function layer = weightedClassificationLayer(classWeights, name) % layer = weightedClassificationLayer(classWeights) creates a % weighted cross entropy loss layer. classWeights is a row % vector of weights corresponding to the classes in the order % that they appear in the training data. % % layer = weightedClassificationLayer(classWeights, name) % additionally specifies the layer name. % Set class weights layer.ClassWeights = classWeights; % Set layer name if nargin == 2 layer.Name = name; end % Set layer description layer.Description = 'Weighted cross entropy'; end function loss = forwardLoss(layer, Y, T) % loss = forwardLoss(layer, Y, T) returns the weighted cross % entropy loss between the predictions Y and the training % targets T. N = size(Y,4); Y = squeeze(Y); T = squeeze(T); W = layer.ClassWeights; loss = -sum(W*(T.*log(Y)))/N; end end end

Implement the `backwardLoss`

function that returns the derivatives of
the loss with respect to the input data and the learnable parameters.

The syntax for `backwardLoss`

is ```
dLdY
= backwardLoss(layer, Y, T)
```

. The input `Y`

contains the
predictions made by the network and `T`

contains the training targets. The
output `dLdY`

is the derivative of the loss with respect to the predictions
`Y`

. The output `dLdY`

must be the same size as the layer
input `Y`

.

The dimensions of `Y`

and `T`

are the same as the
inputs in `forwardLoss`

.

The derivative of the weighted cross entropy loss with respect to the predictions
*Y* is given by

$$\frac{\delta L}{\delta {Y}_{i}}=-\frac{1}{N}\frac{{w}_{i}{T}_{i}}{{Y}_{i}},$$

where *N* is the number of observations and
*w* is a vector of weights for each class.

Create the backward loss function that returns these derivatives.

```
function dLdY = backwardLoss(layer, Y, T)
% dLdY = backwardLoss(layer, Y, T) returns the derivatives of
% the weighted cross entropy loss with respect to the
% predictions Y.
[~,~,K,N] = size(Y);
Y = squeeze(Y);
T = squeeze(T);
W = layer.ClassWeights;
dLdY = -(W'.*T./Y)/N;
dLdY = reshape(dLdY,[1 1 K N]);
end
```

View the completed layer class file.

classdef weightedClassificationLayer < nnet.layer.ClassificationLayer properties % Vector of weights corresponding to the classes in the training % data ClassWeights end methods function layer = weightedClassificationLayer(classWeights, name) % layer = weightedClassificationLayer(classWeights) creates a % weighted cross entropy loss layer. classWeights is a row % vector of weights corresponding to the classes in the order % that they appear in the training data. % % layer = weightedClassificationLayer(classWeights, name) % additionally specifies the layer name. % Set class weights layer.ClassWeights = classWeights; % Set layer name if nargin == 2 layer.Name = name; end % Set layer description layer.Description = 'Weighted cross entropy'; end function loss = forwardLoss(layer, Y, T) % loss = forwardLoss(layer, Y, T) returns the weighted cross % entropy loss between the predictions Y and the training % targets T. N = size(Y,4); Y = squeeze(Y); T = squeeze(T); W = layer.ClassWeights; loss = -sum(W*(T.*log(Y)))/N; end function dLdY = backwardLoss(layer, Y, T) % dLdY = backwardLoss(layer, Y, T) returns the derivatives of % the weighted cross entropy loss with respect to the % predictions Y. [~,~,K,N] = size(Y); Y = squeeze(Y); T = squeeze(T); W = layer.ClassWeights; dLdY = -(W'.*T./Y)/N; dLdY = reshape(dLdY,[1 1 K N]); end end end

If the layer forward functions fully support `dlarray`

objects, then the layer
is GPU compatible. Otherwise, to be GPU compatible, the layer functions must support inputs
and return outputs of type `gpuArray`

(Parallel Computing Toolbox).

Many MATLAB built-in functions support `gpuArray`

(Parallel Computing Toolbox) and `dlarray`

input arguments. For a list of
functions that support `dlarray`

objects, see List of Functions with dlarray Support. For a list of functions
that execute on a GPU, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).
To use a GPU for deep
learning, you must also have a CUDA^{®} enabled NVIDIA^{®} GPU with compute capability 3.0 or higher. For more information on working with GPUs in MATLAB, see GPU Computing in MATLAB (Parallel Computing Toolbox).

- Define Custom Deep Learning Layers
- Define Custom Deep Learning Layer with Learnable Parameters
- Define Custom Deep Learning Layer with Multiple Inputs
- Define Nested Deep Learning Layer
- Define Custom Classification Output Layer
- Define Custom Regression Output Layer
- Specify Custom Layer Backward Function
- Check Custom Layer Validity
- List of Deep Learning Layers
- Deep Learning Tips and Tricks