dlarray

Deep learning array for custom training loops

Description

A deep learning array stores data with optional data format labels for custom training loops, and enables functions to compute and use derivatives through automatic differentiation.

Tip

For most deep learning tasks, you can use a pretrained network and adapt it to your own data. For an example showing how to use transfer learning to retrain a convolutional neural network to classify a new set of images, see Train Deep Learning Network to Classify New Images. Alternatively, you can create and train networks from scratch using layerGraph objects with the trainNetwork and trainingOptions functions.

If the trainingOptions function does not provide the training options that you need for your task, then you can create a custom training loop using automatic differentiation. To learn more, see Define Custom Training Loops.

Creation

Description

example

dlX = dlarray(X) returns a dlarray object representing X. If X is a dlarray, dlX is a copy of X.

example

dlX = dlarray(X,fmt) labels the data in dlX according to the data format in fmt. Labels help in passing deep learning data between functions. See Usage. If X is a labeled dlarray, then fmt replaces the existing labels.

example

dlX = dlarray(v,dim) accepts a vector v and a single character format dim, and returns a column vector dlarray. The first dimension of dlX has the label dim, and the second (singleton) dimension has the label 'U'.

Input Arguments

expand all

Data array, specified as a numeric array of data type double or single, logical array, gpuArray object, or dlarray object. X must be full, not sparse, and must be real, not complex.

Example: rand(31*23,23)

Data format, specified as a character vector or string scalar. Each character in fmt must be one of these labels:

  • S — Spatial

  • C — Channel

  • B — Batch observations

  • T — Time or sequence

  • U — Unspecified

You can specify any number of S and U labels. You can specify at most one of each of the C, B, and T labels.

Each element of fmt labels the matching dimension of dlX. If fmt is not in the listed order ('S' followed by 'C' and so on), then dlarray implicitly permutes both fmt and the data to match the order, but without changing the storage of the data.

fmt must have at least the same number of labels as the number of dimensions of dlX. If you specify more than that number of labels, dlarray creates empty (singleton) dimensions for the additional labels.

For information on fmt, see Usage.

Example: "SSB"

Example: 'CBUSS', which dlarray reorders to 'SSCBU'

Data vector, specified as a numeric vector of data type double or single, logical vector, gpuArray vector object, or dlarray vector object. Here, "vector" means any array with exactly one nonsingleton dimension.

Example: rand(100,1)

Dimension label, specified as a single character of the type allowed for fmt.

Example: "S"

Example: 'S'

Output Arguments

expand all

Deep learning array, returned as a dlarray object. dlX enables automatic differentiation using dlgradient and dlfeval. If you supply the fmt argument, dlX has labels.

  • If X is a numeric or logical array, dlX contains its data, possibly reordered because of labels in fmt.

  • If X is a gpuArray, the data in dlX is also on the GPU. Subsequent calculations using dlX are performed on the GPU.

Usage

dlarray labels enable you to use the functions in this table to execute with assurance that the data has the appropriate format.

FunctionOperationValidates Input DimensionAffects Size of Input Dimension
avgpoolCompute the average of the input data over moving rectangular (or cuboidal) spatial ('S') regions defined by a pool size parameter.'S''S'
batchnormNormalize the values contained in each channel ('C') of the input data.'C' 
crossentropyCompute the cross-entropy between estimates and target values, averaged by the size of the batch ('B') dimension.'S', 'C', 'B', 'T', 'U' (Estimates and target arrays must have the same sizes.)'S', 'C', 'B', 'T', 'U' (The output is an unlabeled scalar.)
dlconvCompute the deep learning convolution of the input data using an array of filters, matching the number of spatial ('S') and (a function of the) channel ('C') dimensions of the input, and adding a constant bias.'S', 'C''S', 'C'
dltranspconvCompute the deep learning transposed convolution of the input data using an array of filters, matching the number of spatial ('S') and (a function of the) channel ('C') dimensions of the input, and adding a constant bias.'S', 'C''S', 'C'
fullyconnectCompute a weighted sum of the input data and apply a bias for each batch ('B') and time ('T') dimension.'S', 'C', 'U''S', 'C', 'B', 'T', 'U' (The output always has labels 'CB', 'CT', or 'CTB'.)
lstm

Apply a long short-term memory calculation to the input data.

'S', 'C', 'T''C'
maxpoolCompute the maximum of the input data over moving rectangular spatial ('S') regions defined by a pool size parameter.'S''S'
maxunpoolCompute the unpooling operation over the spatial ('S') dimensions.'S''S'
mseCompute the half mean squared error between estimates and target values, averaged by the size of the batch ('B') dimension.'S', 'C', 'B', 'T', 'U' (Estimates and target arrays must have the same sizes.)'S', 'C', 'B', 'T', 'U' (The output is an unlabeled scalar.)
softmaxApply the softmax activation to each channel ('C') of the input data.'C' 

These functions require each dimension to have a label, specified either as the labels of their first dlarray input, or as the 'DataFormat' name-value pair argument containing dimension labels.

dlarray enforces the order of labels 'SCBTU'. This enforcement eliminates ambiguous semantics in operations, which implicitly match labels between inputs. dlarray also enforces that the labels 'C', 'B', and 'T' can each appear at most once. The functions that use these labels accept at most one dimension for each label.

dlarray provides functions for removing labels (stripdims), obtaining the dimensions associated with labels (finddim), and listing the labels associated with a dlarray (dims).

For more information on how a dlarray behaves with labels, see Notable dlarray Behaviors.

Object Functions

avgpoolPool data to average values over spatial dimensions
batchnormNormalize each channel of input data
crossentropyCategorical cross-entropy loss
dimsDimension labels of dlarray
dlconvDeep learning convolution
dlgradientCompute gradients for custom training loops using automatic differentiation
dltranspconvDeep learning transposed convolution
extractdataExtract data from dlarray
finddimFind dimensions with specified label
fullyconnectSum all weighted input data and apply a bias
leakyreluApply leaky rectified linear unit activation
lstmLong short-term memory
maxpoolPool data to maximum value
maxunpoolUnpool the output of a maximum pooling operation
mseHalf mean squared error
reluApply rectified linear unit activation
sigmoidApply sigmoid activation
softmaxApply softmax activation to channel dimension
stripdimsRemove dlarray labels

A dlarray also allows functions for numeric, matrix, and other operations. See the full list in List of Functions with dlarray Support.

Examples

collapse all

Create an unlabeled dlarray from a matrix.

rng default % For reproducibility
X = randn(3,5);
dlX = dlarray(X)
dlX = 
  3×5 dlarray

    0.5377    0.8622   -0.4336    2.7694    0.7254
    1.8339    0.3188    0.3426   -1.3499   -0.0631
   -2.2588   -1.3077    3.5784    3.0349    0.7147

Create a dlarray that has a data format with the labels 'S' and 'C'.

rng default % For reproducibility
X = randn(3,5);
dlX = dlarray(X,'SC')
dlX = 
  3(S) × 5(C) dlarray

    0.5377    0.8622   -0.4336    2.7694    0.7254
    1.8339    0.3188    0.3426   -1.3499   -0.0631
   -2.2588   -1.3077    3.5784    3.0349    0.7147

If you specify the labels in the opposite order, dlarray implicitly reorders the underlying data.

dlX = dlarray(X,'CS')
dlX = 
  5(S) × 3(C) dlarray

    0.5377    1.8339   -2.2588
    0.8622    0.3188   -1.3077
   -0.4336    0.3426    3.5784
    2.7694   -1.3499    3.0349
    0.7254   -0.0631    0.7147

Create a dlarray vector with the first label 'T'. The second label, which dlarray creates automatically, is 'U'.

rng default % For reproducibility
X = randn(6,1);
dlX = dlarray(X,'T')
dlX = 
  6(T) x 1(U) dlarray

    0.5377
    1.8339
   -2.2588
    0.8622
    0.3188
   -1.3077

If you specify a row vector for X, dlarray implicitly reorders the result to be a column vector.

X = X';
dlX = dlarray(X,'T')
dlX = 
  6(T) x 1(U) dlarray

    0.5377
    1.8339
   -2.2588
    0.8622
    0.3188
   -1.3077

Tips

Introduced in R2019b