Documentation

This is machine translation

Translated by Microsoft
Mouseover text to see original. Click the button below to return to the English verison of the page.

Note: This page has been translated by MathWorks. Please click here
To view all translated materals including this page, select Japan from the country navigator on the bottom of this page.

fullyConnectedLayer

Create fully connected layer

Syntax

fullconnectlayer = fullyConnectedLayer(outputSize)
fullconnectlayer = fullyConnectedLayer(outputSize,Name,Value)

Description

example

fullconnectlayer = fullyConnectedLayer(outputSize) returns a fully connected layer, in which the software multiplies the input by a weight matrix and then adds a bias vector.

example

fullconnectlayer = fullyConnectedLayer(outputSize,Name,Value) returns a fully connected layer with additional options specified by one or more Name,Value pair arguments.

Examples

collapse all

Create a fully connected layer with an output size of 10.

fullconnectlayer = fullyConnectedLayer(10);

The software determines the input size at training time.

Create a fully connected layer with an output size of 10. Set the learning rate factor for the bias to 2. Manually initialize the weights from a Gaussian distribution with a standard deviation of 0.0001.

layers = [imageInputLayer([28 28 1],'Normalization','none');
          convolution2dLayer(5,20,'NumChannels',1);
          reluLayer();
          maxPooling2dLayer(2,'Stride',2);
          fullyConnectedLayer(10);
          softmaxLayer();
          classificationLayer()];

To initialize the weights of the fully connected layer, you must know the layer's input size. The input size is equal to the output size of the preceding max pooling layer, which, in turn, depends on the output size of the convolutional layer.

For one direction in a channel (feature map) of the convolutional layer, the output is ((28 – 5 + 2*0)/1) +1 = 24. The max pooling layer has nonoverlapping regions, so it down-samples by 2 in each direction, i.e., 24/2 = 12. For one channel of the convolutional layer, the output of the max pooling layer is 12 * 12 = 144. There are 20 channels in the convolutional layer, so the output of the max pooling layer is 144 * 20 = 2880. This is the size of the input to the fully connected layer.

The formula for overlapping regions gives the same result: For one direction of a channel, the output is (((24 – 2 +0)/2) + 1 = 12. For one channel, the output is 144, and for all 20 channels in the convolutional layer, the output of the max pooling layer is 2880.

Initialize the weights of the fully connected layer from a Gaussian distribution with a mean of 0 and a standard deviation of 0.0001.

layers(5).Weights = randn([10 2880])*0.0001;

randn([10 2880]) returns a 10-by-2880 matrix of values from a Gaussian distribution with mean 0 and standard deviation 1. Multiplying the values by 0.0001 sets the standard deviation of the Gaussian distribution equal to 0.0001.

Similarly, initialize the biases from a Gaussian distribution with a mean of 1 and a standard deviation of 0.0001.

layers(5).Bias = randn([10 1])*0.0001+1;

The size of the bias vector is equal to the output size of the fully connected layer, which is 10. randn([10 1]) returns a 10-by-1 vector of values from a Gaussian distribution with a mean of 0 and a standard deviation of 1. Multiplying the values by 0.00001 sets the standard deviation of values equal to 0.00001, and adding 1 sets the mean of the Gaussian distribution equal to 1.

Input Arguments

collapse all

Size of the output for the fully connected layer, specified as an integer value. For classification problems, if this is the last layer before the softmax layer, then the output size must be equal to the number of classes in the data. For regression problems, if this is the last layer before the regression layer, then the output size must be equal to the number of response variables.

Data Types: single | double

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside single quotes (' '). You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

Example: 'WeightLearnRateFactor',1.5,'Name','fullyconnect1' specifies the learning rate for this layer is 1.5 times the global learning rate, and the name of the layer as fullyconnect1.

collapse all

Multiplier for the learning rate of the weights, specified as the comma-separated pair consisting of WeightLearnRateFactor and a scalar value.

trainNetwork multiplies this factor with the global learning rate to determine the learning rate for the weights in this layer. It determines the global learning rate based on the settings specified using trainingOptions.

Example: 'WeightLearnRateFactor',2 specifies that the learning rate for the weights in this layer is twice the global learning rate.

Data Types: single | double

Multiplier for the learning rate of the bias, specified as the comma-separated pair consisting of BiasLearnRateFactor and a scalar value.

The software multiplies this factor with the global learning rate to determine the learning rate for the bias in this layer.

The software determines the global learning rate based on the settings specified using the trainingOptions function.

Example: 'BiasLearnRateFactor',2 specifies that the learning rate for the bias in this layer is twice the global learning rate.

Data Types: single | double

L2 regularization factor for the weights, specified as the comma-separated pair consisting of WeightL2Factor and a scalar value.

The software multiplies this factor with the global L2 regularization factor to determine the learning rate for the weights in this layer.

You can specify the global L2 regularization factor using the trainingOptions function.

Example: 'WeightL2Factor',2 specifies that the L2 regularization for the weights in this layer is twice the global L2 regularization factor.

Data Types: single | double

Multiplier for the L2 weight regularizer for the biases, specified as the comma-separated pair consisting of BiasL2Factor and a scalar value.

You can specify the global L2 regularization factor using the trainingOptions function.

Example: 'BiasL2Factor',2 specifies that the L2 regularization for the bias in this layer is twice the global L2 regularization factor.

Data Types: single | double

Layer name, specified as the comma-separated pair consisting of Name and a character vector.

Example: 'Name','fullconnect1'

Data Types: char

Output Arguments

collapse all

Fully connected layer, returned as a FullyConnectedLayer object.

For information on concatenating layers to construct convolutional neural network architecture, see Layer.

Algorithms

The default for the initial weights is a Gaussian distribution with mean 0 and standard deviation 0.01. The default for the initial bias is 0. You can manually change the initialization for the weights and bias. See Specify Initial Weight and Biases in Fully Connected Layer.

Introduced in R2016a

Was this topic helpful?