Training (train)

If sim and learnp are used repeatedly to present inputs to a perceptron, and to change the perceptron weights and biases according to the error, the perceptron will eventually find weight and bias values that solve the problem, given that the perceptron can solve it. Each traversal through all the training input and target vectors is called a pass.

The function train carries out such a loop of calculation. In each pass the function train proceeds through the specified sequence of inputs, calculating the output, error, and network adjustment for each input vector in the sequence as the inputs are presented.

Note that train does not guarantee that the resulting network does its job. You must check the new values of W and b by computing the network output for each input vector to see if all targets are reached. If a network does not perform successfully you can train it further by calling train again with the new weights and biases for more training passes, or you can analyze the problem to see if it is a suitable problem for the perceptron. Problems that cannot be solved by the perceptron network are discussed in Limitations and Cautions.

To illustrate the training procedure, work through a simple problem. Consider a one-neuron perceptron with a single vector input having two elements:

This network, and the problem you are about to consider, are simple enough that you can follow through what is done with hand calculations if you want. The problem discussed below follows that found in [HDB1996].

Suppose you have the following classification problem and would like to solve it with a single vector input, two-element perceptron network.

• 

Use the initial weights and bias. Denote the variables at each step of this calculation by using a number in parentheses after the variable. Thus, above, the initial values are W(0) and b(0).

• 

Start by calculating the perceptron's output a for the first input vector p₁, using the initial weights and bias.

• 

The output a does not equal the target value t₁, so use the perceptron rule to find the incremental changes to the weights and biases based on the error.

• 

You can calculate the new weights and bias using the perceptron update rules.

• 

• 

Now present the next input vector, p₂. The output is calculated below.

• 

On this occasion, the target is 1, so the error is zero. Thus there are no changes in weights or bias, so

and .

You can continue in this fashion, presenting p₃ next, calculating an output and the error, and making changes in the weights and bias, etc. After making one pass through all of the four inputs, you get the values and . To determine whether a satisfactory solution is obtained, make one pass through all input vectors to see if they all produce the desired target values. This is not true for the fourth input, but the algorithm does converge on the sixth presentation of an input. The final values are

• and

This concludes the hand calculation. Now, how can you do this using the train function?

The following code defines a perceptron like that shown in the previous figure, with initial weights and bias values of 0.

• net = newp([-2 2;-2 2],[0 1]);
  

Consider the application of a single input.

• p = [2; 2];
  

having the target

• t = [0];
  

Set epochs to 1, so that train goes through the input vectors (only one here) just one time.

• net.trainParam.epochs = 1;
  net = train(net,p,t);
  

The new weights and bias are

• w = net.iw{1,1}, b = net.b{1}
  w =
      -2    -2
  b =
      -1
  

Thus, the initial weights and bias are 0, and after training on only the first vector, they have the values [-2 -2] and -1, just as you hand calculated.

Now apply the second input vector . The output is 1, as it will be until the weights and bias are changed, but now the target is 1, the error will be 0, and the change will be zero. You could proceed in this way, starting from the previous result and applying a new input vector time after time. But you can do this job automatically with train.

Apply train for one epoch, a single pass through the sequence of all four input vectors. Start with the network definition.

• net = newp([-2 2;-2 2],[0 1]);
  net.trainParam.epochs = 1;
  

The input vectors and targets are

• p = [[2;2] [1;-2] [-2;2] [-1;1]]
  t = [0 1 0 1]
  

Now train the network with

• net = train(net,p,t);
  

The new weights and bias are

• w = net.iw{1,1}, b = net.b{1}
  w =
      -3    -1
  b =
       0
  

This is the same result as you got previously by hand.

Finally, simulate the trained network for each of the inputs.

• a = sim(net,p)
  a = 
        0     0     1     1
  

The outputs do not yet equal the targets, so you need to train the network for more than one pass. Try more epochs. This run gives a mean absolute error performance of 0 after two epochs:

• net.trainParam.epochs = 1000;
  net = train(net,p,t);
  

Thus, the network was trained by the time the inputs were presented on the third epoch. (As you know from hand calculation, the network converges on the presentation of the sixth input vector. This occurs in the middle of the second epoch, but it takes the third epoch to detect the network convergence.) The final weights and bias are

• w = net.iw{1,1}, b = net.b{1}
  w =
      -2    -3
  b =
       1
  

The simulated output and errors for the various inputs are

• a = sim(net,p)
  a =
               0             1             0             1
  error = a-t
  error =
               0             0             0             0
  

You confirm that the training procedure is successful. The network converges and produces the correct target outputs for the four input vectors.

The default training function for networks created with newp is trainc. (You can find this by executing net.trainFcn.) This training function applies the perceptron learning rule in its pure form, in that individual input vectors are applied individually, in sequence, and corrections to the weights and bias are made after each presentation of an input vector. Thus, perceptron training with train will converge in a finite number of steps unless the problem presented cannot be solved with a simple perceptron.

The function train can be used in various ways by other networks as well. Type help train to read more about this basic function.

You might want to try various demonstration programs. For instance, demop1 illustrates classification and training of a simple perceptron.

Provide feedback about this page

Perceptron Learning Rule (learnp)

Limitations and Cautions

© 1984-2009- The MathWorks, Inc.    -   Site Help   -   Patents   -   Trademarks   -   Privacy Policy   -   Preventing Piracy   -   RSS