Linear Classification (train)

Linear networks can be trained to perform linear classification with the function train. This function applies each vector of a set of input vectors and calculates the network weight and bias increments due to each of the inputs according to learnp. Then the network is adjusted with the sum of all these corrections. Each pass through the input vectors is called an epoch. This contrasts with adapt, discussed in Adaptive Filters and Adaptive Training, which adjusts weights for each input vector as it is presented.

Finally, train applies the inputs to the new network, calculates the outputs, compares them to the associated targets, and calculates a mean square error. If the error goal is met, or if the maximum number of epochs is reached, the training is stopped, and train returns the new network and a training record. Otherwise train goes through another epoch. Fortunately, the LMS algorithm converges when this procedure is executed.

A simple problem illustrates this procedure. Consider the linear network introduced earlier.

Suppose you have the classification problem presented in Linear Filters.

• 

Here there are four input vectors, and you want a network that produces the output corresponding to each input vector when that vector is presented.

Use train to get the weights and biases for a network that produces the correct targets for each input vector. The initial weights and bias for the new network are 0 by default. Set the error goal to 0.1 rather than accept its default of 0.

• P = [2 1 -2 -1;2 -2 2 1];
  T = [0 1 0 1];
  net = newlin(P,T);
  net.trainParam.goal= 0.1;
  net = train(net,P,T); 
  

The problem runs for 64 epochs, achieving a mean square error of 0.0999. The new weights and bias are

• weights = net.iw{1,1}
  weights =
     -0.0615   -0.2194
  bias = net.b(1)
  bias =
      [0.5899]
  

You can simulate the new network as shown below.

• A = sim(net, P)
  A =
      0.0282    0.9672    0.2741    0.4320
  

You can also calculate the error.

• err = T - sim(net,P)
  err =
     -0.0282    0.0328   -0.2741    0.5680
  

Note that the targets are not realized exactly. The problem would have run longer in an attempt to get perfect results had a smaller error goal been chosen, but in this problem it is not possible to obtain a goal of 0. The network is limited in its capability. See Limitations and Cautions for examples of various limitations.

This demonstration program, demolin2, shows the training of a linear neuron and plots the weight trajectory and error during training.

You might also try running the demonstration program nnd10lc. It addresses a classic and historically interesting problem, shows how a network can be trained to classify various patterns, and shows how the trained network responds when noisy patterns are presented.

Provide feedback about this page

LMS Algorithm (learnwh)

Limitations and Cautions

© 1984-2009- The MathWorks, Inc.    -   Site Help   -   Patents   -   Trademarks   -   Privacy Policy   -   Preventing Piracy   -   RSS