Adaptive Filtering (adapt)

The ADALINE network, much like the perceptron, can only solve linearly separable problems. It is, however, one of the most widely used neural networks found in practical applications. Adaptive filtering is one of its major application areas.

Tapped Delay Line

You need a new component, the tapped delay line, to make full use of the ADALINE network. Such a delay line is shown in the next figure. The input signal enters from the left and passes through N-1 delays. The output of the tapped delay line (TDL) is an N-dimensional vector, made up of the input signal at the current time, the previous input signal, etc.

Back to Top

Adaptive Filter

You can combine a tapped delay line with an ADALINE network to create the adaptive filter shown in the next figure.

The output of the filter is given by

In digital signal processing, this network is referred to as a finite impulse response (FIR) filter [WiSt85]. Take a look at the code used to generate and simulate such an adaptive network.

Back to Top

Adaptive Filter Example

First define a new linear network using newlin.

Assume that the input values have a range from 0 to 10. You can now define the single output network.

net = newlin([0,10],1);

Specify the delays in the tapped delay line with

net.inputWeights{1,1}.delays = [0 1 2];

This definition indicates that the delay line connects to the network weight matrix through delays of 0, 1, and 2 time units. (You can specify as many delays as you want, and can omit some values if you like. They must be in ascending order.)

You can give the various weights and the bias values with

net.IW{1,1} = [7 8 9];
net.b{1} = [0];

Finally, define the initial values of the outputs of the delays as

pi = {1 2};

These are ordered from left to right to correspond to the delays taken from top to bottom in the figure. This concludes the setup of the network.

To set up the input, assume that the input scalars arrive in a sequence: first the value 3, then the value 4, next the value 5, and finally the value 6. You can indicate this sequence by defining the values as elements of a cell array in curly braces.

p = {3 4 5 6};

Now, you have a network and a sequence of inputs. Simulate the network to see what its output is as a function of time.

[a,pf] = sim(net,p,pi)

This simulation yields an output sequence

a = 
    [46]    [70]    [94]    [118]

and final values for the delay outputs of

pf = 
    [5]    [6]

The example is sufficiently simple that you can check it without a calculator to make sure that you understand the inputs, initial values of the delays, etc.

The network just defined can be trained with the function adapt to produce a particular output sequence. Suppose, for instance, you want the network to produce the sequence of values 10, 20, 30, 40.

t = {10 20 30 40};

You can train the defined network to do this, starting from the initial delay conditions used above. Specify 10 passes through the input sequence with

net.adaptParam.passes = 10;

Then launch the training with

[net,y,E,pf,af] = adapt(net,p,t,pi);

This code returns the final weights, bias, and output sequence shown here.

wts = net.IW{1,1}
wts =
    0.5059    3.1053    5.7046
bias = net.b{1}
bias =
   -1.5993
y
y = 
    [11.8558]    [20.7735]    [29.6679]    [39.0036]

Presumably, if you ran additional passes the output sequence would have been even closer to the desired values of 10, 20, 30, and 40.

Thus, adaptive networks can be specified, simulated, and finally trained with adapt. However, the outstanding value of adaptive networks lies in their use to perform a particular function, such as prediction or noise cancelation.

Back to Top

Prediction Example

Suppose that you want to use an adaptive filter to predict the next value of a stationary random process, p(t). You can use the network shown in the following figure to do this prediction.

The signal to be predicted, p(t), enters from the left into a tapped delay line. The previous two values of p(t) are available as outputs from the tapped delay line. The network uses adapt to change the weights on each time step so as to minimize the error e(t) on the far right. If this error is 0, the network output a(t) is exactly equal to p(t), and the network has done its prediction properly.

Given the autocorrelation function of the stationary random process p(t), you can calculate the error surface, the maximum learning rate, and the optimum values of the weights. Commonly, of course, you do not have detailed information about the random process, so these calculations cannot be performed. This lack does not matter to the network. After it is initialized and operating, the network adapts at each time step to minimize the error and in a relatively short time is able to predict the input p(t).

Chapter 10 of [HDB96] presents this problem, goes through the analysis, and shows the weight trajectory during training. The network finds the optimum weights on its own without any difficulty whatsoever.

You also can try demonstration nnd10nc to see an adaptive noise cancelation program example in action. This demonstration allows you to pick a learning rate and momentum (see Multilayer Networks and Backpropagation Training), and shows the learning trajectory, and the original and cancelation signals versus time.

Back to Top

Noise Cancelation Example

Consider a pilot in an airplane. When the pilot speaks into a microphone, the engine noise in the cockpit combines with the voice signal. This additional noise makes the resultant signal heard by passengers of low quality. The goal is to obtain a signal that contains the pilot's voice, but not the engine noise. You can cancel the noise with an adaptive filter if you obtain a sample of the engine noise and apply it as the input to the adaptive filter.

As the preceding figure shows, you adaptively train the neural linear network to predict the combined pilot/engine signal m from an engine signal n. The engine signal n does not tell the adaptive network anything about the pilot's voice signal contained in m. However, the engine signal n does give the network information it can use to predict the engine's contribution to the pilot/engine signal m.

The network does its best to output m adaptively. In this case, the network can only predict the engine interference noise in the pilot/engine signal m. The network error e is equal to m, the pilot/engine signal, minus the predicted contaminating engine noise signal. Thus, e contains only the pilot's voice. The linear adaptive network adaptively learns to cancel the engine noise.

Such adaptive noise canceling generally does a better job than a classical filter, because it subtracts from the signal rather than filtering it out the noise of the signal m.

Try demolin8 for an example of adaptive noise cancelation.

Back to Top

Multiple Neuron Adaptive Filters

You might want to use more than one neuron in an adaptive system, so you need some additional notation. You can use a tapped delay line with S linear neurons, as shown in the next figure.

Alternatively, you can represent this same network in abbreviated form.

If you want to show more of the detail of the tapped delay line—and there are not too many delays—you can use the following notation:

Here, a tapped delay line sends to the weight matrix:

• The current signal

• The previous signal

• The signal delayed before that

You could have a longer list, and some delay values could be omitted if desired. The only requirement is that the delays must appears in increasing order as they go from top to bottom.

Back to Top

LMS Algorithm (learnwh)

Advanced Topics

© 1984-2012- The MathWorks, Inc.    -   Site Help   -   Patents   -   Trademarks   -   Privacy Policy   -   Preventing Piracy   -   RSS