Applications for Neural Network Toolbox™ Software

Applications in This Toolbox

Control Systems describes three practical neural network control system applications, including neural network model predictive control, model reference adaptive control, and a feedback linearization controller.

Applications describes other neural network applications.

Business Applications

The 1988 DARPA Neural Network Study [DARP88] lists various neural network applications, beginning in about 1984 with the adaptive channel equalizer. This device, which is an outstanding commercial success, is a single- neuron network used in long-distance telephone systems to stabilize voice signals. The DARPA report goes on to list other commercial applications, including a small word recognizer, a process monitor, a sonar classifier, and a risk analysis system.

Neural networks have been applied in many other fields since the DARPA report was written, as described in the next table.

Industry	Business Applications
Aerospace	High-performance aircraft autopilot, flight path simulation, aircraft control systems, autopilot enhancements, aircraft component simulation, and aircraft component fault detection
Automotive	Automobile automatic guidance system, and warranty activity analysis
Banking	Check and other document reading and credit application evaluation
Defense	Weapon steering, target tracking, object discrimination, facial recognition, new kinds of sensors, sonar, radar and image signal processing including data compression, feature extraction and noise suppression, and signal/image identification
Electronics	Code sequence prediction, integrated circuit chip layout, process control, chip failure analysis, machine vision, voice synthesis, and nonlinear modeling
Entertainment	Animation, special effects, and market forecasting
Financial	Real estate appraisal, loan advising, mortgage screening, corporate bond rating, credit-line use analysis, credit card activity tracking, portfolio trading program, corporate financial analysis, and currency price prediction
Industrial	Prediction of industrial processes, such as the output gases of furnaces, replacing complex and costly equipment used for this purpose in the past
Insurance	Policy application evaluation and product optimization
Manufacturing	Manufacturing process control, product design and analysis, process and machine diagnosis, real-time particle identification, visual quality inspection systems, beer testing, welding quality analysis, paper quality prediction, computer-chip quality analysis, analysis of grinding operations, chemical product design analysis, machine maintenance analysis, project bidding, planning and management, and dynamic modeling of chemical process system
Medical	Breast cancer cell analysis, EEG and ECG analysis, prosthesis design, optimization of transplant times, hospital expense reduction, hospital quality improvement, and emergency-room test advisement
Oil and gas	Exploration
Robotics	Trajectory control, forklift robot, manipulator controllers, and vision systems
Speech	Speech recognition, speech compression, vowel classification, and text-to-speech synthesis
Securities	Market analysis, automatic bond rating, and stock trading advisory systems
Telecommunications	Image and data compression, automated information services, real-time translation of spoken language, and customer payment processing systems
Transportation	Truck brake diagnosis systems, vehicle scheduling, and routing systems

Fitting a Function

Neural networks are good at fitting functions and recognizing patterns. In fact, there is proof that a fairly simple neural network can fit any practical function.

Suppose, for instance, that you have data from a housing application [HaRu78]. You want to design a network that can predict the value of a house (in $1000s), given 13 pieces of geographical and real estate information. You have a total of 506 example homes for which you have those 13 items of data and their associated market values.

You can solve this problem in three ways:

• Use a command-line function, as described in Using Command-Line Functions.
• Use a graphical user interface, nftool, as described in Using the Neural Network Fitting Tool GUI.
• Use nntool, as described in Graphical User Interface.

Defining a Problem

To define a fitting problem for the toolbox, arrange a set of Q input vectors as columns in a matrix. Then, arrange another set of Q target vectors (the correct output vectors for each of the input vectors) into a second matrix. For example, you can define the fitting problem for a Boolean AND gate with four sets of two-element input vectors and one-element targets as follows:

• inputs = [0 1 0 1; 0 0 1 1];
  targets = [0 0 0 1];
  

The next section demonstrates how to train a network from the command line, after you have defined the problem. This example uses the housing data set provided with the toolbox.

Using Command-Line Functions

1. Load the data, consisting of input vectors and target vectors, as follows:

   load house_dataset

2. Create a network. For this example, you use a feed-forward network with the default tan-sigmoid transfer function in the hidden layer and linear transfer function in the output layer. This structure is useful for function approximation (or regression) problems. Use 20 neurons (somewhat arbitrary) in one hidden layer. The network has one output neuron, because there is only one target value associated with each input vector.

   net = newfit(houseInputs,houseTargets,20);

Note    More neurons require more computation, but they allow the network to solve more complicated problems. More layers require more computation, but their use might result in the network solving complex problems more efficiently.

3. Train the network. The network uses the default Levenberg-Marquardt algorithm for training. The application randomly divides input vectors and target vectors into three sets as follows:
• 60% are used for training.
• 20% are used to validate that the network is generalizing and to stop training before overfitting.
• The last 20% are used as a completely independent test of network generalization.

1. To train the network, enter:

   net=train(net,houseInputs,houseTargets);

   During training, the following training window opens. This window displays training progress and allows you to interrupt training at any point by clicking Stop Training.

   

   This example used the train function. All the input vectors to the network appear at once in a batch. Alternatively, you can present the input vectors one at a time using the adapt function. Training Styles describes the two training approaches.

   This training stopped when the validation error increased for six iterations, which occurred at iteration 23. If you click Performance in the training window, a plot of the training errors, validation errors, and test errors appears, as shown in the following figure. In this example, the result is reasonable because of the following considerations:

   • The final mean-square error is small.
   • The test set error and the validation set error have similar characteristics.
   • No significant overfitting has occurred by iteration 17 (where the best validation performance occurs).

     

4. Perform some analysis of the network response. If you click Regression in the training window, you can perform a linear regression between the network outputs and the corresponding targets.

1. The following figure shows the results.

   

The output tracks the targets very well for training, testing, and validation, and the R-value is over 0.95 for the total response. If even more accurate results were required, you could try any of these approaches:

• Reset the initial network weights and biases to new values with init and train again.
• Increase the number of hidden neurons.
• Increase the number of training vectors.
• Increase the number of input values, if more relevant information is available.
• Try a different training algorithm (see Speed and Memory Comparison).

In this case, the network response is satisfactory, and you can now use sim to put the network to use on new inputs.

To get more experience in command-line operations, try some of these tasks:

• During training, open a plot window (such as the regression plot), and watch it animate.
• Plot from the command line with functions such as plotfit, plotregression, plottrainstate and plotperform. (For more information on using these functions, see their reference pages.)

Using the Neural Network Fitting Tool GUI

1. Open the Neural Network Fitting Tool with this command:

   nftool

2. Click Next to proceed.
3. Click Load Example Data Set in the Select Data window. The Fitting Data Set Chooser window opens.

Note    You use the Inputs and Targets options in the Select Data window when you need to load data from the MATLAB® workspace.

4. Select Simple Fitting Problem, and click Import. This brings you back to the Select Data window.
5. Click Next to display the Validate and Test Data window, shown in the following figure.

1. The validation and test data sets are each set to 15% of the original data.
   
   

6. Click Next.

1. The number of hidden neurons is set to 20. You can change this value in another run if you want. You might want to change this number if the network does not perform as well as you expect.

7. Click Next.
8. Click Train.

1. This time the training continued for the maximum of 1000 iterations.

9. Under Plots, click Regression.

1. For this simple fitting problem, the fit is almost perfect for training, testing, and validation data.

1. These plots are the regression plots for the output with respect to training, validation, and test data.

10. View the network response. For single-input/single-output problems, like this simple fitting problem, under the Plots pane, click Fit.

    

1. The blue symbols represent training data, the green symbols represent validation data, and the red symbols represent testing data. For this problem and this network, the network outputs match the targets for all three data sets.

11. Click Next in the Neural Network Fitting Tool to evaluate the network.

    

1. At this point, you can test the network against new data.

   If you are dissatisfied with the network's performance on the original or new data, you can take any of the following steps:

   • Train it again.
   • Increase the number of neurons.
   • Get a larger training data set.

12. If you are satisfied with the network performance, click Next.
13. Use the buttons on this screen to save your results.
• You have the network saved as net1 in the workspace. You can perform additional tests on it or put it to work on new inputs, using the sim function.
• You can also click Generate M-File to create an M-file that can be used to reproduce all of the previous steps from the command line. Creating an M-file can be helpful if you want to learn how to use the command-line functionality of the toolbox to customize the training process.
14. When you have saved your results, click Finish.

Provide feedback about this page

Using the Documentation

Recognizing Patterns

© 1984-2009- The MathWorks, Inc.    -   Site Help   -   Patents   -   Trademarks   -   Privacy Policy   -   Preventing Piracy   -   RSS