A neural network (also called an artificial neural network or ANN) is an adaptive system that learns by using interconnected nodes or neurons in a layered structure that resembles a human brain. A neural network can learn from data, so it can be trained to recognize patterns, classify data, and forecast future events. A neural network breaks down the input into layers of abstraction. It can be trained using many examples to recognize patterns in speech or images just as the human brain does. The neural network behavior is defined by the way its individual elements are connected and by the strength, or weights, of those connections. These weights are automatically adjusted during training according to a specified learning rule until the artificial neural network performs the desired task correctly.
What Are Neural Networks Used For?
Neural networks are a type of machine learning approach inspired by how neurons signal to each other in the human brain. Neural networks are especially suitable for modeling nonlinear relationships, and they are typically used to perform pattern recognition and classify objects or signals in speech, vision, and control systems.
Neural networks, particularly deep neural networks, have become known for their proficiency at complex identification applications such as face recognition, text translation, and voice recognition. These approaches are a key technology driving innovation in advanced driver assistance systems and tasks, including lane classification and traffic sign recognition.
Neural Network Examples
Here are a few examples of how neural networks are used in machine learning applications:
- Segmenting images and videos semantically
- Detecting objects in images, including pedestrians and bicyclists
- Training a biped robot to walk using reinforcement learning
- Detecting cancer by guiding pathologists to classify tumors
- Approximating solutions to differential equations
- And more
How Do Neural Networks Work?
Inspired by biological nervous systems, a neural network combines several processing layers using simple elements operating in parallel. The network consists of an input layer, one or more hidden layers, and an output layer. In each layer there are several nodes, or neurons, and the nodes in each layer use the outputs of all nodes in the previous layer as inputs, such that all neurons interconnect with each other through the different layers. Each neuron is typically assigned a weight that is adjusted during the learning process. Decreases or increases in the weight change the strength of that neuron’s signal.
Typical neural network architecture.
Like other machine learning algorithms, neural networks can be used for classification or regression tasks. Model parameters are set by weighting the neural network through learning on training data, typically by optimizing weights to minimize prediction error.
Types of Neural Networks
The first and simplest neural network was the perceptron, introduced by Frank Rosenblatt in 1958. It consisted of a single neuron and essentially a linear regression model with a sigmoid activation function. Since then, increasingly complex neural networks have been explored, leading up to today’s deep networks, which can contain hundreds of layers.
Deep learning refers to neural networks with many layers, whereas neural networks with only two or three layers of connected neurons are also known as shallow neural networks. Deep learning has become popular because it eliminates the need to extract features from images, which previously challenged the application of machine learning to image and signal processing. Although feature extraction can be omitted in image processing applications, some form of feature extraction is still commonly applied to signal processing tasks to improve model accuracy.
There are three common types of neural networks used for engineering applications:
- Feedforward neural network: Consists of an input layer, one or a few hidden layers, and an output layer (a typical shallow neural network).
- Convolutional neural network (CNN): Deep neural network architecture widely applied to image processing and characterized by convolutional layers that shift windows across the input with nodes that share weights, abstracting the (typically image) input to feature maps. You can use pretrained CNN networks, such as SqueezeNet or GoogleNet.
- Recurrent neural network (RNN): Neural network architecture with feedback loops that model sequential dependencies in the input, as in time series, sensor, and text data; the most popular type of RNN is a long short-term memory network (LSTM).
How Do CNNs Work?
Watch this short video with the specifics of CNNs, including layers, activations, and classification.
Learn more about deep learning:
How Do You Create a Neural Network with MATLAB?
Using MATLAB® with Deep Learning Toolbox™ and Statistics and Machine Learning Toolbox™, you can create deep and shallow neural networks for applications such as computer vision and automated driving.
With just a few lines of code, you can create neural networks in MATLAB without being an expert. You can get started quickly, train and visualize neural network models, and integrate neural networks into your existing system and deploy them to servers, enterprise systems, clusters, clouds, and embedded devices.
Typical Workflow for Building Systems with Neural Networks
Developing AI applications, and specifically including neural networks, typically involves these steps:
- Data Preparation
- Acquire sufficient labeled training data, with much more required to train deep neural networks; labeler apps such as the image, video, and signal labeled can expedite this process.
- Use simulation to generate training data, especially if gathering data from real systems is impractical (failure conditions).
- Augment data to represent more variability in training data.
- AI Modeling
- Train shallow neural networks interactively in Classification and Regression Learner from Statistics and Machine Learning Toolbox, or use command-line functions; this is recommended if you want to compare the performance of shallow neural networks with other conventional machine learning algorithms, such as decision trees or SVMs, or if you have only limited labeled training data available.
- Specify and train neural networks (shallow or deep) interactively using Deep Network Designer or command-line functions from Deep Learning Toolbox, which is particularly suitable for deep neural networks or if you need more flexibility in customizing network architecture and solvers.
- Simulation and Test
- Integrate neural networks in Simulink® models as blocks, which can facilitate integration with a larger system, testing, and deployment to many types of hardware.
- Deployment
- Generate C/C++ code from shallow neural networks trained in Statistics and Machine Learning Toolbox for deployment to embedded hardware and high-performance computing systems.
- Generate optimized CUDA® and C/C++ code from neural networks trained in Deep Learning Toolbox for fast inference on GPUs and other types of industrial hardware (ARM, FPGA).
A neural network is an adaptive system that learns by using interconnected nodes or neurons in a layered structure that resembles a human brain. It can be trained to recognize patterns, classify data, and forecast future events by breaking down input into layers of abstraction.
Neural networks are used for pattern recognition and classification tasks in speech, vision, and control systems, including applications like face recognition, text translation, voice recognition, lane classification, and traffic sign recognition.
Neural networks learn by automatically adjusting the weights of connections between neurons during training according to a specified learning rule, typically by optimizing weights to minimize prediction error until the network performs the desired task correctly.
Shallow neural networks have only two or three layers of connected neurons, while deep neural networks contain many layers (sometimes hundreds) and are used in deep learning applications that eliminate the need for manual feature extraction.
The three common types are feedforward neural networks (with input, hidden, and output layers), convolutional neural networks (CNNs for image processing), and recurrent neural networks (RNNs for sequential data like time series and text).
A CNN is a deep neural network architecture widely applied to image processing, characterized by convolutional layers that shift windows across the input with nodes that share weights, abstracting image input to feature maps.
An RNN is a neural network architecture with feedback loops that model sequential dependencies in time series, sensor, and text data, with long short-term memory networks (LSTMs) being the most popular type.
You can create neural networks in MATLAB using Deep Learning Toolbox and Statistics and Machine Learning Toolbox with just a few lines of code, train models interactively using apps like Classification Learner or Time Series Modeler, and deploy them to various hardware platforms.
