Supervised Learning

How Supervised Learning Works

Supervised learning is the most common type of machine learning. It uses a known data set (called the training data set) to train an algorithm with a known set of input data (called features) and known responses. The training data set includes labeled input data that pairs with desired outputs or response values. From this data set, the supervised learning algorithm seeks to create a model by discovering relationships between the features and output data, and then makes predictions of the response values for a new data set.

Supervised learning can be used in several engineering and scientific fields for developing models that solve real-world problems. For example, consider clinicians having data on previous patients, including age, weight, height, and blood pressure. They know whether the previous patients had heart attacks and want to predict whether a new patient is at risk for a heart attack. This can be done by using supervised learning to model the existing data.

A typical supervised learning workflow involves steps such as data preparation, algorithm selection, model training, and model evaluation. Because supervised learning relies on labeled data to train models, it is crucial that the training data is realistic. For example, if the model is going to be used in a noisy environment, then the training data must also contain realistic levels of noise. Validation is another important step in supervised learning, to ensure that the model generalizes well to unseen data and to prevent overfitting.

Improving Supervised Learning Model

Improving a supervised learning model—that is, increasing its accuracy and predictive power—often involves feature engineering and hyperparameter tuning. Feature engineering is the process of turning raw data into features to be used as inputs of a machine learning model. Hyperparameter tuning involves identifying the set of parameters that provides the best model.

Feature engineering involves steps such as feature selection and feature transformation. In feature selection, you identify the most relevant features, or variables, that provide the best predictive power in modeling your data. During feature transformation, existing features are turned into new features using principal component analysis, nonnegative matrix factorization, factor analysis, and other techniques.

In hyperparameter tuning, you try to find the best settings for the parameters that control the learning process, such as learning rate, batch size, and number of epochs.

Supervised Learning vs. Unsupervised Learning

Supervised and unsupervised learning are both types of machine learning.

The main difference between supervised and unsupervised learning is that supervised learning requires labeled training data to train a machine learning model. In contrast, unsupervised learning makes use of unlabeled data to uncover hidden relationships in the data without human intervention. Supervised learning results can be more accurate than unsupervised learning results due to the presence of labeled data. However, acquiring labeled data often requires human effort and can be time-consuming, expensive, and impractical in certain cases.

Prior to applying supervised learning, unsupervised learning is sometimes used to discover patterns in the input data and identify features for supervised learning. In addition to identifying features, the correct category or response needs to be identified for all observations in the training set. Semi-supervised learning, also a machine learning technique, lets you train models with less labeled data and thus reduce the labeling effort.

Types of Supervised Learning

The main types of machine learning models created by supervised learning algorithms are classification and regression models. For each type of task, classification or regression, different algorithms can be used for modeling your data.

Supervised Learning for Classification

In supervised learning, classification refers to separating the data into specific classes or categories. The goal is to assign a class (or label) from a finite set of classes to an observation. That is, the responses of a classification model are categorical variables.

Typical applications include medical imaging, speech recognition, and credit scoring. Examples include determining whether an email is genuine or spam or whether equipment parts are defective or good.

Supervised Learning for Regression

You can use regression techniques in supervised learning to understand the relationship between response and input variables. They are useful for data sets with a range of values or when the response is a real number, like temperature or time until equipment failure. That is, regression models predict continuous responses. Typical applications include electricity load forecasting, predicting the remaining cycle life of batteries, algorithmic trading, disease incidence, forecasting stock prices, and acoustic signal processing.

Supervised Learning for Classification or Regression

Several supervised learning algorithms can be used for both classification and regression tasks.

Deep learning, a specialized form of machine learning, leverages deep neural networks to handle more complex tasks and larger data sets. Popular deep learning algorithms such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and long short-term memory (LSTM) networks are typically supervised.

Choosing the Right Supervised Learning Algorithm

When you are choosing a supervised learning algorithm for your data, you must consider the trade-offs between different algorithms, such as their computational and memory requirements, predictive accuracy, and interpretability. This process might involve trial and error and can be made easier by applying automated machine learning (AutoML).

Why Supervised Learning Is Important

Supervised learning is a major area of machine learning and artificial intelligence. Engineers and scientists are using supervised learning to build AI models to solve real-world problems. For instance, in virtual sensor modeling, AI models that are used for developing these sensors are trained with supervised learning algorithms. Some of the application areas for supervised learning are:

• Bioinformatics and medical devices: Supervised learning techniques are widely used in healthcare for medical imaging, tumor detection, and drug discovery. Supervised learning can also be used for signal-based tasks such as classifying ECG data to distinguish between arrhythmia, congestive heart failure, and normal sinus rhythm.
• Energy production: By using supervised learning for regression, engineers can predict energy prices, forecast electricity load, and create models for various renewable energy applications.
• Industrial automation: Supervised learning can be used to develop predictive models for anomaly detection, fault detection, and diagnosis.
• Quantitative finance and risk assessment: Supervised learning algorithms are used for credit scoring, algorithmic trading, stock price prediction, and bond classification.

Real-World Supervised Learning Example

Battery state of charge (SOC) is the level of charge of an electric battery relative to its capacity measured as a percentage. SOC must be accurately estimated in many applications, such as in electric vehicles. Lithium-ion SOC estimation using a physics-based modeling approach is very complex due to varying operating conditions and significant device variability even with batteries from the same manufacturer.

Using supervised learning to create a predictive model for SOC estimation is a data-driven approach that can help address these challenges. The model can predict the state of charge of a vehicle’s lithium-ion battery from time series data that represents various battery measurements, such as voltage, current, and temperature, and derived features, such as average voltage and current. Learn more about this example.

Supervised Learning with MATLAB

MATLAB^® enables you to apply supervised learning and build predictive models. You can train, validate, and tune supervised learning models with Statistics and Machine Learning Toolbox^™ and Deep Learning Toolbox^™. You can combine machine learning and deep learning tools with other MATLAB toolboxes to perform industry-specific workflows. Using built-in functions and interactive apps, you can process and label data, train models, visualize model performance, and tune hyperparameters for improving model performance.

Data Preparation

MATLAB supports signal, visual, and text data. You can access and explore your data at the command line or interactively with low-code apps:

• Choose an app to label ground truth data to validate or train supervised learning algorithms such as image classifiers, object detectors, semantic segmentation networks, and deep learning applications.
• Use the Data Cleaner app and data preprocessing Live Editor task to preprocess data.

For feature engineering, MATLAB provides built-in tools for most feature extraction and feature selection methods.

AI Modeling with Low-Code Apps

The Classification Learner and Regression Learner apps help you apply supervised learning to the design, tuning, assessment, and optimization of machine learning models for classification and regression, respectively. With these machine learning apps, you can:

• Train models on different machine learning algorithms.
• Visualize results and evaluate model performance.
• Compare the performance of multiple models.
• Automatically generate code for a trained model.
• Export a trained model to the workspace, Simulink^®, and MATLAB Production Server™.

With the Deep Network Designer app, you can design, analyze, and modify deep learning networks interactively. You can also load pretrained networks or import networks from PyTorch^® and TensorFlow^™.

Beyond AI modeling, with the Experiment Manager app, you can import and manage multiple machine learning and deep learning experiments, keep track of training parameters, adjust hyperparameters, analyze results, and compare code from different experiments.

Supervised Learning and Embedded AI

With MATLAB and Simulink, you can design, simulate, test, verify, and deploy AI models, built with supervised learning, that enhance the performance and functionality of complex embedded systems. Use dedicated Simulink blocks to simulate and test the integration of AI models into complex systems. Automatically generate code optimized for deployment to resource-constrained targets.