Machine learning - mathcing estimation signal to the known model

Hi, I recently started getting into machine learning and training the model to try to get the numerical model based on the measured data.

First, to understand and get familiar with the methods, I started with the known model. I have three sinusoidal signals, , , and , and the output is calculated as:

... Eq. (1)

An example of the signals:

I have data stored in X_data.txt and Y_data.txt for different cases of, , and amplitudes, and calculated , but when I ran the code:

X = readmatrix('X_data.txt');
Y = readmatrix('Y_data.txt');

% Split Data into Training and Testing Sets
cv = cvpartition(size(X,1),'HoldOut',0.3);
XTrain = X(training(cv), :);
YTrain = Y(training(cv), :);
XTest = X(test(cv), :);
YTest = Y(test(cv), :);

% Train a Machine Learning Model
% fitrlinear (linear regression)
mdl = fitrlinear(XTrain, YTrain);

% Validate the Model
YPred = predict(mdl, XTest);
mse = mean((YPred - YTest).^2);
fprintf('Mean Squared Error: %.4f\n', mse);

Mean Squared Error: 0.0214

% Extract the "Equation"
disp(mdl.Beta);

   -0.5468
   -0.5466
   -0.5457

disp(mdl.Bias);

    0.0019

I do not get in the form of Eq. (1), but in the form of

For a test point, e.g.

[X(500,:), Y(500,:)]

ans = 1×4

-0.1418 0.7822 -1.1265 0.1721

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

Eq. (1) gives a correct value:

But the model estimation gives a wrong result:

Do you know how I can get an estimation close to or the same as Eq. (1), even maybe using max and min functions?

Edit/update.

I understand I used a linear model (fitrlinear), which cannot give the output in the form of max and min functions. Are there any other models for that?

Answer 1

[Shantanu Dixit]

Hi Mike,

Since the target output is defined using non-linear operations like 'max' and 'min', a linear model (such as 'fitrlinear') won't be sufficient to accurately capture this behavior. Linear regression is limited to modeling relationships that are strictly linear combinations of the input features, which doesn't apply in your case.

To better approximate your output (as in Eq. (1)), you can consider using non-linear regression models that can learn complex, piecewise, and non-linear patterns in data. Below are some suitable options:

Tree Based Models (Decision Trees / Random Forests / Gradient Boosting): Such models can handle piecewise defined functions like (max/min) by partitioning the input space into regions.

% Train a decision tree
mdl = fitrtree(XTrain, YTrain, ...);  % other arguments
YPred = predict(mdl, XTest);
mse = mean((YPred - YTest).^2);
% You can also try 'fitrensemble' for ensemble-based methods like random forests or boosting.

Neural Networks: Can be used to approximate arbitrary functions (including max/min) with sufficient depth and activation functions like ReLU. You can use MATLAB's 'fitrnet' for training a feedforward neural network for regression (given sufficient data to train the model).

% Train a neural network
mdl = fitrnet(XTrain, YTrain, ...); % other arguments
YPred = predict(mdl, XTest);

For detailed information on training non-linear models you can refer to the following documentation:

Decision Trees: https://www.mathworks.com/help/stats/decision-trees.html

Neural Networks: https://www.mathworks.com/discovery/neural-network.html

fitrnet: https://www.mathworks.com/help/stats/fitrnet.html

fitrtree: https://www.mathworks.com/help/stats/fitrtree.html

Hope this helps!