How to find the Relative Root Mean Square Error for the given data?

I have some data as given below:

u=[-30 30 -50 50];% desired vector
low = [-90 -90 -90 -90];
up = [90 90 90 90];
b = low + (up - low) .* randn(1,4);% Estimated vetors

How will we find the Relative Root Mean Sqaure Error (RRMSE) for this? Further, what does the RRMSE show?

[Manikanta Aditya]

Hello,

The Relative Root Mean Square Error (RRMSE) is a normalized measure of the differences between values predicted by a model and the values actually observed. It is calculated as the Root Mean Square Error (RMSE) divided by the mean of the observed values.

Here’s how you can calculate the RRMSE in MATLAB for your given data:

u = [-30 30 -50 50]; % desired vector
low = [-90 -90 -90 -90];
up = [90 90 90 90];
b = low + (up - low) .* randn(1,4); % Estimated vectors

% Step 1: Calculate squared differences
squared_diffs = (u - b).^2;

% Step 2: Calculate mean of squared differences
mean_squared_diffs = mean(squared_diffs);

% Step 3: Calculate RMSE
rmse = sqrt(mean_squared_diffs);

% Steps 4 and 5: Calculate the square root of the mean squared desired values
mean_squared_u = mean(u.^2);
sqrt_mean_squared_u = sqrt(mean_squared_u);

% Step 6: Calculate RRMSE
rrmse = rmse / sqrt_mean_squared_u;

% Display the RRMSE
disp(['RRMSE: ', num2str(rrmse)]);

RRMSE: 4.2349

Hope this helps!

[Sadiq Akbar]

@Manikanta Aditya

Thanks a lot for your kind response. Ys, it works. But this was a dummy data. My original data is attached here in the form of a mat file 2sn0dB. The above estimated vector "b" is named here in this mat file as "two". So, if I want to do the same process for this, how will I modify the code?

[Manikanta Aditya]

Hey,

Try out this example code and see once:

% Load the data from the .mat file
load('2sn0dB.mat', 'two'); % 'two' is the variable name in the file

% Your desired vector
u = [-30 30 -50 50]; % You might need to adjust this based on your actual desired values

% Reshape or adjust 'u' and 'two' if necessary to ensure they match in dimensions
% This step is crucial if 'u' and 'two' do not have the same dimensions
% For the sake of this example, let's assume 'two' is a 2D matrix where each row represents an estimation corresponding to 'u'
% Thus, we need to replicate 'u' to match the number of rows in 'two'

u_replicated = repmat(u, size(two, 1), 1); % Replicate 'u' to match 'two's dimensions

% Now, calculate the RRMSE
% Step 1: Calculate squared differences
squared_diffs = (u_replicated - two).^2;

% Step 2: Calculate mean of squared differences
mean_squared_diffs = mean(squared_diffs, 'all'); % Use 'all' to average over all elements

% Step 3: Calculate RMSE
rmse = sqrt(mean_squared_diffs);

% Step 4 and 5: Calculate the square root of the mean squared desired values
mean_squared_u = mean(u.^2);
sqrt_mean_squared_u = sqrt(mean_squared_u);

% Step 6: Calculate RRMSE
rrmse = rmse / sqrt_mean_squared_u;

% Display the RRMSE
disp(['RRMSE: ', num2str(rrmse)]);

Thanks.

[Sadiq Akbar]

@Manikanta Aditya

Thanks a lot for your kind response. I ran it but it gives this error:

Arrays have incompatible sizes for this operation.

Error in RRMSE_Mathworks (line 18)

squared_diffs = (u_replicated - two).^2;

Related documentation

>>

[Dyuman Joshi]

@Sadiq Akbar, you should read the comments included in the code above.

[Sadiq Akbar]

Yes, I read and made the dimension of both equal and it worked. But If we want to determine the RMSE and the RRMSE for the above data when u=[-30 30]; Then what is the diffeence between them. Further and most important if we want to draw a plot for both the metrics i.e., RMSE and RRMSE for the above data, then how will be that and what is the difference between both? I mean what extra information is given by RRMSE than RMSE?

Answer 1

[Divyam]

Hi @Sadiq Akbar,

@Manikanta Aditya's comment (https://www.mathworks.com/matlabcentral/answers/2091226-how-to-find-the-relative-root-mean-square-error-for-the-given-data#comment_3090826) perfectly summarizes the workflow for finding the RRMSE.

To compare by drawing a plot for RMSE and RRMSE, you can simply use a barplot and check the values for each using the code below in addition to the code provided by Manikanta:

metrics = [rmse, rrmse];
metric_names = {'RMSE', 'RRMSE'};

figure;
bar(metrics);
set(gca, 'xticklabel', metric_names);
ylabel('Error');
title('Comparison of RMSE and RRMSE');

The key difference between RMSE and RRMSE is that RRMSE normalizes the RMSE value by dividing RMSE by the mean of the observed values. RRMSE is a better measure for comparing error values across different datasets. RRMSE thus makes the error relative to the size of the data and hence removes the influence of scaling on datasets.

Dataset 1:
observed = [100, 120, 150]
predicted = [98, 123, 147]
RMSE = 2.708
RRMSE = 0.021788

Dataset 2:
observed = [1000, 1200, 1500]
predicted = [980, 1230, 1470]
RMSE = 27.0801
RRMSE = 0.021788

Notice how the values for RMSE in the above example vary heavily with the values in the dataset but RRMSE stays constant despite the scaling.