Matlab is giving different answers for the same calculation

Matlab is showing different results for the same calculation when done with direct values instead of using variables.

The calculation is: h = k*Nu/D_h

k = 0.129, Nu = 8.24, D_h = 0.0019

After running the script, the answer produced is: 562.7917

Doing the same calculation in the command window through the variables produces the same answer,

however the answer produced while doing the calculation in the command window using direct values is different.

Running " 0.129*8.24/0.0019 " in the command window produces the result: 559.4526

Matlab Script:

rho = 916;        %kg/m^3
C_p = 1907;         %J/kg
k   = 0.129;      %W/m*K
v   = 1.6400e-05;      %m^2/s
A_c = 1.2985e-05;
P = 0.0275;
D_h = 4*A_c/P

D_h = 0.0019

Flow_Rate = 0.000075;   %m^3/s
vel = Flow_Rate/(12*A_c)

vel = 0.4813

Re = vel*D_h/v

Re = 55.4324

Nu = 8.24

Nu = 8.2400

h = (k*Nu)/D_h

h = 562.7917

Script Results:

Command Window Results:

In case its relevant, the system running Matlab is 64 bit and the version of Matlab being used is:

MATLAB Version: 23.2.0.2515942 (R2023b) Update 7

Operating System: Microsoft Windows 10 Home Single Language Version 10.0 (Build 19045)

Java Version: Java 1.8.0_202-b08 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode

However, the same issue was observed on Matlab online.

The answer for this calculation from other sources is the same as the result when the direct values were used (559.4526).

Why is Matlab producing different results, and can anything be done to resolve this?

Answer 1

[Sam Chak]

Hi @Stephen

The discrepancy observed can be attributed to numerical precision errors introduced by humans. The scripted answer is the accurate one in this case. By default, the displayed answer is rounded to four decimal places. If you directly input the numerical values from the displayed answer into the formula, the computation accuracy will be compromised as it truncates beyond the displayed precision.

[Sam Chak]

%% Script
format long
rho = 916;          % kg/m^3
C_p = 1907;         % J/kg
k   = 0.129;        % W/m*K
v   = 1.6400e-05;   % m^2/s
A_c = 1.2985e-05;
P   = 0.0275;
D_h = 4*A_c/P

D_h = 
   0.001888727272727

Flow_Rate = 0.000075;   % m^3/s
vel = Flow_Rate/(12*A_c);
Re  = vel*D_h/v;
Nu  = 8.24;
format short
h   = (k*Nu)/D_h

h = 562.7917

%% Manually enter the values
h   = 0.129*8.24/0.001888727272727

h = 562.7917

[Stephen]

Thank you!

[Sam Chak]

Hi @Stephen

It's important to acknowledge that human errors are relatively common, especially when working on tasks manually and individually (one-man show), as is the case with experts as well.

[Dyuman Joshi]

Man I gotta change my keyboard, the keys keep getting stuck so much.

Answer 2

[Dyuman Joshi]

That's because the value of D_h is not 0.0019.

The value stored is not the same as value displayed.

rho = 916;        %kg/m^3
C_p = 1907;         %J/kg
k   = 0.129;      %W/m*K
v   = 1.6400e-05;      %m^2/s
A_c = 1.2985e-05;
P = 0.0275;

%Change the format
format long
D_h = 4*A_c/P

D_h = 
   0.001888727272727

The value is a terminating irrational value.

Now, let's see the result of manual calculation in limited precision of floating point values-

out = 0.129*8.24/0.001888727272727

out = 
     5.627916827109015e+02

which in the default (short) format is

format default
out

out = 562.7917

which matches with the output you obtained.

[Stephen]

Thank you!

[Sam Chak]

By the way @Stephen, you should also vote for the helpful explanations as tokens of appreciation.