File Exchange

image thumbnail

weighted total least squares straight line fit

version (3.01 KB) by Mathias Anton
Calculates the parameters (and their uncertainties) to data with uncertainties in both coordinates


Updated 13 Nov 2007

View License

The problem of fitting a straight line to data with uncertainties in both coordinates is solved using a weighted total least-squares algorithm. The parameters are transformed from the usual slope/y-axis intersection pair to slope angle and distance to the origin. The advantages of this are that a) global convergence is assured b) a solution is found even for a vertical line. The complete uncertainty matrix (i.e. variances AND covariance of the fitting parameters) is determined. For non-vertical straight lines the usual parameters (slope/y-axis intersect.) are also given, together with their uncertainty matrix. The algorithm is especially useful for precision measurements, where the knowledge of the complete uncertainty matrix is a must. The algorithm was published in Measurement Science and Technology 18 (2007) pp3438-3442 by M.Krystek and M.Anton, Physikalisch-Technische Bundesanstalt Braunschweig, Germany. An attached script named pearson_york_tetdata.m contains a standard statistical test data set for the problem (see e.g. Lybanon,M in Am.J.Phys.52(1)1984 pp22-26)

Cite As

Mathias Anton (2020). weighted total least squares straight line fit (, MATLAB Central File Exchange. Retrieved .

Comments and Ratings (7)


is there a simply way to fit a custom function and not just a straight line? I've been looking for a matlab function to do total least square with a custom function for ages.

Ariel Matusevich

Dear Mathias,

There might be a little bug in your function.
Consider the following example:


In this case, the program fails, a=0.9064 and b=NaN.

However, if you remove the first data pair (0,0):


The program works perfectly well:
a=2.0000 and b=6.6781e-007.

The same problem occurs with the latest version of the algorithm (wtlsc_line).

Congratulations for your excellent work!

Kind regards,


Hassan R-Esfahlan

I am wondering what is the uncertainties of a and b in fonction of uncertainties in xin and yin?

Hassan R-Esfahlan

I am wondering what is the uncertainties of a and b in fonction of uncertainties in xin and yin?

Adam Baker

Mathias Anton


xin is the input (abscissa)
uxin is the standard (measurement) uncertainty of the x input. yin is the input of the ordinate and uyin the associated standard measurement uncertainty. The input vector uxin should have the same size as xin, i.e. for every x there should be an associated uncertainty value. The same holds for yin and uyin. Of course, the algorithm will yield the same result as an OLS algorithm if no uncertainties in x are present (uxin=zeros(size(xin)).

Marcelo Perlin

Hi Mathias,

Can you provide examples scripts for the use of your function.

I'm trying it out, but I can understand in which cases your proposed method is superior to simple OLS.

What do you mean by:
uxin (standard) uncertainties of xin, same size as xin

uyin (standard) uncertainties of yin, same size as yin

I would be great if you could show that (superiority over OLS and description of inputs) at an example script.


MATLAB Release Compatibility
Created with R2007a
Compatible with any release
Platform Compatibility
Windows macOS Linux

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!