Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
How to import data without losing much precision

Subject: How to import data without losing much precision

From: Jeff

Date: 2 Dec, 2012 22:19:08

Message: 1 of 6

I'm trying to import some data, which was created by C code, into Matlab without losing much precision. Can someone show me an example of how to do so?

I have about 52 decimal places of precision, with N=16 numbers on a line, like this:

u1.csv.txt:
----------
0,0.707106781186547461715008466853760182857513427734375,1,0.7071067811865475727373109293694142252206802368164062,1.22464679914735320717376402945839660462569212467758e-16,-0.707106781186547461715008466853760182857513427734375,-1,-0.7071067811865476837596133918850682675838470458984375,0,0,0,0,0,0,0,0
-7.519528416073247371557097289336037138074431140921661e-05,0.2952479347203849559418391973464323996267921756953001,0.4173620664315570311651056728541320239855849649757147,0.2952479706713651090284420558385036770232545677572489,-7.524612652097207200814894090907733459339112869201927e-05,-0.2950974933097045235536482615801290307899762410670519,-0.4175125078422409145146015213567736168442934285849333,-0.2950975292606799753227907884278735650696035008877516,-7.307347221309693227484088353350377698092188438749872e-05,-0.4908319110134666044518731570223124549556814599782228,-0.6943176939679628171002076508777633989666355773806572,-0.4908316435424278546958368890651414062631374690681696,-7.345173337937191360150656260687845877654922333022114e-05,0.4909784362190551817528658823386322751503030303865671,0.6941711687623717434237127776874842766119400039315224,0.490978168748022819492613649794421348815376404672861
1
...

I've tried using dlmread, csvread and importdata, but they cut off precision.

Now I am trying to use textread, because the format string specifier might let me keep precision. But I have not had any luck. I tried following example 4 in the documentation:

EDU> data=textread('u1.csv.txt', '%60.51f', 16, 'delimiter', ',');
EDU> sprintf('%.20e', data(2,1))
ans =
-7.51952841607324750000e-005

EDU> data=textread('u1.csv.txt', '%f', 'delimiter', ',')
(doesn't import data as a matrix)
EDU> sprintf('%.20e', data(2,1))
ans =
7.07106781186547460000e-001

EDU> data=textread('u1.csv.txt', '%f', 16, 'delimiter', ',')
(reads only one line, but there's probably a way to read the "next" line)
sprintf('%.20e', data(2,1))
ans =
7.07106781186547460000e-001

I've tried other things, too, but I cannot get any more precision. You can see that their all losing precision after about 16 decimal places.

I have (and can create) other formats for the input file. Other formats that I have are u1.fixedWidth.txt and u1.tab.txt, which look like you'd expect them to look (u1.fixedWidth is printed with "%60.52f" and u1.tab.txt is printed as "%.52f\t"). But this also loses precision:

EDU>> data=textread('u1_0000000.fixedWidth.txt', '%60.52f', 16);
EDU>> sprintf('%.20e', data(2,1))
ans =
7.07106781186547460000e-001

Could someone show me an example of how to import my data with the most precision?
Thanks you.

Note that I am using student version of Matlab R2009a (and cannot upgrade it) on a 64-bit Windows 7 machine. I don't really know how much precision my version of Matlab has (or how to find out).

Subject: How to import data without losing much precision

From: Steven_Lord

Date: 3 Dec, 2012 03:54:46

Message: 2 of 6



"Jeff " <spREMOVEHITSjeffAT@SIGNoptonline.net> wrote in message
news:k9gk4s$2hh$1@newscl01ah.mathworks.com...
> I'm trying to import some data, which was created by C code, into Matlab
> without losing much precision. Can someone show me an example of how to do
> so?
>
> I have about 52 decimal places of precision, with N=16 numbers on a line,
> like this:

Double precision only allows you 52 bits of precision (well, plus the
implicitly stored bit) not 52 digits.

*snip*

> I've tried using dlmread, csvread and importdata, but they cut off
> precision.

You could read these in as _strings_ and convert them into symbolic objects
using Symbolic Math Toolbox.

*snip*

> I've tried other things, too, but I cannot get any more precision. You can
> see that their all losing precision after about 16 decimal places.

Yes. See:

http://en.wikipedia.org/wiki/Double-precision_floating-point_format

> Could someone show me an example of how to import my data with the most
> precision?

Why do you believe you need 52 decimal digits of precision for your data?
For comparison, you only need roughly 39 digits of pi to calculate the
volume of the universe to within one atom.

http://en.wikipedia.org/wiki/Pi#Motivations_for_computing_.CF.80

Just because you HAVE the number out to N digits doesn't mean you NEED all
those digits.

--
Steve Lord
slord@mathworks.com
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

Subject: How to import data without losing much precision

From: Jeff

Date: 3 Dec, 2012 21:44:28

Message: 3 of 6

Hi Steven. Thanks for your reply.

"Steven_Lord" <slord@mathworks.com> wrote in message <k9h7q6$4sk$1@newscl01ah.mathworks.com>...
>
>
> "Jeff " <spREMOVEHITSjeffAT@SIGNoptonline.net> wrote in message
> news:k9gk4s$2hh$1@newscl01ah.mathworks.com...
> > I'm trying to import some data, which was created by C code, into Matlab
> > without losing much precision. Can someone show me an example of how to do
> > so?
> >
> > I have about 52 decimal places of precision, with N=16 numbers on a line,
> > like this:
>
> Double precision only allows you 52 bits of precision (well, plus the
> implicitly stored bit) not 52 digits.

The input data is 52 decimal digits, not bits, of precision. I know what I posted was a big mess (there's no preview on this board), but there are 52 decimal digits of precision in the sample input. It's a "long double" in C. If my Matlab has only 52 (or 53) binary digits of precision, which is 16 decimal digits, then I guess I've already imported the most precise information I can.

>
> *snip*
>
> > I've tried using dlmread, csvread and importdata, but they cut off
> > precision.
>
> You could read these in as _strings_ and convert them into symbolic objects
> using Symbolic Math Toolbox.
>
> *snip*
>
> > I've tried other things, too, but I cannot get any more precision. You can
> > see that their all losing precision after about 16 decimal places.
>
> Yes. See:
>
> http://en.wikipedia.org/wiki/Double-precision_floating-point_format
>
> > Could someone show me an example of how to import my data with the most
> > precision?
>
> Why do you believe you need 52 decimal digits of precision for your data?
> For comparison, you only need roughly 39 digits of pi to calculate the
> volume of the universe to within one atom.
>
> http://en.wikipedia.org/wiki/Pi#Motivations_for_computing_.CF.80
>
> Just because you HAVE the number out to N digits doesn't mean you NEED all
> those digits.

Actually, I don't know how much precision I should preserve. My adviser has been characteristically slow to respond to my questions and I've learned that I need to keep working while waiting for his response. If I don't need that much precision, then hopefully I will have at least learned something valuable along the way.

Maybe my real question should have been "How do I determine how much precision my version of Matlab has?"

>
> --
> Steve Lord
> slord@mathworks.com
> To contact Technical Support use the Contact Us link on
> http://www.mathworks.com

Subject: How to import data without losing much precision

From: Steven_Lord

Date: 3 Dec, 2012 22:10:02

Message: 4 of 6



"Jeff " <spREMOVEHITSjeffAT@SIGNoptonline.net> wrote in message
news:k9j6fs$ih3$1@newscl01ah.mathworks.com...

*snip*

> Actually, I don't know how much precision I should preserve. My adviser
> has been characteristically slow to respond to my questions and I've
> learned that I need to keep working while waiting for his response. If I
> don't need that much precision, then hopefully I will have at least
> learned something valuable along the way.
>
> Maybe my real question should have been "How do I determine how much
> precision my version of Matlab has?"

Double arrays in MATLAB store data using standard IEEE 754 double precision.
Single arrays in MATLAB use standard IEEE 754 single precision.
Symbolic arrays from Symbolic Math Toolbox are generally arbitrary
precision, but make sure you don't limit yourself prior to the construction.
For instance:

t = 3^(1/3);
x1 = sym(t);
x2 = sym(3)^(1/3);

x1 and x2 will not be exactly the same as you can see with VPA. In the case
of x1 we took the cube root of 3 in double precision, while in the case of
x2 we converted 3 into a symbolic expression then took the cube root
symbolically.

--
Steve Lord
slord@mathworks.com
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

Subject: How to import data without losing much precision

From: Nasser M. Abbasi

Date: 4 Dec, 2012 02:39:43

Message: 5 of 6

On 12/3/2012 4:10 PM, Steven_Lord wrote:

> Symbolic arrays from Symbolic Math Toolbox are generally arbitrary
> precision, but make sure you don't limit yourself prior to the construction.
> For instance:
>
> t = 3^(1/3);
> x1 = sym(t);
> x2 = sym(3)^(1/3);
>
> x1 and x2 will not be exactly the same as you can see with VPA. In the case
> of x1 we took the cube root of 3 in double precision, while in the case of
> x2 we converted 3 into a symbolic expression then took the cube root
> symbolically.
>

Another way to see this is to do

  x3 = sym('3^(1/3)')

now x3 and x2 are the same:

------------------------
EDU>> vpa(x3)
1.4422495703074083823216383107801
  
EDU>> vpa(x2)
1.4422495703074083823216383107801
--------------------------------

using '' protects the expression in question
from premature evaluation until it gets safe to the
symbolic toolbox home

--Nasser

Subject: How to import data without losing much precision

From: Roger Stafford

Date: 4 Dec, 2012 02:44:08

Message: 6 of 6

"Jeff" wrote in message <k9j6fs$ih3$1@newscl01ah.mathworks.com>...
> Actually, I don't know how much precision I should preserve. My adviser has been characteristically slow to respond to my questions and I've learned that I need to keep working while waiting for his response. If I don't need that much precision, then hopefully I will have at least learned something valuable along the way.
>
> Maybe my real question should have been "How do I determine how much precision my version of Matlab has?"
- - - - - - - - -
  Determining the magnitude of computation errors due to rounding can be a fairly complicated matter. It is of course heavily dependent on the number of bits used to represent numbers. However it also depends in an important way on the nature of the computation that is being performed.

  As a very simple (but admittedly extreme) example of the latter, suppose you wish to compute the two roots of the quadratic equation "a*x^2+b*x+c=0" where a and c are small numbers and b is large and negative, say, (I generated these using 'rand')

 a = 5.2498740541049627e-01;
 b = -4.6332273421032481e+07;
 c = 6.5193872929175320e-02;

Matlab has no trouble with the large root using the familiar formula that we all learned in high school

 x1 = (-b+sqrt(b^2-4*a*c))/(2*a); % = 8.8254066561472103e+07

However, for the other root, using the other familiar formula

 x2 = (-b-sqrt(b^2-4*a*c))/(2*a);

matlab is faced in this case with the very small difference between two very large numbers in the numerator and loses relative accuracy - that is, the ratio of error to total size - in a serious way getting the erroneous answer

 x2 = 7.0959612746310526e-09

On the other hand if an equivalent formula is used (multiply numerator and denominator by "-b+sqrt(b^2-4*a*c)" and simplify to see this)

 x2 = 2*c/(-b+sqrt(b^2-4*a*c));

the answer is

 x2 = 1.4070941940781315e-09

which is in fact correct out to all but the 17th digit.

  The lesson to be learned here is that in all the computations you perform on your data you should definitely take into account the nature of the algorithms you use in handling that data in judging how much precision you require, and of course you should always select for numerical procedures those with the best accuracy.

Roger Stafford

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us