Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Rounding and sprintf

Subject: Rounding and sprintf

From: Maxx Chatsko

Date: 26 Jul, 2010 19:44:07

Message: 1 of 3

Hello all,
   Say I needed to get the inverses of 400 and 405. Matlab computes both of them as 0.0025, when in fact the inverse of 405 is 0.0024691. This makes a HUGE difference when converting between wavelength and wavenumbers in Raman shifts. Any help?
     str2num(sprintf('%5.5d',inv(WL(2)))) %WL(2) = 405

    ans =

     0.0025 %need to get to at least 7 decimal places

I've tried different indicators (%f, %d,...) no dice.
     Thanks
     Maxx

Subject: Rounding and sprintf

From: Walter Roberson

Date: 26 Jul, 2010 19:51:53

Message: 2 of 3

Maxx Chatsko wrote:
> Hello all,
> Say I needed to get the inverses of 400 and 405. Matlab computes both
> of them as 0.0025, when in fact the inverse of 405 is 0.0024691. This
> makes a HUGE difference when converting between wavelength and
> wavenumbers in Raman shifts. Any help?
> str2num(sprintf('%5.5d',inv(WL(2)))) %WL(2) = 405
>
> ans =
>
> 0.0025 %need to get to at least 7 decimal places
>
> I've tried different indicators (%f, %d,...) no dice.

 >> str2num(sprintf('%.7f',inv(405)))
ans =
                  0.0024691


However...

 >> sprintf('%.60g',str2num(sprintf('%.7f',inv(405))))
ans =
0.00246910000000000011410872247097358922474086284637451171875

from which you should conclude that it does not work to round a value by
sprintf'ing it and converting the resulting string to a number. There is no
way in binary floating point arithmetic to store 0.0024691 _exactly_.

Subject: Rounding and sprintf

From: Maxx Chatsko

Date: 26 Jul, 2010 20:43:05

Message: 3 of 3

Walter Roberson <roberson@hushmail.com>

    Thank you

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us