Re: Floating point accuracy

["M. Kotiaho" <kotiaho@m_a_i_l.com>]

Martin Dowie wrote:

> I'm trying to mimic 64-bit unsigned integers (that will represent
> nanoseconds) on a
> PowerPC (Green Hills target compiler for the PowerPC doesn't support 64 bit
> unsigned directly - unless you know differently!).
>
> I'm doing this by having a most sigificant and least significant 32bit
> unsigned in what will
> eventually be a private record and providing my own operators. However my
> routine to
> convert from microseconds to nanoseconds is puzzling me. I had always
> thought that a
> float declared a 'digits 15' would give me 15 significant digits of
> accuracy - I'm sure I'm
> only using up to 13 in these test, yet the last two results return the same
> values.
>
> clearly, I'm missing something...

Yes ... look more closely at your results ... the values you print for "ns"
differ in one digit (last non-zero digit).  The values for "ls" are also
different
(the last value has ...66..., the previous one has ...65...).  It is only for
"ms"
and "MS" that the values are the same.

You should not expect otherwise, since you are losing information when
you take the 'floor operation in computing Float_MS.  It has nothing to do
with the number of significant digits (since you have enough).

Here is the math:

'Last - 1:
Float_MS = Floor( (2**32 - 2)*1000 / 2**32 ) = Floor( 1000 - 2000/2**32 ) = 999

'Last:
Float_MS = Floor( (2**32 - 1)*1000 / 2**32 ) = Floor( 1000 - 1000/2**32 ) = 999

...<snip>...

>
> test 'Last - 1
> ns =  4.29496729400000E+12
> ms =  9.99000000000000E+02
> MS =  4.29067232870400E+12
> ls =  4.29496529600000E+09
>
> ms => 999 ls => 4294965296
> test 'Last
> ns =  4.29496729500000E+12
> ms =  9.99000000000000E+02
> MS =  4.29067232870400E+12
> ls =  4.29496629600000E+09
>
> ms => 999 ls => 4294966296

Regards,
Markku Kotiaho