Floating-Point Numbers and Internal Representation

Floating-Point Numbers and Internal Representation

[Matthias Kretschmer]

Hello,

I had a problem in one of my programs, that was caused by the internal
representation of floating-point numbers in the fpu. Concrete:
calculating the value of an optimum for some large number of objects,
then in a second doing something with all optimal objects. The problem
was, that when doing the calculation the second time, the compiler left
the floating-point number in the fpu which had a higher precision than
the representation I choosed, so comparing for equality returns always
"False". The problem would be solved by some operation truncating the
floating-point number to the precision I orginally wanted or used. I
could of course put all the values in an array or list and then finding
optimum and optimal objects, but I don't want to go this way. In C iirc
I could use a volatile variable to ensure the compiler will put the
value in and read from the variable before comparing, but to achieve
something similiar (truncating the precision to that of the type used)
in Ada?

My current solution is to enhance precision to compiler maximum which
seems to be the machine maximum. But I would like to know if there are
any better solutions?

-- 
Matthias Kretschmer

Re: Floating-Point Numbers and Internal Representation

[Martin Krischik]

Matthias Kretschmer wrote:

> Hello,
> 
> I had a problem in one of my programs, that was caused by the internal
> representation of floating-point numbers in the fpu. Concrete:
> calculating the value of an optimum for some large number of objects,
> then in a second doing something with all optimal objects. The problem
> was, that when doing the calculation the second time, the compiler left
> the floating-point number in the fpu which had a higher precision than
> the representation I choosed, so comparing for equality returns always
> "False". The problem would be solved by some operation truncating the
> floating-point number to the precision I orginally wanted or used. I
> could of course put all the values in an array or list and then finding
> optimum and optimal objects, but I don't want to go this way. In C iirc
> I could use a volatile variable to ensure the compiler will put the
> value in and read from the variable before comparing, but to achieve
> something similiar (truncating the precision to that of the type used)
> in Ada?

Yes it is called "pragma Volatile".

> My current solution is to enhance precision to compiler maximum which
> seems to be the machine maximum. But I would like to know if there are
> any better solutions?

Comparting Floatingpoing values in equality is an allways tricky. And I
could bet I answered this question before.

Independent of CPU/Language used I would allways suggest to use

abs (X - Y) < epsilon

with a sufficient but not to small epsilon instead.

Martin

-- 
mailto://krischik@users.sourceforge.net
Ada programming at: http://ada.krischik.com

Re: Floating-Point Numbers and Internal Representation

[Martin Dowie]

Martin Krischik wrote:
> Independent of CPU/Language used I would allways suggest to use
> 
> abs (X - Y) < epsilon
> 
> with a sufficient but not to small epsilon instead.

Suggested reading:

http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm

There is more than one answer to this problem...

Cheers

-- Martin

Re: Floating-Point Numbers and Internal Representation

[Dmitry A. Kazakov]

On Sun, 4 Dec 2005 14:11:49 +0000 (UTC), Martin Dowie wrote:

> Martin Krischik wrote:
>> Independent of CPU/Language used I would allways suggest to use
>> 
>> abs (X - Y) < epsilon
>> 
>> with a sufficient but not to small epsilon instead.
> 
> Suggested reading:
> 
> http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm
>
> There is more than one answer to this problem...

Epsilon above can well be relative that depends solely on the algorithm.
Usually epsilon is estimated in the course of calculations together with X
and Y, as a part of the algorithm. So the answer is still same. (:-))

BTW, I wouldn't use division to evaluate relative errors as the paper
suggest. Rather:

Half_Epsilon * (abs X + abs Y)

A really different answer would be interval arithmetic. If X and Y were
intervals they would carry the accuracy estimation with them. So one could
directly compare them:

case X < Y or X > Y is -- The result is not Boolean!
   when False | Uncertain =>
       -- The difference cannot be distinguished from accumulated
       -- inaccuracy
   when True =>
       -- They are sufficiently different
end case;

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Floating-Point Numbers and Internal Representation

[Matthias Kretschmer]

On 2005-12-04, Dmitry A. Kazakov <mailbox@dmitry-kazakov.de> wrote:
> On Sun, 4 Dec 2005 14:11:49 +0000 (UTC), Martin Dowie wrote:
>
>> Martin Krischik wrote:
>>> Independent of CPU/Language used I would allways suggest to use
>>> 
>>> abs (X - Y) < epsilon
>>> 
>>> with a sufficient but not to small epsilon instead.
>> 
>> Suggested reading:
>> 
>> http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm
>>
>> There is more than one answer to this problem...

as I hope I said in my original posting, I am very well aware of this,
but I had this special case where I wanted to use explicitly equality.
There are times, when one is sure, that even with rounding errors, the
two compared floating point numbers are really equal or not if one is
using some good enough precision. So from some defined point on all
following digits can be considered garbage, as it was in my case. The
best solution would be that I had used Q (or a fixed subset of it)
instead an approximation of R for my calculations, but I was too lazy :)
Maybe I should take the time and port it to such a solution or implement
something like that myself. At least in many circumstances using exact
numbers instead of unexact ones causes much less trouble :)

-- 
Matthias Kretschmer

Re: Floating-Point Numbers and Internal Representation

[Dmitry A. Kazakov]

On Sun, 4 Dec 2005 16:15:31 +0100, Matthias Kretschmer wrote:

> as I hope I said in my original posting, I am very well aware of this,
> but I had this special case where I wanted to use explicitly equality.
> There are times, when one is sure, that even with rounding errors, the
> two compared floating point numbers are really equal or not if one is
> using some good enough precision. So from some defined point on all
> following digits can be considered garbage, as it was in my case.

This is exactly definition of an interval: [xx.xxx0(0), xx.xxxx9(9)]

> The
> best solution would be that I had used Q (or a fixed subset of it)
> instead an approximation of R for my calculations, but I was too lazy :)

No, this contradicts to your statement about "following digits" being
garbage. In the case of Q they are not.

> Maybe I should take the time and port it to such a solution or implement
> something like that myself. At least in many circumstances using exact
> numbers instead of unexact ones causes much less trouble :)

Ah but this is a completely different story. If you *really* know that
multiplication and division are exact then Ada's fixed point types is just
what you need. Fixed point numbers are perfectly comparable.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Floating-Point Numbers and Internal Representation

[Matthias Kretschmer]

On 2005-12-04, Dmitry A. Kazakov <mailbox@dmitry-kazakov.de> wrote:
> On Sun, 4 Dec 2005 16:15:31 +0100, Matthias Kretschmer wrote:
>
>> as I hope I said in my original posting, I am very well aware of this,
>> but I had this special case where I wanted to use explicitly equality.
>> There are times, when one is sure, that even with rounding errors, the
>> two compared floating point numbers are really equal or not if one is
>> using some good enough precision. So from some defined point on all
>> following digits can be considered garbage, as it was in my case.
>
> This is exactly definition of an interval: [xx.xxx0(0), xx.xxxx9(9)]
>
>> The
>> best solution would be that I had used Q (or a fixed subset of it)
>> instead an approximation of R for my calculations, but I was too lazy :)
>
> No, this contradicts to your statement about "following digits" being
> garbage. In the case of Q they are not.

ah well - as long as you know you have no periodical numbers, it doesn't
matter.

>
>> Maybe I should take the time and port it to such a solution or implement
>> something like that myself. At least in many circumstances using exact
>> numbers instead of unexact ones causes much less trouble :)
>
> Ah but this is a completely different story. If you *really* know that
> multiplication and division are exact then Ada's fixed point types is just
> what you need. Fixed point numbers are perfectly comparable.
>

hmm you're right. Forgot about this. Thanks for the hint. Didn't do
stuff in Ada for a long time, I think I forgot most of the stuff.

-- 
Matthias Kretschmer

Re: Floating-Point Numbers and Internal Representation

[Steve]

"Martin Dowie" <martin.dowie@btopenworld.com> wrote in message 
news:dmutf4$gpt$1@nwrdmz03.dmz.ncs.ea.ibs-infra.bt.com...
> Martin Krischik wrote:
>> Independent of CPU/Language used I would allways suggest to use
>>
>> abs (X - Y) < epsilon
>>
>> with a sufficient but not to small epsilon instead.
>
> Suggested reading:
>
> http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm
>
> There is more than one answer to this problem...
>
> Cheers
>
> -- Martin

Interestingly enough I ran across this site a couple of months ago, and am 
using the AlmostEquals function in some C++ code.

For Ada, I would think you could make use of the 'Adjacent attribute to 
achieve a similar result, but would be independent of the floating point 
representation.

Steve
(The Duck)

Re: Floating-Point Numbers and Internal Representation

[Randy Brukardt]

> Matthias Kretschmer wrote:
>
> > Hello,
> >
> > I had a problem in one of my programs, that was caused by the internal
> > representation of floating-point numbers in the fpu. Concrete:
> > calculating the value of an optimum for some large number of objects,
> > then in a second doing something with all optimal objects. The problem
> > was, that when doing the calculation the second time, the compiler left
> > the floating-point number in the fpu which had a higher precision than
> > the representation I choosed, so comparing for equality returns always
> > "False". The problem would be solved by some operation truncating the
> > floating-point number to the precision I orginally wanted or used. I
> > could of course put all the values in an array or list and then finding
> > optimum and optimal objects, but I don't want to go this way. In C iirc
> > I could use a volatile variable to ensure the compiler will put the
> > value in and read from the variable before comparing, but to achieve
> > something similiar (truncating the precision to that of the type used)
> > in Ada?

If you're only interested in using the memory precision, you can use the
Machine attribute, see A.5.3(60-62).
http://www.adaic.com/standards/95lrm/html/RM-A-5-3.html

But, as others have said, that may not be the best solution, as direct
comparison of float values for equality is often dubious. It's also
relatively expensive on some machines (such as the Intel Pentium
processors), where values in registers are always kept in extended
precision; dropping that precision usually requires writing the values to
memory and back. So it's best to avoid this attribute in performance
critical code portions.

                                Randy.

Re: Floating-Point Numbers and Internal Representation

[Dmitry A. Kazakov]

On Sun, 4 Dec 2005 11:33:32 +0100, Matthias Kretschmer wrote:

> I had a problem in one of my programs, that was caused by the internal
> representation of floating-point numbers in the fpu. Concrete:
> calculating the value of an optimum for some large number of objects,
> then in a second doing something with all optimal objects. The problem
> was, that when doing the calculation the second time, the compiler left
> the floating-point number in the fpu which had a higher precision than
> the representation I choosed, so comparing for equality returns always
> "False".

Why would you like to compare FP values for equality? It should be

abs (X - Y) < Eps

where Eps is known from the algorithm you are using.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Floating-Point Numbers and Internal Representation

[Gautier Write-only]

As a complement to other answers about comparing floating
point numbers, here is a page explaining the use of appropriate
epsilons in Ada according to the circumstances:

http://www.adaic.com/docs/95style/html/sec_7/7-2-7.html

HTH
______________________________________________________________ 
Gautier     --     http://www.mysunrise.ch/users/gdm/index.htm 
Ada programming -- http://www.mysunrise.ch/users/gdm/gsoft.htm 

NB: For a direct answer, e-mail address on the Web site!