understanding floating point types

understanding floating point types

[Ada novice]

Hi,
    I'm trying to understand the floating point representation in the
binary and decimal bases. If D is the number of decimal digits in the
mantissa, then the number if bits B is given by B = D(1og 10/1og 2) +
1 = 3.32 * D + 1 and this rounded up to the next integer value.

Thus for a float type in Ada, mantissa D = 6, then B = 3.32(6) +1 = 21
to the nearest upwards integer value. GNAT GPL 2010 shows that
Float'Mantissa is 21 bits.

Now the TOTAL number of bits in a float (Float'Size in GNAT GPL) is B
= 32 giving the range of exponent as -4B to 4B i.e. -128 to 128. And
this gives in decimal base a exponent (in Float'Last) of log(2^128)/
(log 10) = 38.53, which is 38 in GNAT GPL.

Now if I take the total number of bits for a float i.e. 32 and
subtract the number of bits for the mantissa (21), I get 11.

If I set 11 = 3.32(decimal equivalent) + 1, I get the unknown "decimal
equivalent" as 3 digits.

If I understand correctly, then 2 out of the 3 digits are for the
exponent. And what about the remaining 1 digit? Is it for a sign
before the mantissa or before the exponent?

I guess that I'm confusing between decimal and binary representations.
Some help would be highly appreciated :).

Thanks.

YC

Re: understanding floating point types

[Dmitry A. Kazakov]

On Sun, 22 Aug 2010 02:11:59 -0700 (PDT), Ada novice wrote:

>     I'm trying to understand the floating point representation in the
> binary and decimal bases. If D is the number of decimal digits in the
> mantissa, then the number if bits B is given by B = D(1og 10/1og 2) +
> 1 = 3.32 * D + 1 and this rounded up to the next integer value.
> 
> Thus for a float type in Ada, mantissa D = 6, then B = 3.32(6) +1 = 21
> to the nearest upwards integer value. GNAT GPL 2010 shows that
> Float'Mantissa is 21 bits.
>
> Now the TOTAL number of bits in a float (Float'Size in GNAT GPL) is B
> = 32 giving the range of exponent as -4B to 4B i.e. -128 to 128. And
> this gives in decimal base a exponent (in Float'Last) of log(2^128)/
> (log 10) = 38.53, which is 38 in GNAT GPL.

Float'Machine_Mantissa = 24 =
23 explicit bits + 1 hidden bit (considering IEEE 754 single precision)

Float'Mantissa gives the model binary mantissa, the least number of bits
required to represent specified decimal mantissa (the "number B"). It is a
model attribute, a view of the real machine's number of bits, which is 24.
[*]

Compare:

   type My_Float is digits 1;

My_Float'Machine_Mantissa = 24 (32-bit IEEE 754 is used to model it)
My_Float'Mantissa = 5 (= 4+1)

   type My_Float is digits 2;

My_Float'Machine_Mantissa = 24 (32-bit IEEE 754 is used to model it)
My_Float'Mantissa = 8 (= 7+1)

etc

-----------
* Excluding denormalized numbers

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: understanding floating point types

[Ada novice]

On Aug 22, 11:51 am, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:
> Compare:
>
>    type My_Float is digits 1;
>
> My_Float'Machine_Mantissa = 24 (32-bit IEEE 754 is used to model it)
> My_Float'Mantissa = 5 (= 4+1)
>
>    type My_Float is digits 2;
>
> My_Float'Machine_Mantissa = 24 (32-bit IEEE 754 is used to model it)
> My_Float'Mantissa = 8 (= 7+1)

Thanks. The Float'Machine_Mantissa remains constant in your examples
with 1 and 2 precision digits. Is the Float'Machine_Mantissa a
predefined value that is constant for any precision digit requested?

I have a second query: Say we consider the type My_Float is digits 2.
This means that between the model numbers, the resolution is 1/(2^8)
with the 8 in the expression obtained as 4 X 2. Am I right? Now if we
have to multiply two numbers: say a = 2.33 and b = 3.45. Does Ada
first convert each operation to its nearest model number and then do
the multiplication and represent the result to its nearest model
number? Or does Ada do the multiplication first and then represent the
result to to its nearest model number?


Thanks
YC

Re: understanding floating point types

[Ada novice]

On Aug 22, 12:37 pm, Ada novice <ycalleecha...@gmx.com> wrote:
> Does Ada
> first convert each operation to its nearest model number and then do
> the multiplication and represent the result to its nearest model
> number? Or does Ada do the multiplication first and then represent the
> result to to its nearest model number?
>
> Thanks
> YC


A slip of mine: first convert each operation ...change to first
convert each operand.

YC

Re: understanding floating point types

[Dmitry A. Kazakov]

On Sun, 22 Aug 2010 03:37:35 -0700 (PDT), Ada novice wrote:

> On Aug 22, 11:51�am, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:
>> Compare:
>>
>> � �type My_Float is digits 1;
>>
>> My_Float'Machine_Mantissa = 24 (32-bit IEEE 754 is used to model it)
>> My_Float'Mantissa = 5 (= 4+1)
>>
>> � �type My_Float is digits 2;
>>
>> My_Float'Machine_Mantissa = 24 (32-bit IEEE 754 is used to model it)
>> My_Float'Mantissa = 8 (= 7+1)
> 
> Thanks. The Float'Machine_Mantissa remains constant in your examples
> with 1 and 2 precision digits. Is the Float'Machine_Mantissa a
> predefined value that is constant for any precision digit requested?

No, it is the is best fitting machine number available (compiler's choice):

 � type My_Float is digits 7;

   My_Float'Machine_Mantissa = 53 (64-bit IEEE 754 is used to model it)
   My_Float'Mantissa = 25

> I have a second query: Say we consider the type My_Float is digits 2.
> This means that between the model numbers, the resolution is 1/(2^8)
> with the 8 in the expression obtained as 4 X 2. Am I right? Now if we
> have to multiply two numbers: say a = 2.33 and b = 3.45. Does Ada
> first convert each operation to its nearest model number and then do
> the multiplication and represent the result to its nearest model
> number? Or does Ada do the multiplication first and then represent the
> result to to its nearest model number?

I would expect that all computations performed in machine numbers, not
because it is mandated, I believe it is not, but because it is simpler to
implement. At least with GNAT it is so. Try this:

   type My_Float is digits 2;
   X : My_Float := 1.0 / 3.0;
   ...
   Put_Line (Float'Image (Float (X)));

The output will be: 3.33333E-01

If you can pick any number from the interval [0.33, 0.34[ to represent 1/3
in with 2 decimal digits. Why not 0.333333?

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: understanding floating point types

[Ada novice]

On Aug 22, 3:57 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:
>
>    My_Float'Machine_Mantissa = 53 (64-bit IEEE 754 is used to model it)
>    My_Float'Mantissa = 25
>

Yes. This is good. For Long_Float, Long_Float'Mantissa is 51.


>
> I would expect that all computations performed in machine numbers, not
> because it is mandated, I believe it is not, but because it is simpler to
> implement. At least with GNAT it is so. Try this:
>
>    type My_Float is digits 2;
>    X : My_Float := 1.0 / 3.0;
>    ...
>    Put_Line (Float'Image (Float (X)));
>
> The output will be: 3.33333E-01
>
> If you can pick any number from the interval [0.33, 0.34[ to represent 1/3
> in with 2 decimal digits. Why not 0.333333?
>

I didn't quite get this part. In my previous example with say a = 2.33
and b = 3.45. Does Multiplication of a with b mean converting each of
a and b in model number first and then do the multiplication and
afterwards approximating the result to the nearest model number? Or
the multiplication will just be performed with the two numbers as 2.33
X 3.45 = 8.0385 and then approximating this result to the nearest
model number?

Thanks.

YC

Re: understanding floating point types

[Yannick Duchêne (Hibou57)]

Le Sun, 22 Aug 2010 15:57:42 +0200, Dmitry A. Kazakov  
<mailbox@dmitry-kazakov.de> a écrit:
>> Thanks. The Float'Machine_Mantissa remains constant in your examples
>> with 1 and 2 precision digits. Is the Float'Machine_Mantissa a
>> predefined value that is constant for any precision digit requested?
>
> No, it is the is best fitting machine number available (compiler's  
> choice):
Yes, because it would be too much expensive to create a custom float  
representation to save a few bits. It would even requires to redefine all  
operations (a kind of FPU emulation, like the one Pascal did for old  
i386/i486).

Re: understanding floating point types

[Dmitry A. Kazakov]

On Sun, 22 Aug 2010 10:15:24 -0700 (PDT), Ada novice wrote:

> I didn't quite get this part. In my previous example with say a = 2.33
> and b = 3.45. Does Multiplication of a with b mean converting each of
> a and b in model number first and then do the multiplication and
> afterwards approximating the result to the nearest model number? Or
> the multiplication will just be performed with the two numbers as 2.33
> X 3.45 = 8.0385 and then approximating this result to the nearest
> model number?

No, the result is 8.0385 (assuming 32-bit machine numbers). Try this:

with Ada.Text_IO; use Ada.Text_IO;
procedure Test_Float is
   type My_Float is digits 3;
   X : My_Float := 2.33;
   Y : My_Float := 3.45;
begin
   Put_Line ("Model view:" & My_Float'Image (X*Y));
   Put_Line ("The reality:" & Float'Image (Float (X*Y)));
end Test_Float;

It should print:

Model view: 8.04E+00
The reality: 8.03850E+00

When you print the number using My_Float'Image that rounds the machine
number to the nearest model number. In computations and conversions
machine numbers are left untouched.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: understanding floating point types

[Ada novice]

On Aug 22, 7:22 pm, Yannick Duchêne (Hibou57)
<yannick_duch...@yahoo.fr> wrote:
> Le Sun, 22 Aug 2010 15:57:42 +0200, Dmitry A. Kazakov  
> <mail...@dmitry-kazakov.de> a écrit:>> Thanks. The Float'Machine_Mantissa remains constant in your examples
> >> with 1 and 2 precision digits. Is the Float'Machine_Mantissa a
> >> predefined value that is constant for any precision digit requested?
>
> > No, it is the is best fitting machine number available (compiler's  
> > choice):
>
> Yes, because it would be too much expensive to create a custom float  
> representation to save a few bits. It would even requires to redefine all  
> operations (a kind of FPU emulation, like the one Pascal did for old  
> i386/i486).

Yes. Thanks for this information.

YC

Re: understanding floating point types

[Ada novice]

On Aug 22, 8:16 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:
> On Sun, 22 Aug 2010 10:15:24 -0700 (PDT), Ada novice wrote:
> > I didn't quite get this part. In my previous example with say a = 2.33
> > and b = 3.45. Does Multiplication of a with b mean converting each of
> > a and b in model number first and then do the multiplication and
> > afterwards approximating the result to the nearest model number? Or
> > the multiplication will just be performed with the two numbers as 2.33
> > X 3.45 = 8.0385 and then approximating this result to the nearest
> > model number?
>
> No, the result is 8.0385 (assuming 32-bit machine numbers). Try this:
>
> with Ada.Text_IO; use Ada.Text_IO;
> procedure Test_Float is
>    type My_Float is digits 3;
>    X : My_Float := 2.33;
>    Y : My_Float := 3.45;
> begin
>    Put_Line ("Model view:" & My_Float'Image (X*Y));
>    Put_Line ("The reality:" & Float'Image (Float (X*Y)));
> end Test_Float;
>
> It should print:
>
> Model view: 8.04E+00
> The reality: 8.03850E+00
>
> When you print the number using My_Float'Image that rounds the machine
> number to the nearest model number. In computations and conversions
> machine numbers are left untouched.
>
> --
> Regards,
> Dmitry A. Kazakovhttp://www.dmitry-kazakov.de


Thanks for all this information. I have 2 small queries:

1. You used: type My_Float is digits 3; I tend to confuse the fact
that digits 3 means 3 digits of precision after the decimal point? So
when I consider 2.33 or 3.45, then both numbers are with digits 2 in
my mind. Or does digits 3 mean 3 significant digits which will include
the leading 2 in 2.33 and the leading 3 in 3.45 as well?

2. In your example, you have:

Model view: 8.04E+00
The reality: 8.03850E+00

If I understand correctly, model view represents the number that is
exactly representable by the computer. In our case here, we can get
the reality number since Float has higher precision digits than
My_Float. However if My_float used the same precision digits as Float
(i.e. 6) then this would also represent the number that could be
represented by the computer. If both a and b had a hypothetical 30
digits of precision, then neither Float (6 digits) nor Long_Float (15
digits), nor Long_Long_Float (18 digits) would give the right value
that would represent the model number of the multiplication result
a*b. I mentioned the word hypothetical, as we would not able to use a
number with 30 digits of precision anyway on a computer. But my point
was to say that Model view doesn't have to have less precision digits
as the reality number. But in our example, since My_Float has fewer
digits than Float, in this specific case we can form the reality
number with more precision digits than the model number.


Thanks,
YC

Re: understanding floating point types

[Dmitry A. Kazakov]

On Sun, 22 Aug 2010 12:05:33 -0700 (PDT), Ada novice wrote:

> 1. You used: type My_Float is digits 3; I tend to confuse the fact
> that digits 3 means 3 digits of precision after the decimal point? So
> when I consider 2.33 or 3.45, then both numbers are with digits 2 in
> my mind. Or does digits 3 mean 3 significant digits which will include
> the leading 2 in 2.33 and the leading 3 in 3.45 as well?

Yes, it is relative, in the case of floating-point numbers. Ada has another
model of real numbers, the fixed point numbers, they have an absolute
precision.

> 2. In your example, you have:
> 
> Model view: 8.04E+00
> The reality: 8.03850E+00
> 
> If I understand correctly, model view represents the number that is
> exactly representable by the computer.

Yes, the model number is your requirement. The machine number is an
implementation that fulfills your requirement.

> In our case here, we can get
> the reality number since Float has higher precision digits than
> My_Float. However if My_float used the same precision digits as Float
> (i.e. 6) then this would also represent the number that could be
> represented by the computer. If both a and b had a hypothetical 30
> digits of precision, then neither Float (6 digits) nor Long_Float (15
> digits), nor Long_Long_Float (18 digits) would give the right value
> that would represent the model number of the multiplication result
> a*b.

Neither they would a or b. 30 digits precision require 101 bits of machine
mantissa.

> I mentioned the word hypothetical, as we would not able to use a
> number with 30 digits of precision anyway on a computer. But my point
> was to say that Model view doesn't have to have less precision digits
> as the reality number.

The machine number precision must be equal or greater than the model one.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: understanding floating point types

[Ada novice]

On Aug 22, 9:34 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:

> The machine number precision must be equal or greater than the model one.
>
Thanks for the explanation. If a model number is a number which can be
accurately representable by a computer, then I don't understand the
term machine number. On my computer I can have Long_Long_Float (18
digits). In this case is the model number precision not equal to the
machine number precision? Do you mean that machine number precision is
set say to the precision of Long_Long_Float (18 digits)? This would
make sense since any other float calculation (with fewer precision
digits) would have a model number precision less than the machine
number precision (Long_Long_Float---18 digits).

YC

Re: understanding floating point types

[J-P. Rosen]

Le 23/08/2010 08:29, Ada novice a �crit :
> On Aug 22, 9:34 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:
> 
>> The machine number precision must be equal or greater than the model one.
>>
> Thanks for the explanation. If a model number is a number which can be
> accurately representable by a computer, then I don't understand the
> term machine number. 
A model number is a number that must be represented exactly on any
implementation. A machine number is a number that is represented exactly
on a given implementation.

Of course, the machine numbers are a superset of the model numbers =>
you can get more accuracy than required, not less.


-- 
---------------------------------------------------------
           J-P. Rosen (rosen@adalog.fr)
Visit Adalog's web site at http://www.adalog.fr

Re: understanding floating point types

[Dmitry A. Kazakov]

On Sun, 22 Aug 2010 23:29:54 -0700 (PDT), Ada novice wrote:

> On Aug 22, 9:34�pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:
> 
> Thanks for the explanation. If a model number is a number which can be
> accurately representable by a computer, then I don't understand the
> term machine number.

The machine number is used by the compiler to implement the model number.
The properties of the machine numbers depend on the hardware and the
compiler, for example it may have binary mantissa and hexadecimal exponent.
It may have IEEE 754 non-numeric elements like NaN. It can be an analogue
computer with a DAC on the ALU's input and ADC on the output. It can be an
oracle who gives answers. The machine number is a floating-point model of
that thing.

> On my computer I can have Long_Long_Float (18
> digits). In this case is the model number precision not equal to the
> machine number precision?

The model number of 18 decimal digits precision requires at least 61 binary
bits of mantissa. That is S'Mantissa. This is independent on any hardware.

The machine number of Intel FPU has 64 bits. The attribute
S'Machine_Mantissa is this implementation-dependent number of bits.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: understanding floating point types

[Martin]

On Aug 22, 8:05 pm, Ada novice <ycalleecha...@gmx.com> wrote:
[snip]
> 1. You used: type My_Float is digits 3; I tend to confuse the fact
> that digits 3 means 3 digits of precision after the decimal point?

Not quite - it's "3 significant (decimal) digits", e.g. 7_654.32 with
a float defined as 'digits 3', the "765" are 'guaranteed' what follows
those isn't (although in practice, any cpu I'm likely to be using,
will get more than the first 3 digits accurate).

-- Martin

Re: understanding floating point types

[Ada novice]

Thanks to all of you for the nice explanations.

YC

Re: understanding floating point types

[Ada novice]

I have just read the following from an Ada book by Fintan Culwin:
----
type VeryAccurate is digits 16
type NotAccurate is digits 2

Values of the type VeryAccurate would have limited range with a high
degree of precision. Values of the type NotAccurate would have a much
greater range with only two digits of precision.
----

My understanding is that using "B = 3.32*D + 1", a type with digits 16
(D = 16) will have a higher B value and hence a higher exponent range
-4*B to 4*B than a type with digits 2. Will a higher exponent range
not give a higher range?

What am I understanding wrong here?

Thanks
YC

Re: understanding floating point types

[Jacob Sparre Andersen]

Ada novice <ycalleecharan@gmx.com> writes:

> I have just read the following from an Ada book by Fintan Culwin:
> ----
> type VeryAccurate is digits 16
> type NotAccurate is digits 2
>
> Values of the type VeryAccurate would have limited range with a high
> degree of precision. Values of the type NotAccurate would have a much
> greater range with only two digits of precision.
> ----
>
> My understanding is that using "B = 3.32*D + 1", a type with digits 16
> (D = 16) will have a higher B value and hence a higher exponent range
> -4*B to 4*B than a type with digits 2. Will a higher exponent range
> not give a higher range?
>
> What am I understanding wrong here?

You're using a random equation.

Fintan Culwin is probably assuming that you allocate a fixed number of
bits to represent a floating point number.  The fewer of those bits,
which are allocated to precision, the more can be allocated to the
exponent (and thus the range).

But in practice your compiler may choose exactly the same representation
for both of the types in the example above.

Greetings,

Jacob
-- 
"The same equations have the same solutions."

Re: understanding floating point types

[Ada novice]

On Aug 24, 4:05 pm, Jacob Sparre Andersen <spa...@nbi.dk> wrote:
> Ada novice <ycalleecha...@gmx.com> writes:
> > I have just read the following from an Ada book by Fintan Culwin:
> > ----
> > type VeryAccurate is digits 16
> > type NotAccurate is digits 2
>
> > Values of the type VeryAccurate would have limited range with a high
> > degree of precision. Values of the type NotAccurate would have a much
> > greater range with only two digits of precision.
> > ----
>
> > My understanding is that using "B = 3.32*D + 1", a type with digits 16
> > (D = 16) will have a higher B value and hence a higher exponent range
> > -4*B to 4*B than a type with digits 2. Will a higher exponent range
> > not give a higher range?
>
> > What am I understanding wrong here?
>
> You're using a random equation.
>
> Fintan Culwin is probably assuming that you allocate a fixed number of
> bits to represent a floating point number.  The fewer of those bits,
> which are allocated to precision, the more can be allocated to the
> exponent (and thus the range).
>
> But in practice your compiler may choose exactly the same representation
> for both of the types in the example above.
>
> Greetings,
>
> Jacob
> --
> "The same equations have the same solutions."

I was thinking the same regarding having "limited" number of bits
available but wasn't sure. Thanks for the confirmation. Maybe what he
is saying applies more to the embedded programming way of thinking.

YC