* floating point number radix
@ 1985-10-19 2:39 Doug Bryan
1985-10-20 20:17 ` dik
0 siblings, 1 reply; 4+ messages in thread
From: Doug Bryan @ 1985-10-19 2:39 UTC (permalink / raw)
We have a question for all you model number gurus out there:
[3.5.7(6)]
The reference manual states that the minimum number of binary digits, B,
required after the point in the binary mantissa of a floating point
number is...
ceiling (D * ln(10)/ln(2) + 1.0)
where D is the minimal number of decimal digits in the decimal mantissa.
Consider D = 3...
ceiling (3 * ln(10)/ln(2) + 1.0) = ceiling (10.9657) = 11
But... is not 10 binary digits sufficient to represent 3 decimal digits?
2**(-10) < 10**(-3)
We think the "+ 1.0" in the above expression can be removed.
doug bryan and geoff mendal
-------
^ permalink raw reply [flat|nested] 4+ messages in thread
* floating point number radix
@ 1985-10-19 2:39 Doug Bryan
0 siblings, 0 replies; 4+ messages in thread
From: Doug Bryan @ 1985-10-19 2:39 UTC (permalink / raw)
We have a question for all you model number gurus out there:
[3.5.7(6)]
The reference manual states that the minimum number of binary digits, B,
required after the point in the binary mantissa of a floating point
number is...
ceiling (D * ln(10)/ln(2) + 1.0)
where D is the minimal number of decimal digits in the decimal mantissa.
Consider D = 3...
ceiling (3 * ln(10)/ln(2) + 1.0) = ceiling (10.9657) = 11
But... is not 10 binary digits sufficient to represent 3 decimal digits?
2**(-10) < 10**(-3)
We think the "+ 1.0" in the above expression can be removed.
doug bryan and geoff mendal
-------
^ permalink raw reply [flat|nested] 4+ messages in thread
* floating point number radix
@ 1985-10-19 17:50 "Keith F. Lynch"
0 siblings, 0 replies; 4+ messages in thread
From: "Keith F. Lynch" @ 1985-10-19 17:50 UTC (permalink / raw)
Date: Fri 18 Oct 85 19:39:57-PDT
From: Doug Bryan <BRYAN@SU-SIERRA.ARPA>
[3.5.7(6)]
The reference manual states that the minimum number of binary digits, B,
required after the point in the binary mantissa of a floating point
number is...
ceiling (D * ln(10)/ln(2) + 1.0)
where D is the minimal number of decimal digits in the decimal mantissa.
Consider D = 3...
ceiling (3 * ln(10)/ln(2) + 1.0) = ceiling (10.9657) = 11
But... is not 10 binary digits sufficient to represent 3 decimal digits?
2**(-10) < 10**(-3)
We think the "+ 1.0" in the above expression can be removed.
Consider D=2. 2**(-6) > 10**(-2). The "+ 1.0" should stay.
...Keith
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: floating point number radix
1985-10-19 2:39 Doug Bryan
@ 1985-10-20 20:17 ` dik
0 siblings, 0 replies; 4+ messages in thread
From: dik @ 1985-10-20 20:17 UTC (permalink / raw)
In article <8510190414.AA24234@UCB-VAX> BRYAN@SU-SIERRA.ARPA (Doug Bryan) writes:
>[3.5.7(6)]
>The reference manual states that the minimum number of binary digits, B,
>required after the point in the binary mantissa of a floating point
>number is...
>
> ceiling (D * ln(10)/ln(2) + 1.0)
>
>where D is the minimal number of decimal digits in the decimal mantissa.
>
>Consider D = 3...
>
> ceiling (3 * ln(10)/ln(2) + 1.0) = ceiling (10.9657) = 11
>
>But... is not 10 binary digits sufficient to represent 3 decimal digits?
> 2**(-10) < 10**(-3)
>We think the "+ 1.0" in the above expression can be removed.
>
>doug bryan and geoff mendal
>
No. The relative precision of the binary representation should not be
less than that of the decimal representation. For difits 3 the relative
precision of the decimal representation ranges from 1 in 500 to 1 in 999,
and with 10 bits for the binary representation from 1 in 512 to 1 in 1023.
And indeed we find the following:
819/8192 = .099975
818/8192 = .099853
817/8192 = .099731
So there is no representation for .0998, a valid digits 3 number.
--
dik t. winter, cwi, amsterdam, nederland
UUCP: {seismo|decvax|philabs}!mcvax!dik
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~1985-10-20 20:17 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
1985-10-19 17:50 floating point number radix "Keith F. Lynch"
-- strict thread matches above, loose matches on Subject: below --
1985-10-19 2:39 Doug Bryan
1985-10-20 20:17 ` dik
1985-10-19 2:39 Doug Bryan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox