* Re: Float & precision [not found] ` <7uc86m$cob$1@holly.prod.itd.earthlink.net> @ 1999-10-17 0:00 ` fluffy_pop 1999-10-17 0:00 ` David C. Hoos, Sr. 1999-10-19 0:00 ` Robert Dewar 0 siblings, 2 replies; 3+ messages in thread From: fluffy_pop @ 1999-10-17 0:00 UTC (permalink / raw) On Sun, 17 Oct 1999 05:23:55 -0500, "David C. Hoos, Sr." <david.c.hoos.sr@ada95.com> wrote: Thank you very much for spending the time to help. I was vaguely aware of what you are talking about. >There is a way to manipulate the bits of the mantissa apart from the >exponent, but this will not solve the problem. > >The problem is that a decimal value is in general not exactly >representable in binary in finite length. > ... >Thus, the nearest two values to 1.3, expressed in IEEE 32-bit floating >point (gnat Float on a PC), displayed here in decimal to 18 significant >digits of precision are: > 1.29999995231628418E+00 > 1.30000090599060059E+00 Why could I not adjust my result after each sigle operation, intermediate steps. Someone suggested to multiply the intermediate result by a certain power of ten (0.0001 for the particular example I had given him) and to typecast it into an integer (long integer would be much better), and then to go back (do the two operations in reverse). Of course this will not work as simply as that since my intermediate result could be of any order of magnitude with positive or negative exponent. So I may have to multiply it by 10^x or divide it by 10^x, and then I'd have to figure out what x should be based on the intermediate result, so as to bring the value within the interger or long integer's range. Will a typecast into a discrete (integer) value not eliminate those non significant digits, thus rounding to a specific position ? I'm just sort of playing, going beyond the scope of my school work, trying to think of a relatively simple way to do it. After all my pocket calculator does it without problems. It's an old cheap Casio model. I don't know if operates with 32 or 64 bits or maybe something else. Thanks again. Marc Galipeau -- What I really am is "fluffy", no "_dong", no "_puff", no "_woo", no nothing, just plain fluffy. ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Float & precision 1999-10-17 0:00 ` Float & precision fluffy_pop @ 1999-10-17 0:00 ` David C. Hoos, Sr. 1999-10-19 0:00 ` Robert Dewar 1 sibling, 0 replies; 3+ messages in thread From: David C. Hoos, Sr. @ 1999-10-17 0:00 UTC (permalink / raw) <fluffy_pop@dsuper.net> wrote in message news:380b47e6.444543814@news.dsuper.net... > On Sun, 17 Oct 1999 05:23:55 -0500, "David C. Hoos, Sr." > <david.c.hoos.sr@ada95.com> wrote: > ><snip> > Of course this will not work as simply as that since my intermediate > result could be of any order of magnitude with positive or negative > exponent. So I may have to multiply it by 10^x or divide it by 10^x, > and then I'd have to figure out what x should be based on the > intermediate result, so as to bring the value within the interger or > long integer's range. > > Will a typecast into a discrete (integer) value not eliminate those > non significant digits, thus rounding to a specific position ? > No. Not unless your integer has more bits than the mantissa of the floating point type, and then, only if a scale factor is chosen such that there is no overflow. By the way, "typecast" is not an Ada term. You probably mean "type conversion" or "type convert." > I'm just sort of playing, going beyond the scope of my school work, > trying to think of a relatively simple way to do it. After all my > pocket calculator does it without problems. It's an old cheap Casio > model. I don't know if operates with 32 or 64 bits or maybe something > else. The reason your pocket calculator _appears_ to do it without problems, is that it does the calculations internally with a couple more digits than it displays. You can get the same effect in Ada with the gnat compiler by doing your computations in long_long_float, then converting to long_float for display. ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Float & precision 1999-10-17 0:00 ` Float & precision fluffy_pop 1999-10-17 0:00 ` David C. Hoos, Sr. @ 1999-10-19 0:00 ` Robert Dewar 1 sibling, 0 replies; 3+ messages in thread From: Robert Dewar @ 1999-10-19 0:00 UTC (permalink / raw) In article <380b47e6.444543814@news.dsuper.net>, fluffy_pop@dsuper.net wrote: > After all my > pocket calculator does it without problems. It's an old cheap > Casio model. I don't know if operates with 32 or 64 bits or > maybe something else. It probably operates in decimal, so if you are in the decimal world, it will work without apparent oddities coming from binary to decimal translation, but it is of course infeasible to use decimal on high speed modern computers Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~1999-10-19 0:00 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <3807ba57.277208181@news.dsuper.net> [not found] ` <slrn80goik.62n.gisle@apal.ii.uib.no> [not found] ` <38088ba5.330797551@news.dsuper.net> [not found] ` <7uc86m$cob$1@holly.prod.itd.earthlink.net> 1999-10-17 0:00 ` Float & precision fluffy_pop 1999-10-17 0:00 ` David C. Hoos, Sr. 1999-10-19 0:00 ` Robert Dewar
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox