From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.4 X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: 103376,7a2d45f282a1da1c X-Google-Attributes: gid103376,public X-Google-ArrivalTime: 2003-08-22 07:08:17 PST Path: archiver1.google.com!news1.google.com!newsfeed.stanford.edu!logbridge.uoregon.edu!newshub.sdsu.edu!elnk-nf2-pas!elnk-pas-nf1!newsfeed.earthlink.net!newsfeed2.easynews.com!newsfeed1.easynews.com!easynews.com!easynews!cyclone.swbell.net!bos-service1.ext.raytheon.com!dfw-service2.ext.raytheon.com.POSTED!not-for-mail Message-ID: <3F4623A7.6718B8C0@raytheon.com> From: Mark Johnson X-Mailer: Mozilla 4.76 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 Newsgroups: comp.lang.ada Subject: Re: float with 24-bit resolution References: <3F3CCB0F.543478AF@adrianhoe.nospam.com.my> <3f40a43e@baen1673807.greenlnk.net> <3F43D1AB.10205@attbi.com> <3F44CF31.570ADCFA@raytheon.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Fri, 22 Aug 2003 09:07:35 -0500 NNTP-Posting-Host: 192.27.48.39 X-Complaints-To: news@ext.ray.com X-Trace: dfw-service2.ext.raytheon.com 1061561296 192.27.48.39 (Fri, 22 Aug 2003 09:08:16 CDT) NNTP-Posting-Date: Fri, 22 Aug 2003 09:08:16 CDT Organization: Raytheon Company Xref: archiver1.google.com comp.lang.ada:41787 Date: 2003-08-22T09:07:35-05:00 List-Id: Ludovic Brenta wrote: > > Mark Johnson writes: > > [snip explanation of integer based fixed point decimal values] > > What you describe is a second-best solution, and by your own accord > you only used it because the target hardware did not support > packed-decimal representations. Your divide-by-10 routine was the > weak point; 10 happens not to be a power of two, and therefore any > division of a binary number by 10 is bound to introduce rounding and > imprecision. Perhaps you don't understand the solution I proposed. Using the example I provided, to get the digits for display of 0.42, you would: - divide 42 by 10, getting result 4 and remainder 2 (last digit) - note that 4<10, getting result 0 and remainder 4 (second digit) - generate the decimal point - generate the zero and display the value 0.42. No round off errors or any imprecision. This approach can be generalized to any fixed point decimal type. Depending on the range of values / precision required, it may require the occasional use of double precision values and integer division that produces both the result and remainder. As I mentioned before, you certainly have to do the "right thing" when it comes to doing arithmetic and conversions to other forms, but fixed point decimal arithmetic without packed decimal data types can be done without introducing any errors. > You may minimise the impact of that by doing the > division as late as possible, e.g. for display purposes; this is > probably what made the solution acceptable to you. However, if your > program must write to a text file that serves as input to another > program (and then another, etc.), then the loss of precision quickly > becomes unacceptable. Once you understand how to generate the correct values w/o any error, you will find that this statement does not apply. --Mark