Re: floating point comparison

[Samuel Mize <smize@link.com>]

W. Wesley Groleau x4923 wrote:
> What I'm trying to say is that it should be possible to come up with a
> few simple guidelines that cover most cases, and a guideline for
> identifying the cases that require a mathematician--excuse me, a
> numerical analyst.

Well, I'll take a WAG at it.  Let's see how quickly it gets shredded.
It's based on fairly cold memory, and it may miss important points.

***** DRAFT -- POSTED FOR COMMENT ONLY -- DO NOT USE *****

This estimation tells you if you are very safe using floating point.
If not, you need to use numerical analysis to determine the actual
characteristics of your data.

First, let's distinguish between accuracy and precision.  Accuracy
defines how close to the right number you are; precision defines how
finely you distinguish between numbers.  For instance, if we estimate
pi as 854.83049859234875034629348, this is very precise, but wildly
inaccurate.  You cannot be more accurate than you are precise.

Typically you are computing a result from some inputs.  First you
must determine how accurate your inputs are.  Express this as the
largest possible error for each input (its "error margin").

Each computation reduces the accuracy of its result.  If your
result's error margin is well below the accuracy you need for
your answer, you can use floating point.

ADDITION/SUBTRACTION: The error margin of the result is the largest
of the two input error margins.  If the smaller input error is
above 1% of the larger, assume that the larger error margin doubles.

MULTIPLICATION/DIVISION: The result error margin is the sum of the
two input error margins.

Thus, given the accuracies shown in the declarations below:
  declare
    A: float; -- error is 0.01
    B: float; -- error is 0.001
    C: float; -- error is 0.000001
  begin
    X := A + B; -- error is 0.02 (double, smaller margin within 1%)
    Y := X + A; -- error is 0.04
    Z := Y + C; -- error stays 0.04, C does almost nothing to result

    X := Z * B; -- error is 0.041
    Y := X * A; -- error is 0.051

SPECIAL WARNINGS: Don't depend on results from a computation where
one input value is about as big, or smaller than, the other input's
error margin.  Repeated computations that don't appear to affect the
error margin eventually will.  Assume that a loop will execute the
greatest possible number of times.

If you need to compare two numbers and see if they are "about" the
same, test whether their difference is less than the sum of their
error margins.  (If this doesn't give you close enough results,
your inputs aren't accurate enough to support that calculation.)

If you can't define an absolute error bound for an input, this
estimation can't help you.  You need to read up on numerical
analysis.  For instance, if your input is correct to five digits,
but it ranges from 0.0001 to 1000000, this estimation has to assume
that the input's error margin is 10.0 (the error of the largest
value). This is not likely to be helpful if it is meaningful to
measure down to 0.0001 with five-digit precision.

Sam Mize