comp.lang.ada
 help / color / mirror / Atom feed
From: tmoran@acm.org
Subject: Re: OT: definition of "significant figures"
Date: Mon, 01 Aug 2005 01:50:21 -0500
Date: 2005-08-01T01:50:21-05:00	[thread overview]
Message-ID: <vYWdne85mKqwW3DfRVn-tw@comcast.com> (raw)
In-Reply-To: e3nqe19bqp99h20anetgc7m63ai8ol84nv@4ax.com

I think a better formula for the number of digits to display is
   Digit_Count := Integer(Long_Float'ceiling(Log(Abs(Avg),10.0)))
                  -Integer(Log(Sd,10.0));
where the positive addition to the digits comes from a 'ceiling on
log(avg), but a subtraction from the number of digits comes from a
rounded value of log(sd)
  Using this formula, one gets
>         The average of 2.1, 33.1, 2.2
avg = 12.46666, sd =  16.24
    log(avg) = 1.096, 'ceiling = 2
    log(sd)  =-1.211, rounded  = 1
so the digit count is 1, rather than log(avg/sd)= -0.1
  If you don't use the 'ceiling, consider the case where the x(i) are
in the range 99.990 .. 99.999, vs the same set with .01 added, so
their range is 100.000 .. 100.009   Since their std devs are the same,
in the .00x range, their averages ought to be represented with the same
number of digits to the right of the decimal.  The 100's shouldn't
lose a digit compared to the 9's.



  parent reply	other threads:[~2005-08-01  6:50 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <e3nqe19bqp99h20anetgc7m63ai8ol84nv@4ax.com>
2005-08-01  4:11 ` OT: definition of "significant figures" tmoran
2005-08-01  6:50 ` tmoran [this message]
2005-08-01 16:58   ` tmoran
2005-08-01 23:56     ` tmoran
2005-07-29  4:23 tmoran
2005-07-29 14:46 ` Jacob Sparre Andersen
2005-07-30 23:44   ` Dr. Adrian Wrigley
2005-07-31  7:02     ` tmoran
2005-08-01  7:31     ` Jacob Sparre Andersen
2005-07-31  8:36 ` Dmitry A. Kazakov
2005-07-31 19:08   ` tmoran
replies disabled

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox