Re: New IEEE Language Popularity Ratings

[rieachus@comcast.net]

On Saturday, August 6, 2016 at 11:53:56 AM UTC-4, brbar...@gmail.com wrote:

> 1.  Do I trust the algorithms in the library?
> 
> Numerical procedures may need care to ensure they've been implemented carefully.
> For example, if one has to do matrix inversion, it's important to use a Singular
> Value Decomposition (SVD) algorithm implemented with long floats (or double
> precision).  I don't know whether spreadsheet inversions do that, so I might
> not want to use a spreadsheet - even if the spreadsheet has matrix inversion.

I'm going to take potshots at the inversion advice here, random numbers to follow.

I have always used LUP decomposition for inversion of large matrices. (And scaling if the values have a wide range.)  L is a lower triangular matrix, U an upper triangular matrix, and P is a permutation matrix. (All zeroes except one one in each row and column.)  Computing the inverse from the LUP decomposition is easy, and multiplying out the LUP decomposition will tell you how badly you have mangled the data.  (And yes, you should also multiply the original matrix by the inverse to get a residuals matrix.)  

As for using double precision, consider that a default.  Packages for x86 family chips can use the x87 IEEE Extended support that is still there.  There are (mostly software) routines that provide even more precision.  Why so much precision, when you are not measuring the diameter of the galaxy to the nearest atom? Because mixing float addition and multiplication burns off precision very quickly.  In fact, for some peculiar data, I have used discrete arithmetic, and an arbitrary precision package.  Not needed often, but if you are investing your money based on the results?  An extra few minutes of CPU time are cheap.