Hi Leonard,
On Thursday, 2007-06-07 23:04:10 +0300, Leonard Mada wrote:
> Unfortunately, the formulas/algorithms given for 'a' and 'b' seem to be
> computationally unstable
> - they both use terms of the form n*sum(x^2) - (sum(x))^2 which are
> notoriously unstable
>
> Although I did NOT do a formal calculation of the error term, having
> *two* occurrences of this unstable calculation, and both times as
> *divisors* makes the formula very unreliable.
Thanks for pointing out. Did you try it out in OOoCalc and noticed
shortcomings?
> SUGGESTIONS:
> - remove formula from specification
I think that is not a good option because it would leave the definition
without a mathematical description. Even if the given formula may
numerically be instable at some point it is mathematically correct.
However, we could add a note saying so.
> - consult statistician knowledgeable of stable coding algorithms
That of course would be favourable. As I also know from the OOo mailing
lists that you're interested in that area, could you help us or are you
in contact with one?
Eike
--
OpenOffice.org Engineering at Sun: http://blogs.sun.com/GullFOSS
