sqrt rounding

[eobermuhlner]

sqrt rounding

[eobermuhlner]

Actually the last digit in the calculated precision is not possible to calculate absolutely correct due to rounding issues beyond the specified precision.

For this reason the documentation of the Java 9 sqrt() function has a caveat:

The value of the returned result is always within one ulp of the exact decimal value for the precision in question. If the rounding mode is HALF_UP, HALF_DOWN, or HALF_EVEN, the result is within one half an ulp of the exact decimal value.

This holds also true for the calculations in BigDecimalMath.
I should probably add this to the documentation of BigDecimalMath.

The BigDecimalMath.sqrt() calculates with 6 digits additional precision and then rounds the result to the specified MathContext - which in average is more accurate than the Java 9 implementation of sqrt() (at least it was last time a looked at it).

[MikeMcl]

This holds also true for the calculations in BigDecimalMath.

Not quite. The examples show the result can be off by just over half an ulp with rounding mode HALF_UP.

Anyway, I think your choice to calculate to 6 additional digits before rounding is perfectly reasonable. I just raised the issue for completeness because the behaviour isn't documented.

I think its only in Java 15 that BigDecimal sqrt gets the rounding spot on (at the cost of considerable added complexity).