Unexpected mathx.Round() result when rounding to 2 decimal places

[Dmitrevicz]

Unexpected mathx.Round() result when rounding to 2 decimal places

[icza]

The problem is with IEEE-754 standard. The numbers you pass to mathx.Round() are not the ones you think they are because they cannot be represented exactly with the IEEE-754 standard, only a very close, smaller number.

Let's increase the precision to 20 fraction digits to see more clearly:

func print(f, unit, exp float64) {
    rounded := mathx.Round(f, unit)
    fmt.Printf("input: %.20f | got: %.3f | expected: %.3f\n", f, rounded, exp)
}

This will output (try it on the Go Playground):

input: 2.34500000000000019540 | got: 2.350 | expected: 2.350
input: 1.27499999999999991118 | got: 1.270 | expected: 1.280
input: 4.93499999999999960920 | got: 4.930 | expected: 4.940

As you can see, the number you pass is 1.2749999..., and this rounded to 2 decimal places is 1.27. And the other, the number you pass is 4.9349999..., and this rounded to 2 decimal places is 4.93.

matx.Round() doesn't need fixing, you're just not passing the numbers you think you are.

[Dmitrevicz]

so there is no solution to achive what I want?

[icza]

You want to pass 1.275 to mathx.Round() and expect to get 1.28. But you can't pass 1.275 as a float64 because there is no float64 value being exactly 1.275.

[icza]

In decimal base you can write 1.275 with finite digits, because this is a "nice" number in base 10. But there is no finite representation of 1.275 in the binary IEEE-754 standard.

Just like you can't write 1 / 3 with finite digits in decimal base, as it equals to 0.3333333......

[Dmitrevicz]

Thanks for your answers