fp_cmd returns wrong results with FP_RDC=MONTY

[relic-toolkit]

fp_cmd returns wrong results with FP_RDC=MONTY

[dfaranha]

Thanks! I think this is caused by a mismatch between the fp_print() and fp_cmp() functions. To facilitate debugging, fp_print() converts back from the Montgomery representation, while fp_cmp() compares raw digit vectors. Far from intuitive, I know. :(

Perhaps fp_cmp() should only perform equality checks, since a finite field does not admit a total order anyways?

[FiloSottile]

I see. I don’t know what the right answer is, but its semantics should not change based on a configuration option. In practice I do need lexicographic ordering for a point compression scheme I have to interoperate with. Does comparing Montgomery representations have any meaning?

[dfaranha]

Agree, will move this to the DV module which does not care about fields and change fp_cmp() for equality checks, which work among different representations.

Comparing Montgomery representations does not conserve any ordering, since everything is multiplied by (R mod p); and you should not transmit coordinates in Montgomery form anyway. Maybe you should first export to byte vectors with fp_write_bin() and then perform the comparison?

[FiloSottile]

I meant that if comparing Montgomery representations is meaningless it would be reasonable to make fp_cmp convert back before comparing, but yeah also comparing fp_write_bin outputs works for me.

[dfaranha]

Fixed the issue according to the solution suggested above in commit 78a1266

[dfaranha]

Thanks again for reporting!