When using jslt function number() and passing string with big decimal for eg "180000000000" returns scientifc/exponential notation

[jugup]

When using jslt function number() and passing string with big decimal for eg "180000000000" returns scientifc/exponential notation

[larsga]

It doesn't help to tell Jackson not to use DoubleNode, because that's what JSLT uses internally. I didn't want to use BigDecimal internally for performance reasons.

But if your consumer uses a JSON parser that can't read 1.8E7 then ... well ... it's a part of JSON, so their JSON parser is broken.

[larsga]

I assume it's the last line here,

output = mapper.writeValueAsString(transformedData);

where the number becomes "1.8E7"? If so, this is not a JSLT question, since JSLT is returning a JsonNode with a decimal number in it. The choice to represent it as "1.8E7" in string form is made by Jackson, not JSLT. I don't think it's possible to modify JSLT (or your JSLT code) to change that.

[jugup]

If its jackson, wouldnt {"amount": 18000000.00000} also give out the exponential result, but instead it works fine, but not when {"amount": "18000000.00000"}

and we have our Jackson mapper to not use scientific notation

[larsga]

I agree it's strange that Jackson does not appear to be obeying your settings, but to see why I think this has nothing to do with JSLT, try running the code like this:

            JsonNode transformedData;
            JsonNode inputJson = mapper.readTree("18000000.00000");
            output = mapper.writeValueAsString(inputJson);

You should still get "1.8E7", even with no JSLT involved at all.

[jugup]

Tried the code, and we are getting expected result
18000000.00000

[jugup]

So it does seem that the number() function is converting it into scientific notation

[larsga]

Nope.

This code:

import com.fasterxml.jackson.core.JsonParseException;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.schibsted.spt.data.jslt.Parser;
import com.schibsted.spt.data.jslt.Expression;
import com.schibsted.spt.data.jslt.impl.ExpressionImpl;

import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.node.JsonNodeFactory;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.JsonGenerator.Feature;

public class Test {

  public static void main(String[] args) throws Exception {
    // Create mapper for parsing
    ObjectMapper mapper = new ObjectMapper().enable(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS)
      .setNodeFactory(JsonNodeFactory.withExactBigDecimals(true))
      .enable(JsonGenerator.Feature.WRITE_BIGDECIMAL_AS_PLAIN);

    // Enable features to handle long decimal values rather than
    // converting them in exponential format
    mapper.enable(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS);
    mapper.enable(Feature.WRITE_BIGDECIMAL_AS_PLAIN);

    // Initialize data and mapping
    JsonNode inputJson = mapper.readTree("18000000.00000");
    //JsonNode transformedData = template.apply(inputJson);
    String output = mapper.writeValueAsString(inputJson);
    System.out.println("output: " + output);
  }
}

produces:

MacBook-Pro-3:tmp larsga$ java Test
output: 18000000.00000

[jugup]

Hello @larsga, as you have also tried the code, when you do

    // Initialize data and mapping
    JsonNode inputJson = mapper.readTree("18000000.00000");
    //JsonNode transformedData = template.apply(inputJson);
    String output = mapper.writeValueAsString(inputJson);
    System.out.println("output: " + output);

it outputs 18000000.00000 but initially you mentioned it would return 1.8E7.

I think there is some confusion. Maybe we can go over the problem statement again.

1. While parsing big numbers using jackson and Feature.WRITE_BIGDECIMAL_AS_PLAIN it does not scientific/exponential result, number is in plain text.
2. When use JSLT with number() function and parse the big number, returned out put is in scientific/exponential notation. Which we think is unexpected because without JSLT just with jackson it works fine (as tried by both of us), maybe the function is causing some issues?

[larsga]

You're right, of course, I was too quick there. Sorry.

The test above showed that when JSLT is not used, Jackson gives the correct output. Your claim is that putting the string through the number() function somehow changes that. So let's test that, too.

I get the same result as you: output: 1.8E7. OK, so something is going on in Jackson.

This code:

    // Initialize data and mapping
    JsonNode inputJson = mapper.readTree("18000000.00000");
    System.out.println("" + inputJson + " " + inputJson.getClass());

    inputJson = mapper.readTree("\"18000000.00000\"");
    JsonNode transformedData = template.apply(inputJson);
    String output = mapper.writeValueAsString(transformedData);
    System.out.println("" + transformedData + " " + transformedData.getClass());

produces:

18000000.00000 class com.fasterxml.jackson.databind.node.DecimalNode
1.8E7 class com.fasterxml.jackson.databind.node.DoubleNode

So the problem here is that Jackson is serializing DecimalNode in a different way from DoubleNode.

Note that what you are having problems with is the string representation of the number, but JSLT is returning DoubleNode; that is, a number, not a string. So the problem is still how Jackson converts the number to a string. It's not JSLT that's producing the string, but Jackson.

You've set WRITE_BIGDECIMAL_AS_PLAIN, but this isn't a BIgDecimal, but rather a DoubleNode. If you convert the node to a DecimalNode the problem goes away. Add this line at the end:

    System.out.println("" + DecimalNode.valueOf(transformedData.decimalValue()));

and you get:

18000000.00000 class com.fasterxml.jackson.databind.node.DecimalNode
1.8E7 class com.fasterxml.jackson.databind.node.DoubleNode
1.8E+7

[larsga]

Sorry. Still too quick. I'm in a rush here.

Last line:

    System.out.println(mapper.writeValueAsString(DecimalNode.valueOf(transformedData.decimalValue())));

Output:

18000000.00000 class com.fasterxml.jackson.databind.node.DecimalNode
1.8E7 class com.fasterxml.jackson.databind.node.DoubleNode
18000000

[jugup]

Yeah, was able to reproduce the same outcome. Tried out following seting on mapper to not use DoubleNode

            ObjectMapper mapper = new ObjectMapper().enable(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS)
                    .setNodeFactory(JsonNodeFactory.withExactBigDecimals(true))
                    .configure(JsonGenerator.Feature.WRITE_BIGDECIMAL_AS_PLAIN, true)
                    .configure(DeserializationFeature.USE_BIG_INTEGER_FOR_INTS, true)
                    .configure(SerializationFeature.WRITE_BIGDECIMAL_AS_PLAIN, true);

but still it doesnt work. Do you know any solution which can help us to overcome this issue?

[larsga]

I'm afraid I don't know of any way to change Jackson's behaviour here.

But I do have one question for you. Why do you care? It's a perfectly usable serialization of the number.

[jugup]

Yeah, i agree, it shouldnt have mattered so much, but have one cosumer who is not able to convert it back to actual number, will try to convence that this is how it is.