How to debug and handle ScalarError?

[rakaly]

How to debug and handle ScalarError?

[nickbabcock]

What are you deserializing locked and total as? The parser can handle numbers and dots just fine 😄

#[test]
fn test_deserialize_manpower_pool() {
    #[derive(Deserialize, Debug, PartialEq)]
    struct Root {
        manpower_pool: ManpowerPool,
    }

    #[derive(Deserialize, Debug, PartialEq)]
    struct ManpowerPool {
        available: i32,
        locked: String,
        total: String,
    }

    let data = br#"
    manpower_pool={
        available=95915
        locked=3379.3.22.15
        total=3782.1.21.20
    }
"#;

    let txt_out: Root = jomini::text::de::from_windows1252_slice(&data[..]).unwrap();
    assert_eq!(
        txt_out.manpower_pool,
        ManpowerPool {
            available: 95915,
            locked: String::from("3379.3.22.15"),
            total: String::from("3782.1.21.20"),
        }
    );
}

[LokiSharp]

I think it should be a number, but I don't know why there be a number with dot as string.

Is there a data type or format similar to 'pdxdate' that includes a dot as part of the number string?

[nickbabcock]

I'm not sure what exactly you're looking for, but most everything is possible if you want to write your own deserializer function. Below shows prototype of how you can get rid of the (potentially omitted) dots and interpret the fields as numbers.

fn deserialize_pool_field<'de, D>(deserializer: D) -> Result<i32, D::Error>
where
    D: Deserializer<'de>,
{
    struct PoolVisitor;

    impl<'de> de::Visitor<'de> for PoolVisitor {
        type Value = i32;

        fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
            formatter.write_str("a pool field")
        }

        fn visit_str<E>(self, v: &str) -> Result<Self::Value, E>
        where
            E: de::Error,
        {
            let first = v.split('.').next().unwrap();
            Ok(first.parse::<i32>().unwrap())
        }
    }

    deserializer.deserialize_str(PoolVisitor)
}

#[test]
fn test_deserialize_manpower_pool() {
    #[derive(Deserialize, Debug, PartialEq)]
    struct Root {
        manpower_pool: ManpowerPool,
    }

    #[derive(Deserialize, Debug, PartialEq)]
    struct ManpowerPool {
        available: i32,
        #[serde(deserialize_with = "deserialize_pool_field")]
        locked: i32,
        #[serde(deserialize_with = "deserialize_pool_field")]
        total: i32,
    }

    let data = br#"
    manpower_pool={
        available=95915
        locked=3379.3.22.15
        total=3782
    }
"#;

    let txt_out: Root = jomini::text::de::from_windows1252_slice(&data[..]).unwrap();
    assert_eq!(
        txt_out.manpower_pool,
        ManpowerPool {
            available: 95915,
            locked: 3379,
            total: 3782,
        }
    );
}

[LokiSharp]

manpower_pool={
	available=95915
	locked=73401974
	total=76930819
}

Oh, I think there may be a bug in Melt. It would be helpful to set the binary with the debug_saves=1 setting to see if this is true.

[LokiSharp]

Here is my saves https://drive.google.com/file/d/1RAOwDeYgVQgg1XOQa_tN49t5p4dRfoy7/view?usp=sharing

the path is states -> 605 ->manpower_pool

[nickbabcock]

Thanks for the saves. It probably is a problem with the melter, as the binary format isn't self descriptive, so we have to rely on heuristics. We've worked around this issue in other games by informing the melter of fields that we either know are dates or know are numbers. I'm assuming we'll be able to do the same fix here 😄

[LokiSharp]

Thank you for your response. This save was melted by pdxtools. I tried to melt it with hoi4save, but I received an error message Error: Hoi4Error(UnknownToken { token_id: 10805 }). Is there an updated version of hoi4save available?

[nickbabcock]

Thank you for your response. This save was melted by pdxtools. I tried to melt it with hoi4save, but I received an error message Error: Hoi4Error(UnknownToken { token_id: 10805 }). Is there an updated version of hoi4save available?

I updated pdx tools, did you want to try it again and let me know if it works?

And yeah when hoi4save is run locally, you need to supply the mapping from binary tokens to text as paradox doesn't allow me to distribute them with the code.

[LokiSharp]

Yes, it works. I have created my own token table for melting HOI4 saves. However, when I compared those save files, I encountered a new issue with pdxtools. Some tokens were missing or incorrect, such as 'rk_token_XXX'.

[LokiSharp]

This is my own token table extends hoi4bin2txt at paradoxwikis

https://drive.google.com/file/d/1NvMnrI3NAOMSunsdxFyExdPgXTK2ifvD/view?usp=share_link

[crschnick]

There are version differences between the tokens. Our token update is for 1.12.10, I don't know which version yours are for. With hoi4, there are breaking changes between versions, so it's always a little be complicated to support all save versions from one token set.

[LokiSharp]

When I use PDX tools to melt my save file, I notice some incorrect tokens such as rk_token_XXX and aurelien_delay_XXX. For example, the aurelien_delay_14487 token should be can_be_fired, and rk_token_112 should be all_playthrough_data.

[nickbabcock]

Looks like you're working with a save file from the latest HOI4 beta. We don't typically track beta releases, but in this case, I've updated pdx tools, so those tokens you mention are now detected.