Line separator missing in js/ts output

[Vloz]

Line separator missing in js/ts output

[teamplayer3]

Can you try it with rust nightly. udoprog/genco#29

[teamplayer3]

Can you recheck your whole project with the latest state on main? Now it should support large integers.

[teamplayer3]

@Vloz Tried to fix it in 91faf79. Could you check, please?

[Vloz]

Doesnt seems work, from 6 to 6 linebreak missing in the readme example.
The function is being call 3times but the output has no linebreak (in the initial post, i added manually some for visibility, but output is always one line).

Had also difficulties with mid range number (like timestamp) which seems to led to incorrect parses. (maybe because i pass the message as Uint8Array rather than a number[]).

(Went to asm deserializer for the moment, crazy easy to implement with wasm-bindgen + tsify).

[teamplayer3]

Have you used the state of the main branch for testing it again? I always tested it with windows, when I used \r for line breaks. I had the same problem as you described in Linux. Now, it works for me.

I will check the Uint8Array thing. Can you give an example on how you pass the timestamp?

[Vloz]

Have you used the state of the main branch for testing it again?

Couldn't build with postcard-bindgen = { git = "https://github.com/teamplayer3/postcard-bindgen", branch = "main" }, cargo acted like no function were exposed (except PostcardBindings macro).
So i edited the original crate (from .cargo/registry) and add a println! right before it + Cargo clean to be sure modification were taken into account.
That how i can tell the function is being call 3times with the readme example, yet the output is still 1line.

I will check the Uint8Array thing. Can you give an example on how you pass the timestamp?

const data : ArrayBuffer = await (e.data as Blob).arrayBuffer(); //received data from websocket
const message = PsBindGenMessage.deserialize("SocketMessage", new Uint8Array(data) as unknown as number[]);

u64 value worked fine at value like 100 and 2^16+1, yet at some point i didn't try to figure the value is wrongly parsed and next fields are returned as undefined.
The same Uin8Array works fine when loaded from wasm postcard.

let status_code = StatusCode::Error { detail: String::from("Cargo watcher failed, the command produced no valid metadata (exit code: ExitStatus(ExitStatus(101))):") };

let message = SocketMessage::Status {
    code: status_code, //enum with string, works fine
    start_time: 6_000_000, //u64, Fail with this value, works with 100 or 65537
    mem: 100, // here and below will turn undefined if the value above fail
    xtime: 100,
    live_uploads: 100,
};` 

[teamplayer3]

postcard-bindgen = { git = "https://github.com/teamplayer3/postcard-bindgen", branch = "main" }

You have to pass the feature generating. This feature enables all the generating stuff. This is used to allow the macro derives in no_std crates and generate the bindings in another crate. Look at examples/no_std.

Could you run the example small with cargo run --example small --features=std,generating? This should work.

[Vloz]

You have to pass the feature generating. This feature enables all the generating stuff.

Sorry, build it again from git this time and still one line output.

Could you run the example small with cargo run --example small --features=std,generating? This should work.

Same result.
I am running on Windows 10.0.19045.

[teamplayer3]

Could you print the generated string with println!("{:?}", generate_bindings!(Test).js_file.as_str()); in your project? This should show the line endings. I'm a little confused why it outputs without line breaks.

[Vloz]

Could you print the generated string with println!("{:?}", generate_bindings!(Test).js_file.as_str()); in your project? This should show the line endings. I'm a little confused why it outputs without line breaks.

There is only one \n at the very end of the &str.
If this is an isolated issue we d rather close it for the moment (until somebody with more insight face it).

[Vloz]

Can you try it with rust nightly. udoprog/genco#29

Works! Both 0.2.0 and 0.3.0 output correctly formatted js once project to nightly.

Can you recheck your whole project with the latest state on main? Now it should support large integers.

Couldn't use my initial code with 0.3.0.
Still get parse with wrong and undefined value in 0.2.0, but with 0.3.0 throw me "variant not implemented" with [0, 197, 237, 230, 184, 252, 48, 128, 224, 177, 3, 100, 0] that is supposed to be "Status" with the following model:

Back in 0.2.0 the enum was correctly recognize:

[teamplayer3]

I checked your code example. If I serialize the message with postcard

postcard::to_vec::<_, 20>(&SocketMessage::Status {
            code: StatusCode::Starting,
            start_time: 12_123_123,
            mem: 100,
            cpu: 6_000,
            live_uploads: 0
        })
        .unwrap()

I get [0, 0, 243, 247, 227, 5, 100, 240, 46, 0]. If I deserialize the bytes in JavaScript again, I get the correct values:

{
  tag: 'Status',
  value: {
    code: { tag: 'Starting' },
    start_time: 12123123,
    mem: 100,
    cpu: 6000,
    live_uploads: 0
  }
}

Bytes interpreted:

[ 
  0,  <-- outer enum variant (Status)
  0, <-- enum StatusCode (Starting)
  243, 247, 227, 5,  <-- start_time
  100, <-- mem
  240, 46, <-- cpu 
  0 <-- live_uploads
]

Fixed braces, but this shouldn't be an issue. Can you check it again?

[Vloz]

Can you check it again?

Sorry the enum variant issue was because of some value i trimmed and forgot to put back. 😥
I get indeed [0, 0, 243, 247, 227, 5, 100, 240, 46, 0], but the value are still wrong somehow:

 {
tag: "Status",
value:{
code : {tag: "Starting"},
cpu: 100,
live_uploads: 6000,
mem:5,
start_time:undefined
}
}

I am copy/pasting this generated code to the console, then deserialize("SocketMessage",[0, 0, 243, 247, 227, 5, 100, 240, 46, 0])

[Vloz]

Sorry just figure out i was in 0.2.0 again... FFS 😣

[Vloz]

Everything is working fine with 0.3.0 and Rust nightly! 🙏

Thank you very much for the help and sorry for the incorrect report. 🤕

[teamplayer3]

No problem. Thank you for reporting and helping to fix some bugs💪 do you stick with wasm-bindgen or do you use this crate now? It would be interesting to have some benchmarks in comparison to the wasm-bindgen solution.

[Vloz]

do you stick with wasm-bindgen or do you use this crate now?

Hard to choose be honnest.
Could not figure out how to share wasm-bindgen types with my rust app (wasm is compiled as cdylib and macro fail compiling outside of this context). Plus tsify wont support variant enum key/value type as postcard-bidgen does because it need a specific serde macro not allowed by postcard (throwing "wont support" if set).
Ended up doing copy/past type from one project to the other.

While i can just put Postcard-bindgen types on a side project, run js generation in the build.rs and share its structs with my rust app without any struggle.

But outside of that, wasm-bindgen is more straightforward when used with serde-wasm-bindgen and tsify:

#[derive(Serialize, Deserialize, Tsify)]
#[tsify(into_wasm_abi)]
pub enum SocketMessage {
...
}

#[wasm_bindgen]
pub fn parse_socket_message(bytes: &[u8]) -> SocketMessage {
    postcard::from_bytes::<SocketMessage>(bytes).unwrap()
}

It would be interesting to have some benchmarks in comparison to the wasm-bindgen solution.

Thats a quite limited one... 😋 But for the 10bytes message we tested, sent with randomized u32 values called 1k/s:

	Postcard Bindgen	Wasm
First 10 calls	81µs	54µs
Stabilized at 4K	25µs	20µs
Size	10kB raw	28kB bin

[teamplayer3]

Could not figure out how to share wasm-bindgen types with my rust app (wasm is compiled as cdylib and macro fail compiling outside of this context). Plus tsify wont support variant enum key/value type as postcard-bidgen does because it need a specific serde macro not allowed by postcard (throwing "wont support" if set).
Ended up doing copy/past type from one project to the other.
While i can just put Postcard-bindgen types on a side project, run js generation in the build.rs and share its structs with my rust app without any struggle.

Thanks for sharing your thoughts. I wrote this crate a while ago to have a small binary on the rust side (derive macros only expand for the crate in which the bindings generate) because I used it for communication with an embedded device and to communicate with a mobile app over bluetooth.

Thats a quite limited one... 😋 But for the 10bytes message we tested, sent with randomized u32 values called 1k/s:

Ah, very cool. Maybe pstcard-bindgen could be a little faster, if not checking the types in JavaScript when using with typescript.

[teamplayer3]

Disabled js type checks in branch no-js-typechecks. Maybe you can run your benchmark again. But I don't know if it's that much of a difference if the types have few fields.

[Vloz]

Sorry just drop a few hours ago the benchmark setup to restore the whole project architecture and kept the wasm solution, all the tests were done within my main project (not using git yet). 🤕