DWARF doesn't work

[turbolent]

DWARF doesn't work

[RReverser]

Yay, with manual config + -DDWARF_OLD=1 it finally worked, thanks!

[RReverser]

Sorry, away from computer but the mini sample above + command for WASI SDK 19.0 should result in the same binary.

If not, I can rebuild one myself and share a bit later.

[RReverser]

Here's mini.wasm (zipped as Github doesn't like Wasm directly): mini.zip

[RReverser]

As for the missing line information: Can you please ensure that libdwarf is detected when building w2c2?

Ah no, I don't have libdwarf installed and didn't know it's necessary, that's probably it for #line:

-- Checking for module 'libdwarf'
--   Package 'libdwarf', required by 'virtual:world', not found

Might be useful to add this as a requirement to README, I don't think it's mentioned.

I'll install and recheck now.

[RReverser]

At least I can confirm the corrupted names issue is gone.

[SamuraiCrow]

The name that looks corrupted is name-mangled by C++. If you compile it in C mode it will not look like that.

[RReverser]

Hmm I don't think mangling ever produces invalid chars like that. It's supposed to still produce ASCII names.

[RReverser]

I recompiled it as regular C, it's still there:

U32 wasi_snapshot_preview1__fd_write(struct miniInstance*,U32,U32,U32,U32);

void wasi_snapshot_preview1__proc_exit(struct miniInstance*,U32);

void f5(miniInstance*) __asm__("mini_P���}U");

void f6(miniInstance*) __asm__("mini___imported_wasi_snapshot_preview1_fd_fdstat_get");

U32 f7(miniInstance*) __asm__("mini___imported_wasi_snapshot_preview1_fd_seek");

U32 f8(miniInstance*) __asm__("mini___imported_wasi_snapshot_preview1_fd_write");

[RReverser]

I probably should've also mentioned that w2c2 complains about some name subsections, but that should be fine - those are probably the new subsections from https://github.com/WebAssembly/extended-name-section proposal that could be nice to support in C translation at some point but not critical:

> ~/w2c2/w2c2/build/w2c2 -g mini.wasm mini2.c
w2c2: skipping unsupported name section subsection 7
w2c2: skipping unsupported name section subsection 9
w2c2: skipping custom section 'producers' (size 43)
w2c2: skipping custom section 'target_features' (size 14)

[RReverser]

FWIW on larger files with just the name section and no DWARF I'm also often getting this in -g mode:

w2c2: skipping unsupported unknown section
w2c2: skipping unsupported name section subsection 0
w2c2: skipping unsupported name section subsection 0
w2c2: failed to read module: invalid section read

and the translation fails. I can't reproduce this on trivial repro examples yet, but I suspect it's coming from the same issue that causes the corrupted name above.

[SamuraiCrow]

Maybe you're right. I'm not sure the WASI standard compiler uses the same name mangling scheme but it might be a bug in Clang even yet.

[RReverser]

It's not, I checked that wasm-objdump -j name -x mini.wasm and any other tools show correct names. I'm pretty sure this is a bug in w2c2's parser.

[SamuraiCrow]

Ok. Thanks for checking.

[RReverser]

Also interesting datapoint - the name numbering also seems wrong - compare indices in wasm-objdump and in generated header:

> wasm-objdump -j name -x mini.wasm

mini.wasm:      file format wasm 0x1

Section Details:

Custom:
 - name: "name"
 - func[0] <__imported_wasi_snapshot_preview1_fd_close>
 - func[1] <__imported_wasi_snapshot_preview1_fd_fdstat_get>
 - func[2] <__imported_wasi_snapshot_preview1_fd_seek>
 - func[3] <__imported_wasi_snapshot_preview1_fd_write>
 - func[4] <__imported_wasi_snapshot_preview1_proc_exit>
 - func[5] <__wasm_call_ctors>
 - func[6] <_start>
 - func[7] <a>
 - func[8] <b>
 - func[9] <c>
 - func[10] <__original_main>
 - ...

vs w2c2:

U32 wasi_snapshot_preview1__fd_close(struct miniInstance*,U32);

U32 wasi_snapshot_preview1__fd_fdstat_get(struct miniInstance*,U32,U32);

U32 wasi_snapshot_preview1__fd_seek(struct miniInstance*,U32,U64,U32,U32);

U32 wasi_snapshot_preview1__fd_write(struct miniInstance*,U32,U32,U32,U32);

void wasi_snapshot_preview1__proc_exit(struct miniInstance*,U32);

void f5(miniInstance*) __asm__("mini_P����V");

void f6(miniInstance*) __asm__("mini___imported_wasi_snapshot_preview1_fd_fdstat_get");

U32 f7(miniInstance*) __asm__("mini___imported_wasi_snapshot_preview1_fd_seek");

U32 f8(miniInstance*) __asm__("mini___imported_wasi_snapshot_preview1_fd_write");

U32 f9(miniInstance*) __asm__("mini___imported_wasi_snapshot_preview1_proc_exit");

U32 f10(miniInstance*) __asm__("mini___wasm_call_ctors");

...

Something happens right after imports that throws w2c2 off.

[RReverser]

Also can reproduce with Rust inputs:

fn a() -> i32 {
    42
}

fn b() -> i32 {
    a()
}

fn c() -> i32 {
    b()
}

fn main() {
    println!("{}", c());
}

$ rustc mini.rs --target wasm32-wasi -o mini.wasm
$ ~/w2c2/w2c2/build/w2c2 -g mini.wasm mini2.c
w2c2: ignoring duplicate function name dummy used by functions 147 and 161
w2c2: skipping unsupported name section subsection 7
w2c2: skipping unsupported name section subsection 9
w2c2: skipping custom section 'producers' (size 80)
w2c2: skipping custom section 'target_features' (size 14)

Same result - no #line directives and the header still has corrupted name entry:

U32 wasi_snapshot_preview1__fd_write(struct miniInstance*,U32,U32,U32,U32);

U32 wasi_snapshot_preview1__environ_get(struct miniInstance*,U32,U32);

U32 wasi_snapshot_preview1__environ_sizes_get(struct miniInstance*,U32,U32);

void wasi_snapshot_preview1__proc_exit(struct miniInstance*,U32);

void f4(miniInstance*) __asm__("mini_ ��y�U");

void f5(miniInstance*) __asm__("mini___imported_wasi_snapshot_preview1_environ_get");

U32 f6(miniInstance*) __asm__("mini___imported_wasi_snapshot_preview1_environ_sizes_get");

U32 f7(miniInstance*) __asm__("mini___imported_wasi_snapshot_preview1_proc_exit");

U32 f8(miniInstance*) __asm__("mini___wasm_call_ctors");

[turbolent]

Thank you for reporting this!

Could you please share the WASM binaries and provide some more information about the tools you used? What version of WASI SDK, Rust, and libdwarf did you use?

That would allow me to debug the issue.

Did you build w2c2 with libdwarf support enabled?

[turbolent]

@RReverser Thank you for sharing the module.

I tried to reproduce the issue, but everything is working OK for me.

Function names are generated properly in mini.h:

void f5(miniInstance*) __asm__("mini___imported_wasi_snapshot_preview1_fd_close");

void f6(miniInstance*) __asm__("mini___imported_wasi_snapshot_preview1_fd_fdstat_get");

U32 f7(miniInstance*) __asm__("mini___imported_wasi_snapshot_preview1_fd_seek");

U32 f8(miniInstance*) __asm__("mini___imported_wasi_snapshot_preview1_fd_write");

U32 f9(miniInstance*) __asm__("mini___imported_wasi_snapshot_preview1_proc_exit");

U32 f10(miniInstance*) __asm__("mini___wasm_call_ctors");

U32 f11(miniInstance*,U32) __asm__("mini__start");

U32 f12(miniInstance*,U32,U32) __asm__("mini_a");

U32 f13(miniInstance*,U32,U64,U32,U32) __asm__("mini_b");

U32 f14(miniInstance*,U32,U32,U32,U32) __asm__("mini_c");

void f15(miniInstance*,U32) __asm__("mini___original_main");

void f16(miniInstance*) __asm__("mini___wasi_fd_close");

Line annotations are generated properly in mini.c:

#line 3 "/home/rreverser/emqjs/mini.c"
U32 f7(miniInstance*i) {
U32 l0=0;
U32 si0;
#line 4 "/home/rreverser/emqjs/mini.c"
si0=42U;
#line 4 "/home/rreverser/emqjs/mini.c"
l0=si0;
#line 4 "/home/rreverser/emqjs/mini.c"
si0=l0;
#line 4 "/home/rreverser/emqjs/mini.c"
goto L0;
#line 4 "/home/rreverser/emqjs/mini.c"
L0:;
return si0;
}

#line 4 "/home/rreverser/emqjs/mini.c"
U32 f8(miniInstance*i) {
U32 l0=0;
U32 si0;
#line 7 "/home/rreverser/emqjs/mini.c"
si0=f7(i);
#line 7 "/home/rreverser/emqjs/mini.c"
l0=si0;
#line 8 "/home/rreverser/emqjs/mini.c"
si0=l0;
#line 8 "/home/rreverser/emqjs/mini.c"
goto L0;
#line 8 "/home/rreverser/emqjs/mini.c"
L0:;
return si0;
}

Maybe this is an architecture/OS problem? I'm on macOS/arm64, but will also try on Linux/x86-64, and am using libdwarf v0.6.0.

Could you please share some more details about the compiler you are using to build w2c2, and what version of libdwarf you are using? Thanks!

[turbolent]

The warnings

w2c2: skipping unsupported name section subsection 7
w2c2: skipping unsupported name section subsection 9

resulted from the module using the extended-name-section proposal.

I added support for those section to the module parser and improved the message, so now they read as

w2c2: skipping unsupported global names subsection (size 18)
w2c2: skipping unsupported data segment names subsection (size 17)

[turbolent]

There was a bug in the allocation of function names, it didn't account for names of imported functions.
This should get fixed by #71.

As for the missing line information: Can you please ensure that libdwarf is detected when building w2c2?
For example, you should see something like:

-- Checking for module 'libdwarf'
--   Found libdwarf, version 0.6.0

Please let me know if you're still unable to produce line information once it's found, and I'll reopen the issue and investigate.

[RReverser]

Hm I installed libdwarf1 & libdwarf-dev from apt, but it still complains it's not found. I wonder what it wants from me :/

[RReverser]

Hm I installed libdwarf1 & libdwarf-dev from apt, but it still complains it's not found. I wonder what it wants from me :/

@turbolent So... could you please suggest which package it expects on Linux?

[turbolent]

@RReverser libdwarf's API is unfortunately changing constantly and significantly.

w2c2 supports both older and newer APIs of libdwarf. This was already supported by the Makefile-based build system, but I also added support for the switch to the CMake configuration in 6feda49.

Furthermore I added a section with installation instructions for libdwarf:
https://github.com/turbolent/w2c2/blob/main/README.md#installing-libdwarf-required-for-source-line-mapping

As mentioned earlier, if you could provide some more information about your build environment, e.g. what OS/distribution you are using and what version of libdwarf you have installed, then I could assist you better.

Based on the naming of the libraries you mentioned, I assume you are on a Debian/Ubuntu-derived system.
The latest version they provide is 20210528, which is quite old and does not provide a pkg-config file, so CMake cannot find it automatically.

As described in the README, passing the following options to CMake might work for you:

-DDWARF_FOUND=1 -DDWARF_LIBRARIES=-ldwarf -DDWARF_LIBRARY_DIRS=/usr/lib -DDWARF_INCLUDE_DIRS=/usr/include/libdwarf -DDWARF_OLD=1

That works for me on Ubuntu 20.04, which ships libdwarf 20200114.

[RReverser]

As mentioned earlier, if you could provide some more information about your build environment, e.g. what OS/distribution you are using and what version of libdwarf you have installed, then I could assist you better.

Right, sorry, given that the root seemed to be just me missing libdwarf, I assumed this is no longer relevant. It's Ubuntu 22.10.

I'll try the updated README instructions, thanks!