wasm: add DWARF sections

[walkingeyerobot]

What version of Go are you using (go version)?

$ go version
go version go1.12.5 linux/amd64

Output source maps for use with browser developer tools to enable simple step debugging inside browsers. Details of the spec can be found here: https://sourcemaps.info/spec.html

[agnivade]

If I am reading the link correctly, it is the spec for JS source maps. Wasm source maps are not part of a specification yet and it is still in progress. Discussion is ongoing here - WebAssembly/design#1051.

[walkingeyerobot]

It's the spec for sourcemaps within browser dev tools, not specific to javascript. Emscripten is already capable of outputting these for c++ compiled to wasm. See their docs at https://emscripten.org/docs/porting/Debugging.html for more detail.

[agnivade]

I see. Thanks.

[torbenschinke]

Meanwhile Chrome got native dwarf-support for wasm-files, see [1]. It could be a better / simpler alternative to embed debug symbols into the wasm file directly.

Also, one can derive source maps from embedded symbols using tools like [2]. So a dwarf support may be sufficient.

It looks like gc does not embed those symbols currently? Do you have any insights?

[1] https://developers.google.com/web/updates/2019/12/webassembly#enter_dwarf
[2] https://chromium.googlesource.com/external/github.com/kripken/emscripten/+/1.38.11/tools/wasm-sourcemap.py

[jeremyfaller]

The linker explicitly strips DWARF from generated wasm binaries. I would need to about the wasm binary format to add support for DWARF, but it's probably entirely doable to just tweak the linker to spit out DWARF.

[torbenschinke]

Yury Delendik has summarized his findings in [1] and it looks like each DWARF section has a custom section within the wasm binary, just like it shall be in ELF binaries.

[1]https://yurydelendik.github.io/webassembly-dwarf/

[torbenschinke]

@jeremyfaller Are you planning to work on adding support for DWARF?

[jeremyfaller]

Yeah, I'll take a look. It won't make 1.15 (that's frozen), but I'll put something together (on master) that can make 1.16.

[jeremyfaller]

This is looking less likely for me to get to for 1.16. I'm being lent over to the COVID exposure notification work. I would be happy to do code reviews for anyone that takes over. For now, I'm un-committing myself to delivering for 1.16.

[haselkern]

Hi @jeremyfaller! I would be willing to put some time and effort into this. Could you give me a quick rundown of what would need to be done to make this work?

[haselkern]

I have extended the compiler to include DWARF symbols in the Wasm-File, and got Chrome to recognize that there are debug symbols available. But it cannot yet load the underlying source code.

wasm-validate from the wabt project project passes and the .wasm file is executed correctly. However dwarfdump reports error: main.wasm: Invalid data was encountered while parsing the file. So somehow there is something wrong with the binary format in the output file.

Since I'm no expert on this, any pointers in the right direction would be greatly appreciated, though I will continue to look into this. I have attached a zip containing a compiled Wasm-file, if that is a help to anyone.

wasm-dwarf.zip

[thanm]

Which dwarf dumper are you using? Is this the one based on https://www.prevanders.net/dwarf.html ?

I would try building and running llvm-dwarfdump instead, that dumper definitely has support for wasm.

[haselkern]

I'm on a Mac using Apples llvm-dwarfdump and could successfully use that to inspect a Wasm file with debug information generated from Rust code. Thank you for your suggestion though!

[haselkern]

I'm now at a point where I could use some more help. I had the problem that the .debug_abbrev section was not written correctly, but I have since fixed that. I wrote a small helper program that uses Go's built in dwarf functionality and that is able to correctly (?) print out DWARF information from a Wasm file generated from Rust and also from my modified Go compiler! Unfortunately Chrome still does not pick up the source map and dwarfdump also still reports error: main.wasm: Invalid data was encountered while parsing the file.

I have attached a zip containing my helper program and both Wasm files. You can run go run helper.go -file go.wasm (or rust.wasm) and the helper will print the sections present in the Wasm file and then all DIEs.

If anyone has an idea as to what I might be doing wrong, do not hesitate to contact me!

wasm.zip

[happybeing]

@nbaksalyar is this something you can help answer?

[RReverser]

Hi @haselkern! I'm very excited to see DWARF support getting ported to the WebAssembly target.

On the Chrome side, we're currently making some improvements to the WebAssembly debugging, and things are a bit in flux, which might be at least partially responsible for why you're not seeing sources (assuming that DWARF is otherwise valid).

We'll be ready to share more in a week at the ChromeDevSummit as part of the https://developer.chrome.com/devsummit/sessions/debugging-webassembly-with-modern-tools/ session. Perhaps it's worth syncing up after that?

Meanwhile, llvm-dwarfdump should be the easiest tool for verifying that DWARF data is correctly encoded in WebAssembly files. (UPD: and it does look like it fails on your go.wasm)

[RReverser]

Okay, Rust/Gimli-based dwarfdump gives a lot better output (as usual :P). I'd suggest trying it out for further debugging.

1. In case you don't have Rust toolchain installed, install it from https://rustup.rs/.
2. Install Gimli's dwarfdump example with $ cargo install gimli --example dwarfdump.

Here is the output log it produced: out.log

In there, you can see a bunch of "Unable to get header for file [number]" in the DWARF data - looks like files are not properly declared.

Additionally, all (?) the locations like DW_AT_stmt_list, DW_AT_ranges, DW_AT_low_pc, DW_AT_high_pc and DW_AT_type seems to be set to zero (0x00000000).

[RReverser]

Hi again! We now have released the mentioned new and enhanced DWARF debugging experience for WebAssembly - check out this post about various new features and usage instructions: https://developers.google.com/web/updates/2020/12/webassembly (or check out the short talk here https://youtu.be/VBMHswhun-s instead).

I believe, once issues, described in my previous comment, are addressed and Go emits correct DWARF information, it should mostly work out of the box with Go-emitted Wasm too, but there isn't much I can help with until then.

[gopherbot]

Change https://golang.org/cl/283012 mentions this issue: wasm: include DWARF information in Wasm builds

[haselkern]

I opened a change request so that other people could look into this. I could not figure out why the locations were all set to zero, considering that this information is written correctly on other architectures. Any ideas on what would need to be changed are very welcome!

[RReverser]

Any ideas on what would need to be changed are very welcome!

Is it possible you need special handling for Wasm relocations? I'm not too familiar with Go, is there a way to emit object files before relocation & linking is done?

If it is, then you should be able to use dwarfdump tool from above to check whether at least individual objects contain correct debug info.

If they do, then the issue is in the relocation/linker implementation, if they don't, then it's during emitting itself.

[haselkern]

If anyone can give me a clear direction to what exactly is missing, I would be happy to implement it. But as it is now I will stop working on this, as I have made no meaningful progress in the past few weeks.

[3p3r]

I'm trying to implement this feature into the compiler. Had these patches applied: https://go.googlesource.com/go/+/418ef9ce78ffe2c27341dacb6cbe88bc1787b797

If you have any pointers how I can best continue, would love to hear them. @jeremyfaller you seem to had interest in implementing this at some point. maybe you could shed some light on this?

[3p3r]

little bit of an update: I did manage to get symbols loaded in Chrome correctly with the correct file paths (with a patched go1.18). however, the actual source data is missing as indicated in the patch description.

I am having difficulties debugging the ld (linker) package with dlv. Any pointers on how I can set up the linker for debugging? How do the maintainers go about working on the linker normally?