First draft of DebugInfomationFormat.md.

[dekimir]

@yurydelendik @sunfishcode @jfbastien @dschuff: PTAL

[sunfishcode]

The name section is aimed at debugging wasm at the wasm level, which is a related but independent concern that is not redundant with debugging at the high-level-language level.

[dekimir]

Ah, I see. Definitely not redundant, then. I'll drop this sentence.

[lukewagner]

(Line notes tend to disappear on GH, so I'll put my comment here, even though it's extending the convo in line notes above.) First of all, thanks for posting this and kicking off a discussion! I think @sunfishcode raises an important "meta" topic that would help set the context for discussing this particular proposal.

In general, I see there being a spectrum of expressiveness/user experience/complexity here:

1. current "names" section (possibly extended with line/file info, as @yurydelendik pointed out)
2. something at the level of source maps (perhaps tweaked to be more wasm-appropriate)
3. something with the DWARF-like abilities (mini language/VM built inside)
4. the vision @sunfishcode outlined (which we've also discussed in the past with @flagxor, @titzer, et al) of exposing a low-level debug interface and opaque module metadata so users can do whatever they want

I see the value of 1 (just enough to provide useful backtraces in both devtools and Error.stack). 2 has potential in that it could leverage existing devtool work supporting sourcemaps. 3 vs. 4 has been a question for a while and I lean towards 4 simply because specifying and implementing 3 seems like a huge undertaking (universal debugging format sounds even harder than universal VM :).

IIUC, this proposal is starting closer to 2 but is talking about extending toward 3. Is that accurate? If so, I'd be interested to talk about 3 vs. 4 since, if we think 4 is the better long strategy, then that could set the context for this proposal as being more squarely focused at 2.

[dekimir]

Thanks, @lukewagner. The way I (very imperfectly) understand things, 1 and 2 are much better suited to debugging compiled code than source code. For instance, I don't know how these would support showing a source variable value.

I'm warming up to 4 because of its flexibility and extendability, but I need to understand more details. For starters, how do we express arbitrary debug info in the text format? And the browser API will have to be bidirectional: dev tools UI would tell the debug library, say, that the user wants to set a breakpoint in some source location, and, conversely, the debug library would have to ask the browser for current wasm memory contents. We'll have to spell out those details and evaluate how future-proof they are.

Interestingly, this proposal could be interpreted as one specific instance of the general strategy suggested by 4. If we adopted 4, this could be the first of multiple schemes for encoding and interpreting debug info.

[lukewagner]

@dekimir You're absolutely right that, with 1, you're basically just adding names to assembly code (just like adding a symbols section to native binary), and 2 breaks down pretty quickly, so we can't stop there. That being said, for many quick-and-dirty tasks, 1 and 2 may be sufficient and their low(er) size overhead may make them attractive for some use cases.

For starters, how do we express arbitrary debug info in the text format?

I've been thinking as a black box: a .wasm would have some "user debug info" section that is handed to the debugger simply as a big ArrayBuffer.

And the browser API will have to be bidirectional...

Right, I've been thinking of these as two completely separate interfaces:

• For the devtools<-->debugger interface, I think most/all browsers already have some remote debugging message-based protocol which hopefully we could distill down into something minimal and browser-agnostic.
• For the debugger<-->debuggee interface, we have some experience in this direction already with the Debugger API which we use to implement all our FF debugging and I feel it could also be distilled down into something browser- (and wasm vs. JS!) agnostic. (On a side note, in my experience, the Debugger API has been fantastic (compared to the previous C debug API) b/c it allows us to write unit tests and fuzz directly in the engine, no need to involve the whole browser/devtools just to test debugging.)

Agreed that, given 4, your proposal could be fully implemented in "user space". (It'd probably be useful to have it as a leading prototypical use case, too, since it'd be much easier to get up and running than, say, full DWARF.)

[dekimir]

@lukewagner my question was how would .wast represent that big ArrayBuffer? :)

[lukewagner]

Oh, I see, sorry. So I guess it depends what you're doing. If you're writing the tool that generates the debug info, then I think we want something like what @yurydelendik asked for a while ago where you can tag various nodes in the .wast with a label and a .wast-to-.wasm tool would also spit out (somehow, separate file or new section) a tag-to-bytecode-offset map. Then you could use this with whatever debug info you extracted from LLVM or whatever tool you used to generate the initial .wast to produce the binary debug info section of the final .wasm.

There is the separate question of how to render a .wasm that contains a user-space debug info section; I don't think there's a lot we can do here since it's all opaque by definition; I'd suggest just some string encoding like we currently have for data segments. Particular toolchains with support for a particular debug format could render their debug info to a particular text format, I suppose, but this would be separate from the wasm-spec-defined text format.

[dekimir]

@lukewagner: sounds like we're talking about .wast not being fully equivalent to the corresponding .wasm? IOW, the front end could output a .wasm containing debug info in a dedicated section, but the corresponding .wast wouldn't have it?

And if the .wast contains a string encoding of the debug info, then don't the @ tags lose their purpose? Presumably the encoded info would remain valid even if the AST had no tags, right?

[lukewagner]

Right, if we have a "black box" debug-info section, I'm not sure there's a lot the .wast can do other than say "here are the bytes".

If we just did what I described above, then the @tags wouldn't be part of the official text language, they'd just be a tool to extract byte offsets from an assembler.

[jfbastien]

Full DWARF 4 seems pretty huge? Isn't that a Turing-complete language? :)
Would you have a specific DWARF feature to point at instead?

[dekimir]

These examples are just to jog one's imagination for where future evolution may take us. They're not meant to suggest that we intend to go in that direction, just that we don't want to gratuitously prevent it. Another example could be "enough of DWARF to allow stock gdb to run on wasm modules".

Anyway, see the discussion below -- we are moving to a different, more flexible design that allows any debug-info format to be used. I'll mothball this PR soon and create another one with the new design description.

[dekimir]

Here's another question that occurred to me: if the debug-info format is not fixed, and there are different debuggers out there (which I think is a fair assumption given the state of JS debugging today), how do we ensure that the user needs to compile their source only once? IOW, how do we avoid requiring something akin to -g=chrome_devtools and -g=firefox_devtools?

[lukewagner]

I think, in the general case, we want a wasm module to contain the URL of its debugger that runs portably on all the browsers using the two portable interfaces we described above. Maybe after some time there will be a de facto standard that browsers can build in debuggers for as a convenience/optimization, but I think we shouldn't start with that since it could lead to precisely the type of problems you're describing.

[dekimir]

Superseded by WebAssembly#708.