Source map improvements #1716

vouillon · 2024-10-18T10:18:26Z

I started working on improving the source map output of Wasm_of_ocaml, and got a bit side-tracked...

Many changes to preserve debug information from the bytecode as well as possible and put this information in the source map at appropriate places for the JavaScript debuggers.

compiler/lib/js_output.ml

compiler/lib/js_traverse.ml

compiler/lib/ocaml_compiler.ml

compiler/lib/specialize_js.ml

compiler/tests-compiler/effects_continuations.ml

toplevel/examples/lwt_toplevel/dune

compiler/lib/generate.ml

compiler/lib/parse_bytecode.ml

compiler/lib/js_output.ml

Explain a bit how they are used by the debuggers.

The debuggers do not stop on some statements, like function declarations. So there is no point in outputting some debug information there.

The best position is at the beginning of the expression after the `=`.

There are some locations which are marked as ghost locations, but which actually make sense.

compiler/lib/js_output.ml

Chrome will stop right after a return before returning from a function. This will yield to an ambiguity if there is a statement there. To prevent this, we add a space after a return statement when source maps are enabled.

The Chrome debugger uses this information to get the original names of variables. It does not look at all the identifier occurences but only at where they are bound. Also, we provide the original name, not the shortened name, which is not very useful.

- Do not repeat the output if the location has not changed - Start a new source map mapping before outputing the debug info as a comment (we need to start a mapping right after a return statement, moving it after a comment would not work) - Outputting an identifier name will not change the location (there is no point in doing that)

This is simpler than to associate a location to all instructions. Also, this avoid the issue of locations getting lost because an instruction gets optimized away.

In particular, no longer track the location of variables. This may make sense during code generation, though: the location of a statement should be the earliest location of the effectful expressions it contains (if there is any).

If the first block of the function starts with an event, we use the event's location. This is useful in case of tail-calls, where some code is generated before this first block. If we don't have an event, we set the initial location to unkown to prevent previous locations to bleed into the function body.

There is some code at the beginnning of exception handlers for which we have no debugging information.

Use a monad to abstract the operations on the expression queue and the effect information.

Statements will get the earliest location of the expressions performing a call it contains, if there is any. Then, for an expression `if (e) {...}`, we don't move first to the location of the conditional before going back to the location within `e` which might be strictly before. When queueing expressions, we ignore their locations if they don't perform a call; when flushed, we use the current location. This again prevent locations to be reordered.

The generated code is also hidden by using a dummy file.

Firefox assumes that a mapping stops at the end of a line, which is inconvenient. When this happens, we repeat the mapping on the next line.

Outputting this source map would fail on 32-bit architectures since its size, once Base64-encoded, would be over the string length limit of 16 MiB.

…tion

…ore robust

We ignore any event at the end of a block. This happens when the block ends with a return or a branch. In case of a return, it's a tail call, so we don't have an event anyway. For a branch, the target block will start with an event which is would take precedence anyway.

When pretty-printing, there is no ambiguity since we always have a space after the return statement. In compact mode, use a newline instead.

vouillon requested a review from hhugo October 18, 2024 10:18

vouillon force-pushed the source-maps branch 4 times, most recently from 2913f20 to 4e10e29 Compare October 18, 2024 14:30