Repository navigation
feat: add Scopes of functions and inlined calls to the source map - #344
JesusFileto wants to merge 19 commits into
Conversation
Tools that show Wasm stack traces read source maps, not DWARF. walrus already rewrites DWARF to the emitted code's offsets, so the map's mappings can come from those line tables without a second parse. ModuleConfig::generate_source_map(true) turns it on, and Module::emit_wasm_with_source_map(url) returns the module and its map, with a sourceMappingURL section that points at url in place of any the module had.
Optimized Rust inlines most calls, so with only mappings a stack frame loses the calls inlined into it. Scopes, the ECMA-426 proposal that Chrome DevTools reads, describe the source functions and the calls inlined at each offset, so a reader can expand a frame into them. They come from the debugging information entries walrus already rewrites: a Module scope per source around a Function scope per source function, and a range per function and per inlined call, with its call site. Naming functions, and telling which ones are copies of one source function, depends on the language, so the Rust rules (demangling with rustc-demangle, dropping generic arguments and shims, the shared `<_ as T>::f` name) live in rust_names.rs behind an internal trait. Scopes make the map about four to six times larger, so ModuleConfig::source_map_scopes(false) leaves them out. They are on by default.
There was a problem hiding this comment.
This PR adds ECMA-426 Scopes to the source map that #343 builds from the rewritten DWARF. Each source gets a Module scope holding a Function scope per source function, and each module function and each inlined call gets a generated range with its call site, so tools like Chrome DevTools can rebuild the inlined Rust frames in a stack trace. My guess is that wasm-bindgen will use this for symbolicated stack traces. Scopes are on by default, and ModuleConfig::source_map_scopes(false) turns them off. Function naming is language-specific, so the Rust rules sit behind an internal trait meant to become a hook later. I checked the encoding against the current proposal-scopes branch of the spec: per-source scopes with null entries, segmented ranges, per-string accumulator resets, the flag bits, and the E/I order and frame rules for inlined ranges. All of that matches. Part of that branch's text was cut off when I fetched it, so I couldn't confirm the RangeDefinition field encoding. One problem remains: with legacy mangling, <impl Trait for Type> path segments are stripped as if they were turbofish generics, which gives misleading scope names (inline comment).
Not merge-ready:
- Legacy-mangled
<impl Trait for Type>::fnames are reduced to a misleadingmodule::f, so inlined frames from trait impls on primitives and other legacy-mangled code get wrong names.
Directory 0 of a line program is the unit's compilation directory, and the path already starts from it, so a relative one (as left by `--remap-path-prefix`) came out twice: `myproj/myproj/src/lib.rs`. Skip directory 0, as addr2line does.
Directory 0 of a line program is the unit's compilation directory, and the path already starts from it, so a relative one (as left by `--remap-path-prefix`) came out twice: `myproj/myproj/src/lib.rs`. Skip directory 0, as addr2line does.
Legacy demangling writes trait impls as `core::fmt::num::imp::<impl core::fmt::Display for u32>::fmt`. The `<` after `::` was stripped like a turbofish, leaving `imp::fmt`, a free function that doesn't exist, which every impl from that macro merged under. Keep `<impl ...>` segments without the generics inside them, and name copies `<impl core::fmt::Display for _>::fmt`, as v0 names get `<_ as core::fmt::Display>::fmt`.
Declared scopes that start at the same place merge when their names
end with the same segment, so that copies whose self types differ are
one scope. For closures that segment is `{closure#0}`, so an
attribute macro's functions merged: the body of an `async fn fetch`
(`fetch::{closure#0}`), its exported wrapper's closure and its own
inner closure all became `_::{closure#0}`, the main frame of a Worker.
Compare the path from the last named segment instead, so
`fetch::{closure#0}` and `__wasm_bindgen_generated_fetch::{closure#0}`
stay apart.
# Conflicts: # src/module/debug/mod.rs
The map took its rows by writing the rewritten DWARF and parsing the line programs back, so generate_source_map alone converted and wrote all of .debug_info only to discard it. Split convert_line_program into read_sequences, walk_lines (which reports each output function's rows as LineEvents) and the writer, and make the source map a second consumer of those events. With generate_dwarf the conversion hands each unit's lines to the map; without it, dwarf::lines reads and walks only the line programs, so no entries are converted and nothing is serialized. The written DWARF and the maps don't change.
# Conflicts: # CHANGELOG.md
guybedford
left a comment
There was a problem hiding this comment.
Thanks for this. The encoding looks right: I checked it against the decode algorithms in the proposal-scopes branch, and ran it on optimized Rust modules (v0 names on 1.100, legacy names on 1.91) and a small clang++ module, decoding each map with a separate decoder. Every map decoded with its nesting, ordering and call-site constraints intact, and nothing panicked with overflow checks on.
My comments are all about names, where the same-position merge takes in more than copies of one function. The one I'd most like an answer on is which name a reader shows for a function's outermost frame.
This also needs a rebase once #343 lands (ba5f1de here is #343's fdc9f21), and the description's "only the top commit is new" is out of date.
| /// differ become `_`. A function's outermost frame still gets its | ||
| /// instance's own name from the name section. |
There was a problem hiding this comment.
Is this what readers do? My understanding is that DevTools names a frame from its range's definition whenever it has one, including a function's outermost range, but I haven't checked that in DevTools.
If so, real frames lose their self type. Two Display impls generated by one macro_rules! both become <_ as core::fmt::Display>::fmt, where the map without Scopes leaves them as <krate::A as core::fmt::Display>::fmt and <krate::B as core::fmt::Display>::fmt from the name section.
Could you confirm what DevTools shows for a top-level frame whose scope name was merged?
There was a problem hiding this comment.
Yea your right and I removed the comment. I also changed the implementation for scopes to only merge copies of the same impls but keep the name sections for macro's seperate impls. To do this I had to nest the scopes together, so an upstream change to DevTools would be needed. However, this is valid under the spec and is just a limitation of DevTools at the moment.
There was a problem hiding this comment.
Thanks for the quick turnaround. The narrower merge and the <_>::f and nested-path names are what I was after, and the per-impl scopes do make the map itself exact.
I sent you the wrong way on this one though. I asked the question before reading how DevTools names frames, and it matters. In SourceMapScopesInfo.ts the top frame is named from its range's definition, but each caller of an inlined call is named by looking its call site's position up in the original scopes. With the impls of one macro nested on the same lines, that lookup returns the innermost one, so a caller can be shown under a sibling impl's name.
On my test module that is 157 of 4,743 inlined call sites, where the previous revision had 1: <u64>::carrying_mul shown for <u32>::carrying_mul, IterMut::next for Iter::next, and in the crate itself <krate::A as Display>::fmt for B's.
DevTools as it ships is the main reader for these maps, and there a vague name is better than a wrong one. So could you go back to your original shape for this case: same-place scopes whose names differ only in their self type share one scope, <_ as Display>::fmt or <_>::checked_mul, whatever their {impl#N}? Everything else from this round stays. The per-impl scopes would be worth bringing back if DevTools comes to name callers from the parent range's definition.
The Scopes took the functions and inlined calls from the rewritten .debug_info, so with Scopes on, generate_source_map converted and wrote all of the DWARF even without generate_dwarf, only to read it back. The only thing they take from the rewrite is each entry's code, which is CodeMap::ranges of its input ranges, as the rewritten DW_AT_low_pc, DW_AT_high_pc and range lists have it. So walk each input unit's entries along with its lines, map their ranges through CodeMap::ranges, and find their files through the unit's converted line program. Without generate_dwarf the DWARF is no longer converted or written. The map is the same apart from the order of its sources, as each unit's entries now add their files with its lines.
Declared scopes that start at the same place and end in the same named segment were merged whatever the rest of their names, so different functions that one macro generates became one scope: <u64 as LowerHex>::fmt and <u64 as UpperHex>::fmt became _::fmt, and so did <*const _>::is_null::runtime and do_panic::runtime from const_eval_select!. Merge them only when their names differ in just the self type, and leave the others as separate scopes, which nest nests or cuts as it does for the closures of different functions.
The self type ended at the first " as ", which is inside it when it is a qualified path itself, so <<A as X>::Out as T>::f and <<B as X>::Out as T>::f shared <_ as X>::Out as T>::f, with an unbalanced "<". End it at the " as " outside any brackets.
v0 names write inherent impls as <u32>::checked_mul, which had no " as " to find, so copies for different types were left apart, after falling back to _::checked_mul before. Give them <_>::checked_mul, as <_ as T>::f for trait impls and <impl _> for legacy names.
A DW_TAG_namespace without a DW_AT_name is pushed as an empty name, so a function in a C++ anonymous namespace was named ::helper. Name it (anonymous namespace)::helper, as llvm-cxxfilt writes it.
The closure that hands each unit's lines to the source map checked for the map again, although it only runs when there is one. Make it from `map.as_mut()` instead, and pass the resulting `Option<&mut LineSink>` to both `dwarf::convert` and `dwarf::lines`, naming the type rather than casting with `as _`.
The names described each test's assertion in full. Name them after what they cover instead, closer to the rest of the crate's tests.
Declared scopes at one place were merged when their names differed in
just the self type, so the separate impls that one macro generates
became one scope: impl Display for A and for B from one macro_rules!
became <_ as core::fmt::Display>::fmt, and <u32>::checked_mul and
<usize>::checked_mul from uint_impl! became <_>::checked_mul. DevTools
names every frame from the scopes, so their frames lost which impl
they are.
rustc puts the functions of a trait impl, or of an inherent impl of a
primitive, in a {impl#N} namespace, so the qualified DW_AT_name
without generic arguments is the same for the copies of one impl or
function, and different for separate impls. Merge scopes only when it
is the same. Copies of one generic impl, such as
<[u8] as PartialEq>::eq and <[u16] as PartialEq>::eq, still share
<_ as core::cmp::PartialEq>::eq, while separate impls, such as the
ones above, keep their own scopes, which nest nests as they cover the
same source. Separate impls of one generic type, such as impl Encode
for Vec<A> and for Vec<B>, get the same name, as names drop generic
arguments. Impls in separate const _ blocks still merge, since only
their linkage names tell them apart, and so do a macro's inherent
impls on different types with full debuginfo, whose methods DWARF puts
in the type rather than in a namespace. Functions without a DW_AT_name
show nothing of where they come from, so their scopes at one place
stay apart. The definition replaces the declaration key, which told
closures of different functions apart, as their paths already do.
Brings in the line sink built from `map.as_mut()`. The conflict in debug/mod.rs keeps this branch's naming and unit arguments, inside the closure built from `map.as_mut()`.
feat: source maps that let stack traces show inlined Rust calls
Stacked on #343; only the top commit is new here.
Optimized Rust inlines most calls, so with only mappings the calls inlined into a Wasm frame are lost. Scopes (the ECMA-426 proposal that Chrome DevTools reads) describe the source functions and the calls inlined at each offset, so a reader can expand a frame into those calls. They come from the debugging information entries that walrus already rewrites at emit.
The flag.
ModuleConfig::source_map_scopes(false)gives the map without Scopes. They are on by default, so maps grow: on a Rust module with 13,400 functions (6.1 MB of Wasm), the map goes from 2.4 MB to 9.5 MB (0.70 to 1.70 MB gzipped), and emit from 0.95 s to 1.65 s.The map. Adds
scopesandranges, with theirnames: aModulescope per source around aFunctionscope per source function, a stack-frame range per function, and one per inlined call, nested in its caller's, with its call site.sourcesalso lists the files that only scopes and call sites name.Names. The generator reads DWARF only. Rust naming lives in
src/module/debug/rust_names.rs, behind an internal trait, and demangles linkage names with the newrustc-demangledependency. Names drop generic arguments and shims. Only copies of one source function share a scope:_for the self type:<[u8] as PartialEq>::eqand<[u16] as PartialEq>::eqgive<_ as core::cmp::PartialEq>::eq, and<*mut u8>::addand<*mut u16>::addgive<_>::add.impl Display for Aandfor B, oruint_impl!'s<u32>::checked_muland<usize>::checked_mul.DW_AT_nametells them apart, because rustc puts each trait impl, and each inherent impl of a primitive, in its own{impl#N}namespace. Separate impls of one generic type, such asimpl Encode for Vec<A>andfor Vec<B>, get separate scopes with the same name, since names drop generic arguments.const _: () = { … }blocks, and, with full debuginfo, a macro's inherent impls on different types. Only their linkage names tell them apart, which a later change could use.The plan is to expose the trait later as a hook, with these rules as the default, so other languages can supply names. Unreadable DWARF panics with
expect, like the conversion.