Android backtraces don't work since gimli is used as the symbolizer #54
Comments
|
Hi, thanks for the report! |
|
https://github.com/inferrna/conrod-android-skeleton it's my tries to actualise this https://github.com/jbg/conrod-android-skeleton |
|
Hmm the missing backtrace symbolization seems to be a limitation of the backtrace library right now, but I will look a bit more around. Apparently, the rust compiler switched from libbacktrace to gimli this year and android support seems to be lacking right now: rust-lang/backtrace-rs#351 |
dvc94ch
changed the title
|
apparently there is support in backtrace-rs now. according to rust-lang/backtrace-rs#442 some changes to the AndroidManifest.xml are required. those should probably be added in debug builds. however I still can't get it to work <?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools"
package="com.myapp">
<application
tools:replace="android:extractNativeLibs"
android:extractNativeLibs="true" />
</manifest> |
|
Just to mention that I'm seeing the same issue and also tried setting I've tried with ndk 22b and 23b so far and I'm building for arm64-v8a. As far as I could tell the library had debug symbols (checked with |
|
I'm not so sure now that Actually in my case I'm seeing that even the offsets aren't valid - they are far to large - even if I use the legacy apk packaging to ensure the shared libraries are separate files. What I've realized I can do as a workaround though is to check for the backtrace that's automatically printed by Android itself because that at least has correct addresses / offsets. I then wrote a python script that just scrapes the offsets which can be piped to #!/usr/bin/python3
import sys
import re
for line in sys.stdin:
search = re.search('#[0-9]+ +pc +([0-9A-Fa-f]+) +', line)
if search != None:
print(search.group(1))So then I can copy paste a backtrace into a file and get a backtrace with filenames, demangled functions and line numbers via:
A notable advantage of this approach is that it doesn't require packaged / uploaded libraries to have symbols (so long as you can reference a copy of the library with symbols when symbolizing) and doesn't require the use of For example with this backtrace from adb:2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: #00 pc 0000000000021d94 /system/lib64/libc.so (abort+124) 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#1 pc 000000000199ad18 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#2 pc 0000000001997b1c /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#3 pc 00000000019979c0 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#4 pc 00000000019976e8 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#5 pc 0000000001995a88 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#6 pc 0000000001997420 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#7 pc 000000000014b604 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#8 pc 000000000014b6a8 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#9 pc 0000000000950224 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#10 pc 0000000000b69ec0 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#11 pc 0000000000a1f490 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#12 pc 0000000000b64a24 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#13 pc 0000000000b599f0 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#14 pc 0000000000b594d8 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#15 pc 0000000000b64774 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#16 pc 000000000099f968 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#17 pc 0000000001617d68 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#18 pc 000000000014fa14 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#19 pc 000000000014be74 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so (android_main+144) 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#20 pc 0000000001966740 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so (_rust_glue_entry+348) 2022-05-10 04:54:22.928 29166-29166/? A/DEBUG: rust-mobile/ndk#21 pc 0000000001976654 /data/app/co.realfit.agdkbevy-g7hRl_WmsH9ztNPfd-IkzA==/lib/arm64/libmain.so The above script will pull out:0000000000021d94 000000000199ad18 0000000001997b1c 00000000019979c0 00000000019976e8 0000000001995a88 0000000001997420 000000000014b604 000000000014b6a8 0000000000950224 0000000000b69ec0 0000000000a1f490 0000000000b64a24 0000000000b599f0 0000000000b594d8 0000000000b64774 000000000099f968 0000000001617d68 000000000014fa14 000000000014be74 0000000001966740 0000000001976654 Which can be piped to addr2line to get:?? ??:0 std::sys::unix::abort_internal::hf2e00ab422f03a90 /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c//library/std/src/sys/unix/mod.rs:259 rust_panic /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c//library/std/src/panicking.rs:748 std::panicking::rust_panic_with_hook::hcdc16bc1d6a90d9f /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c//library/std/src/panicking.rs:716 std::panicking::begin_panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::h80c5bec89bedcc17 /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c//library/std/src/panicking.rs:588 std::sys_common::backtrace::__rust_end_short_backtrace::hbdfac0e0f8306365 /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c//library/std/src/sys_common/backtrace.rs:138 rust_begin_unwind /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c//library/std/src/panicking.rs:584 core::panicking::panic_fmt::h28a757e2227ef488 /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c//library/core/src/panicking.rs:143 core::result::unwrap_failed::hc5bc744b126af212 /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c//library/core/src/result.rs:1749 core::result::Result$LT$T$C$E$GT$::unwrap::hdee8c853a0eb3fec /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/result.rs:1065 bevy_render::renderer::initialize_renderer::_$u7b$$u7b$closure$u7d$$u7d$::h0cd0ec9c415b7d46 C:\Users\Robert\src\bevy\crates\bevy_render\src\renderer/mod.rs:224 _$LT$core..future..from_generator..GenFuture$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$::poll::hceb4b9af4b98638b /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/future/mod.rs:84 futures_lite::future::block_on::_$u7b$$u7b$closure$u7d$$u7d$::hcea9b5092dbbef10 C:\Users\Robert\.cargo\registry\src\github.com-1ecc6299db9ec823\futures-lite-1.12.0\src/future.rs:89 std::thread::local::LocalKey$LT$T$GT$::try_with::h5f1204b6f3f53c68 /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/thread/local.rs:413 std::thread::local::LocalKey$LT$T$GT$::with::hce01bea0642e5b3f /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/thread/local.rs:389 futures_lite::future::block_on::hb3d920eab81bb4be C:\Users\Robert\.cargo\registry\src\github.com-1ecc6299db9ec823\futures-lite-1.12.0\src/future.rs:79 _$LT$bevy_render..RenderPlugin$u20$as$u20$bevy_app..plugin..Plugin$GT$::build::h6259e21f62b5446d C:\Users\Robert\src\bevy\crates\bevy_render\src/lib.rs:140 bevy_app::plugin_group::PluginGroupBuilder::finish::hc623480a2b4343d0 C:\Users\Robert\src\bevy\crates\bevy_app\src/plugin_group.rs:112 bevy_app::app::App::add_plugins::h860e64c350394fbd C:\Users\Robert\src\bevy\crates\bevy_app\src/app.rs:799 android_main C:\Users\Robert\src\agdk-rust\examples\agdk-bevy/src/lib.rs:38 _rust_glue_entry C:\Users\Robert\src\agdk-rust\game-activity\src/lib.rs:735 android_app_entry C:\Users\Robert\src\agdk-rust\game-activity/csrc/game-activity/native_app_glue/android_native_app_glue.c:208 |
|
Interesting how you managed to symbolize backtraces from logcat crash reports. I think that getting rust backtraces is a different issue. Many years ago before gimli became the default, backtraces did work, and they should work again. |
|
Yeah, at the very least it would be good if backtrace-rs could determine valid library offsets - ideally without requiring legacy apk packing. I guess I could live without the on-device symbolizing considering the implicit need to package libraries with symbols which in turn makes packaging and uploading slower for testing. Presumably for some versions of Android it can work with the extractNativeLibs workaround, or else maybe there has been a regression somewhere in backtrace-rs / gimli because for all the backtraces i've seen people posting for Android issues they all look to have bogus offsets. Maybe a small thing but I think it's more awkward to scrape the system-provided backtrace from adb than it would be to scrape a Rust backtrace in situations where you're filtering the adb log for a single process (just because the system backtrace gets dumped by another system process). |
|
Is there any solution? |
|
Having the same issue here. No filled traces for the Rust code. (Android 13, NDK 25, ARM64) |
Got this non-informative backtrace in logcat:
Then, while trying
cargo apk gdbgot this
Found in log
The app conrod_android_skeleton was set as "Debugging app" in settings (also same result was when it wasn't set).
So, it obviously is a problem on my side. But I'd like to have an option for attaching all debug symbols right into binary.
The text was updated successfully, but these errors were encountered: