Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pkgsMusl.firefox: fix build #191060

Closed
wants to merge 250 commits into from
Closed

pkgsMusl.firefox: fix build #191060

wants to merge 250 commits into from

Conversation

yu-re-ka
Copy link
Contributor

@yu-re-ka yu-re-ka commented Sep 13, 2022 •
edited
Loading

Description of changes

Depends on #190796

I had some discussion in the musl irc channel about the malloc problem. There is two problems actually: lld crashes when built with the default stack size. But even when built with a larger stack size, it would crash when built with the musl mallocng allocator, because that allocator produces a very bad result for this allocation pattern and would exceed the default max_map_count value while linking firefox. Building musl with the oldalloc allocator works, but is very slow. Instead, I am LD_PRELOADing mimalloc here to work around this issue.

There is still an issue with dynamic linker path when running the resulting firefox. it is necessary to put /path-of-unwrapped-firefox/lib/firefox in the LD_LIBRARY_PATH manually, otherwise it will not start. i don't know why exactly. Fixed

Thunderbird builds and works as well.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.11 Release Notes (or backporting 22.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

Grumbel and others added 4 commits April 28, 2022 15:40
@Grumbel
guile: fix non-existing libunistring include path
2bd3cb6 
The include/ directory for libunistring is inside the libunistring.dev
package, not libunistring, thus the .pc file was pointing to a
non-existing path.
@SuperSandro2000
git: add git-jump to PATH
0ca0615
@r-ryantm
enchant: 2.3.2 -> 2.3.3
a5361b3
@r-ryantm
libaio: 0.3.112 -> 0.3.113
f2fffab
@Et7f3
cc-wrapper: remove duplicate C{C,XX}${role_post}
d937f6f 
This might be typo but I don't know which other variable was intended to be named. This can also come from a merge conflict.
@S-NA
Copy link
Contributor

S-NA commented Sep 14, 2022

There is still an issue with dynamic linker path when running the resulting firefox. it is necessary to put /path-of-unwrapped-firefox/lib/firefox in the LD_LIBRARY_PATH manually, otherwise it will not start. i don't know why exactly.

That seems oddly familiar to something I encountered awhile back when adding LTO support. It ended up that Mozilla had a Python script that tripped up on whitespace differences from GNU readelf and LLVM readelf which in turn caused /path-of-unwrapped-firefox/lib/firefox/dependentlibs.list to fail to be properly generated. I contributed a fix upstream awhile back for it so it is not the issue here but I suggest looking at the contents of dependentlibs.list and seeing if it has a list of shared objects. If it is missing items then something in the pkgsMusl environment is breaking an assumption of dependentlibs.py. Otherwise Firefox is failing to load dependentlibs.list.

@yu-re-ka yu-re-ka mentioned this pull request Sep 15, 2022
Merged
13 tasks
@yu-re-ka
Copy link
Contributor Author

The dependentlibs.list was generated correctly. It is the same between the glibc and musl build. An strace reveals very different behavior:

musl

[yuka@yaya:~/musl-nixos/firefox-104.0.2]$ rg mozsandbox log
81:read(3, "libmozsandbox.so\nliblgpllibs.so\n"..., 1024) = 89
82:open("/home/yuka/musl-nixos/outputs/out/lib/firefox/libmozsandbox.so", O_RDONLY|O_LARGEFILE) = 4
87:open("/home/yuka/musl-nixos/outputs/out/lib/firefox/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 4
1281:open("/nix/store/x5x7f006wbs09ww83pki17wli7qljaj2-icu4c-71.1/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1282:open("/nix/store/67i5mqcag7r35ydgh6g4yij0vg2z6xrd-libffi-3.4.2/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1283:open("/nix/store/0qyvydcbx9hfhmrzlfk7j7sfvyg2mwbv-nspr-4.34.1/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1284:open("/nix/store/vn1l1a1yxbwmp8nawc03p20vldla2r3f-zlib-1.2.12/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1285:open("/nix/store/7nirw85kh6zkjxlijjhwf2m9cd6iviiw-nss-3.82/lib64/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1286:open("/nix/store/sksgkzmxnj1q7ym2yzhhfr3f1rqyjji2-freetype-2.12.1/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1287:open("/nix/store/7ywq4vsqgl99p9vg06akay3yg1ri3hl0-fontconfig-2.14.0-lib/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1288:open("/nix/store/kzqrav33fgal77hv1rryby0k2pj0mph7-glib-2.72.3/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1289:open("/nix/store/zzhyvzgmxhhhx13gcspwwb2ffir6gmyi-gtk+3-3.24.34/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1290:open("/nix/store/k5bfc8lvj8ks9bil5m1hcpd4pjqfd4rs-atk-2.38.0/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1291:open("/nix/store/m1l9cwkxvpabklf29x021k7rxc3jqrfx-cairo-1.16.0/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1292:open("/nix/store/mi47j9c9ixhjbzn1zq86y9l0lg4z39xh-gdk-pixbuf-2.42.8/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1293:open("/nix/store/b3c2ikzmzcj76ixds4i55xpzwc0wx5rw-pango-1.50.8/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1294:open("/nix/store/nbf5i7m3l8n56j0c5h6pciygz5xcxgin-harfbuzz-5.1.0/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1295:open("/nix/store/kcmiv0h368pgz3fic5p311hj97s1vy1c-libpng-apng-1.6.37/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1296:open("/nix/store/dgk6wjwsq004n8rmkmaydpyz4ly8azmp-libwebp-1.2.4/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1297:open("/nix/store/sd70as1lqg1nvysing79lcp89fi02bii-libevent-2.1.12/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1298:open("/nix/store/5nk3wcm7q3zdqp9z7wfk108i15yhzcaa-libvpx-1.12.0/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1299:open("/nix/store/qlvc1ympanf4qalp2pwqfjs8wpx5nczh-pixman-0.38.4/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1300:open("/nix/store/z055cqq1sxl6bmzxhkg19lj0dwvlmpj3-dbus-1.14.0-lib/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1301:open("/nix/store/l8h95clnh22vvpjn9wb147jcq2w2q2yg-dbus-glib-0.112/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1302:open("/nix/store/2044lgh7qxpjkabb0i97y5clly5b3lpw-libXext-1.3.4/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1303:open("/nix/store/87k78lavc9ndkihjvvm8wrhqh9wl4jcs-libX11-1.7.2/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1304:open("/nix/store/yqx8yjbna5c9ndfq4bgkwd1w1qx77sp8-libxcb-1.14/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1305:open("/nix/store/8aspn3pkbj1a49hmqx81nkqxr2hq8sqr-libXcomposite-0.4.5/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1306:open("/nix/store/jxn57fm0jyf9qqnr2i7m1ckaw25ysgwh-libXfixes-6.0.0/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1307:open("/nix/store/bmmldybsss3d83w146lxib7sy6q1lyb0-libXcursor-1.2.0/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1308:open("/nix/store/k1x2rx52nbl9a67yrf7j9kk71gr5avbw-libXi-1.8/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1309:open("/nix/store/63wqbdk9zsb5gxpd05wgss8hcqrgrj87-libXrandr-1.5.2/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1310:open("/nix/store/lw0gsp8p4ws7mrhm4fxvybkbkxfjihns-libXdamage-1.1.5/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1311:open("/nix/store/via7achzpkbxh4pl4q0jmapp49r04bg4-libXtst-1.2.3/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1312:open("/nix/store/347fqzxbcm48ajcwhc80qyrppw2qldb4-gcc-11.3.0-lib/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1313:open("/nix/store/lbyphf5vcnsjv9n1kx2h4rvws3hz0m6g-libXrender-0.9.10/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1314:open("/nix/store/fjmgvqwwj52bvk4nb64j3diyb5fwm9bl-alsa-lib-1.2.7.2/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1315:open("/nix/store/40fiaz98s2dzgi0pp87zkvx8hhv2i6c2-musl-1.2.3/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1316:open("/nix/store/347fqzxbcm48ajcwhc80qyrppw2qldb4-gcc-11.3.0-lib/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1317:open("/nix/store/40fiaz98s2dzgi0pp87zkvx8hhv2i6c2-musl-1.2.3/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1319:open("/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1320:open("/usr/local/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1321:open("/usr/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1326:Error loading shared library libmozsandbox.so: No such file or directory (needed by /home/yuka/musl-nixos/outputs/out/lib/firefox/libxul.so)) = 142

[yuka@yaya:~/musl-nixos/firefox-104.0.2]$

glibc

[yuka@mari:~]$ rg mozsandbox log
148:read(3, "libmozsandbox.so\nliblgpllibs.so\n"..., 4096) = 89
149:openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/libmozsandbox.so", O_RDONLY) = 4
154:openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = 4
27593:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/glibc-hwcaps/x86-64-v3/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27595:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/glibc-hwcaps/x86-64-v2/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27597:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/tls/haswell/x86_64/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27599:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/tls/haswell/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27601:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/tls/x86_64/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27603:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/tls/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27605:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/haswell/x86_64/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27607:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/haswell/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27609:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/x86_64/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27611:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = 11
27932:[pid 17397] read(11, "libmozsandbox.so\nliblgpllibs.so\n"..., 4096) = 89

[yuka@mari:~]$

Note that glibc is only trying to open the libmozsandbox.so a second time in a child process

@github-actions github-actions bot added the 6.topic: stdenv Standard environment label Sep 21, 2022
@ofborg ofborg bot added the 8.has: package (new) This PR adds a new package label Sep 21, 2022
@github-actions github-actions bot removed 6.topic: stdenv Standard environment 6.topic: rust labels Sep 23, 2022
@github-actions github-actions bot added 6.topic: kernel The Linux kernel 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 6.topic: printing 6.topic: python 6.topic: qt/kde 6.topic: ruby 6.topic: systemd 6.topic: TeX Issues regarding texlive and TeX in general 8.has: module (update) This PR changes an existing module in `nixos/` labels Oct 31, 2022
@yu-re-ka
Copy link
Contributor Author

I fucked up the rebase, opening a new PR to not create any more notifications

@yu-re-ka yu-re-ka closed this Oct 31, 2022
@ofborg ofborg bot added 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild 10.rebuild-linux-stdenv This PR causes stdenv to rebuild 8.has: clean-up and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Oct 31, 2022
@ghost
Copy link

ghost commented Nov 1, 2022

I fucked up the rebase, opening a new PR to not create any more notifications

It's not your fault; github's UI for this is atrocious.

Hint: before changing the base, mark your PR as a draft. That way if this happens it won't ping half of humanity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
6.topic: kernel The Linux kernel 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 6.topic: printing 6.topic: python 6.topic: qt/kde 6.topic: ruby 6.topic: rust 6.topic: systemd 6.topic: TeX Issues regarding texlive and TeX in general 8.has: clean-up 8.has: module (update) This PR changes an existing module in `nixos/` 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 501+ 10.rebuild-darwin: 5001+ 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ 10.rebuild-linux-stdenv This PR causes stdenv to rebuild
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.