build for arm64 failed #51
Comments
|
This isn't related to arm64 - this error is actually due to a bug in the latest release of portfinder: http-party/node-portfinder#138. That said, I'm surprised this is appearing for you here. This repo has a lockfile which still locks portfinder to 1.0.28 (which doesn't have this bug, and builds just fine): httptoolkit-server/package-lock.json Lines 21337 to 21345 in 3ce7fce That should stop portfinder updating unexpectedly like this. Any idea why that lockfile isn't being used in your case? Regardless, as a quick workaround if you switch to using that portfinder version then everything should work correctly. By the way, do let me know if you get this working, or if you run into any other issues. I don't have an arm64 test machine myself, but I'm definitely interested in supporting this in future, and I know others (like @shirshak55) are interested in this too. See httptoolkit/httptoolkit-desktop#45 for some more context. |
|
fixed as 1.0.32 |
this is odd |
upgrade portfinder version to 1.0.30,the above error has been resolved. but when I run "$ npm run build:release",it have new error.
make: 进入目录“/home/kylin/httptoolkit-server/node_modules/cpu-features/build”
make: 进入目录“/home/kylin/httptoolkit-server/node_modules/ssh2/lib/protocol/crypto/build”
info sentry-cli Using cached binary: /home/kylin/.npm/sentry-cli/a85cf2-sentry-cli-Linux-aarch64
prebuild-install WARN install No prebuilt binaries found (target=3 runtime=napi arch=arm64 libc= platform=linux)
Installing override npm dependencies... added 1325 packages from 903 contributors and audited 1346 packages in 236.89s 64 packages are looking for funding found 9 vulnerabilities (2 low, 3 moderate, 4 high)
wrote manifest to /home/kylin/httptoolkit-server/oclif.manifest.json
wrote manifest to /home/kylin/httptoolkit-server/oclif.manifest.json WARNING in ./node_modules/express/lib/view.js 81:13-25 WARNING in ./node_modules/node-fetch/lib/index.mjs WARNING in ./node_modules/ssh2/lib/protocol/crypto/build/Release/sshcrypto.node 1:0 ERROR in ./node_modules/cpu-features/build/Release/cpufeatures.node 1:0 ERROR in ./node_modules/dns2/client/doh.js 26:36
npm ERR! A complete log of this run can be found in: |
|
Interesting... So, it looks like with this, installation and the basic build works OK, but building a release fails. That's a great start though. If you run If it does, and then you open The error shown where when bundling the release code with webpack is related to the httptoolkit-server/webpack.config.js Line 61 in e107327 That will exclude it from release bundling entirely, so it generally won't be available at runtime, but that's fine because we don't actually need it (and most builds don't include it anyway!). If that works for you, I'd happily accept a quick PR for this, I think we basically never want to include |
|
when i run kylin@kylin-v10sp1-arm64-2107:~/httptoolkit-server$ npm run start
Config checked in 30 ms I have add cpu-features to webpack.config.js One of the bugs has been resolved, the other one still exists kylin@kylin-v10sp1-arm64-2107:~/httptoolkit-server$ npm run build:release
wrote manifest to /home/kylin/httptoolkit-server/oclif.manifest.json WARNING in ./node_modules/express/lib/view.js 81:13-25 WARNING in ./node_modules/node-fetch/lib/index.mjs WARNING in ./node_modules/ssh2/lib/protocol/crypto/build/Release/sshcrypto.node 1:0 ERROR in ./node_modules/dns2/client/doh.js 26:36
|
That looks very promising! Suggests that actually everything at runtime is set up correctly & working OK.
Right, yes, so the error here is due to failing to parse is present in the latest version 2.0.5 but the package lock says (here) that you should be installing version 2.0.1, which doesn't include that This is relatively new JS syntax, though I'm not sure specifically why webpack doesn't handle it. That is almost certainly fixable, but I think it would be better to work out why you're not using the versions in the lockfile. In theory you can use any semver-compatible version for all dependencies if you like, but it's very likely that you'll run into awkward untested configurations with issues like this if you're not using the versions from the lockfile. If we avoid that, then we can focus on ARM-specific issues. Which versions of node & npm are you using? What happens if you run |
|
Thank you very much for your support, it can be packaged normally now This is the compiled tar package |
|
Now, I want to know how to use local httptoolkit-desktop tarball to package httptoolkit-desktop. I replaced /opt/HTTP Toolkit/resources/httptoolkit-server in the httptoolkit_1.10.0_arm64.deb on the x86 architecture with the content of the httptoolkit-server I compiled on the arm architecture, and now it can run normally |
|
Yes, so the normal httptoolkit-desktop start & build process expects to pull a server from an official build. If you want to create a full desktop build without that, you can do so by manually extracting the build yourself into If you do so, then run The only check it does is that a server exists and the version matches, i.e. the |
|
@pimterry sorry was preoccupied with something. I ran npm install on this repo and I immediately get the following error. I don't know which package is requiring this dependency. I have m1 laptop, so I can help here with no worries. While we are on it, we can also switch other dependencies like request with the popular got package. But, that requires us to take radicle steps like changing npm project to esm only. |
Ah, ok, this is coming from node-datachannel, which is used in MockRTC, which powers some new WebRTC interception features. They're actually working on M1 prebuilds to fix this, and but the main problem is that they're struggling to find somebody with an M1 to test the result. Could you help? The issue is here: murat-dogan/node-datachannel#120. I'm not sure exactly what the test process should be for this, but I think you probably need to:
If you run into any issues or you have any questions, ping in that issue and I'm sure they'll be very happy to help.
No need I think - we don't actually use |
|
@pimterry where should I put node_datachannel.node file? Like I have node_modules folder and thousands of folders inside it. Where should I go from here. |
|
(Replied on the other issue) |
|
@shirshak55 now that that's fixed, can you try the server again here on your machine? I've just updated that subdependency to the new version with the prebuild for ARM64, so in theory if you pull that then everything should work. Maybe... |
|
@pimterry hey, sorry for being late. I think it works. |
|
Superb! Thanks @shirshak55. I think we can close this issue now then, looks like low-level ARM64 support is all good in the server. That should help for people running with the server+browser setup manually in the short-term. Full native support in the main HTTP Toolkit app is still going to be a work in progress for a while, since we'll need to start publishing server releases for ARM first, to fix httptoolkit/httptoolkit-desktop#45, and then I think we'd need a way to build the desktop app for ARM within a GitHub Actions runner, which as far as I can tell isn't currently possible (but then might be some workarounds, or I wouldn't be surprised if GitHub added ARM runners in future which would make this much easier). |
|
@pimterry what I can do is provide the HTTP toolkit server app manually. But yea there is going to be trust issues I guess. Or, I think you can purchase a mac mini and use it to publish the server which shouldn't cost that much for now? |
We could do manual publishing to work around these limitations, you're totally right, but more than trust it's about the practical hassle of that. We run a fresh build for every single commit, plus a new deploy every few weeks or so. It's really nice that the app is built immediately for every target, automatically tested after every commit, and the release process is totally scripted so it's definitely exactly the same every time. That makes everything very reliable, and switching to a manual approach would have a lot of downsides for not much upside. Fortunately, this is just a problem for the main desktop app. For the server we can cross-build for ARM64 from the normal Linux CI environment, because it's almost all JS, with just a couple of node modules that are (now) all prebuilt for ARM so just get downloaded. That's means no compilation involved, just downloading the correct prebuilds for each target. We just need to configure the build process to build these new ARM targets, and then that'll be included in the next release. I've just done a test run for that here: https://github.com/httptoolkit/httptoolkit-server/actions/runs/2920847127. If you have a minute, could can you download & extract the The desktop is harder I think, because I'm not sure Electron-Builder will happily build for the wrong architecture like this. We currently run a separate build on a different OS to generate each Electron app version. To be honest I've never tried though, there might be some good options there, once the server part is sorted out. GitHub doesn't yet have hosted runners for Mac M1 (actions/runner-images#2187) but they did announce support a few weeks ago for self-hosting an M1 GitHub runner, so if it comes down to it I might buy that M1 mini after all... 😄 |
I want build in arm64.but, when i run "npm run prepack",There are the following errors:
$ node -v
v14.17.1
kylin@kylin-v10sp1-arm64-2107:~/httptoolkit-server$ uname -a
Linux kylin-v10sp1-arm64-2107 5.4.18-35-generic #21-KYLINOS SMP Tue Jul 20 13:31:32 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
kylin@kylin-v10sp1-arm64-2107:~/httptoolkit-server$ npm run prepack
node_modules/portfinder/lib/portfinder.d.ts:39:1 - error TS1128: Declaration or statement expected.
39 export setBasePort(port: number);
node_modules/portfinder/lib/portfinder.d.ts:49:27 - error TS1005: ',' expected.
49 export setHighestPort(port: number);
~
Found 4 errors.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] build:src:
rm -rf lib && tsc -b --forcenpm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] build:src script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /home/kylin/.npm/_logs/2022-08-12T05_50_15_861Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] prepack:
npm run build:src && oclif-dev manifestnpm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] prepack script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /home/kylin/.npm/_logs/2022-08-12T05_50_15_891Z-debug.log
The text was updated successfully, but these errors were encountered: