-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to fix "Dynamic require of "os" is not supported" #1921
Comments
As far as I know, esbuild cannot convert external "static" commonjs require statements to static esm imports - not even for the side effect free node platform built in modules. This would be a useful feature for esbuild to have, as it's a fairly common use case for NodeJS applications and libraries. One could write an esbuild plugin for that, but a comprehensive solution would require reparsing each JS source file and the build would be considerably slower. |
Ok, I thought about writing one but I had the same reservations you had. But indeed that would be a super nice feature for node apps. |
I create a new issue #1927 and I think it may be related. |
Although this functionality would be better in esbuild itself, here's a super hacky script to transform node built-in $ cat importify-esbuild-output.js
var fs = require("fs");
var arg = process.argv[2];
var data = !arg || arg == "-" ? fs.readFileSync(0, "utf-8") : fs.readFileSync(arg, "utf-8");;
var rx = /\b__require\("(_http_agent|_http_client|_http_common|_http_incoming|_http_outgoing|_http_server|_stream_duplex|_stream_passthrough|_stream_readable|_stream_transform|_stream_wrap|_stream_writable|_tls_common|_tls_wrap|assert|async_hooks|buffer|child_process|cluster|console|constants|crypto|dgram|diagnostics_channel|dns|domain|events|fs|http|http2|https|inspector|module|net|os|path|perf_hooks|process|punycode|querystring|readline|repl|stream|string_decoder|sys|timers|tls|trace_events|tty|url|util|v8|vm|wasi|worker_threads|zlib)"\)/gm;
var modules = new Map;
var out = data.replace(rx, function(req, mod) {
var id = "__import_" + mod.toUpperCase();
modules.set(mod, id);
return id;
});
modules.forEach(function(val, key) {
console.log("import %s from %s;", val, JSON.stringify(key));
});
console.log("\n%s", out); Example input: $ cat 0.js
console.log(require("path").extname("foo/bar.hello"));
var os = require("os");
console.log(require("path").extname("ok.world")); Expected output:
Unmodified esbuild esm bundle output: $ cat 0.js | esbuild --bundle --platform=node --format=esm
var __require = /* @__PURE__ */ ((x) => typeof require !== "undefined" ? require : typeof Proxy !== "undefined" ? new Proxy(x, {
get: (a, b) => (typeof require !== "undefined" ? require : a)[b]
}) : x)(function(x) {
if (typeof require !== "undefined")
return require.apply(this, arguments);
throw new Error('Dynamic require of "' + x + '" is not supported');
});
// <stdin>
console.log(__require("path").extname("foo/bar.hello"));
var os = __require("os");
console.log(__require("path").extname("ok.world")); ...piped through $ cat 0.js | esbuild --bundle --platform=node --format=esm | node importify-esbuild-output.js
import __import_PATH from "path";
import __import_OS from "os";
var __require = /* @__PURE__ */ ((x) => typeof require !== "undefined" ? require : typeof Proxy !== "undefined" ? new Proxy(x, {
get: (a, b) => (typeof require !== "undefined" ? require : a)[b]
}) : x)(function(x) {
if (typeof require !== "undefined")
return require.apply(this, arguments);
throw new Error('Dynamic require of "' + x + '" is not supported');
});
// <stdin>
console.log(__import_PATH.extname("foo/bar.hello"));
var os = __import_OS;
console.log(__import_PATH.extname("ok.world")); ... and then piped through esbuild again to perform some DCE: $ cat 0.js | esbuild --bundle --platform=node --format=esm | node importify-esbuild-output.js | esbuild --bundle --platform=node --format=esm
// <stdin>
import __import_PATH from "path";
console.log(__import_PATH.extname("foo/bar.hello"));
console.log(__import_PATH.extname("ok.world")); to produce:
|
In addition to the workaround above: esbuild actually will print a warning when converting
Maybe we can use that info to write a plugin: https://gist.github.com/hyrious/7120a56c593937457c0811443563e017 One note: the modification actually turns a commonjs module into mixed module, however it seems esbuild can handle this case correctly: import xxx from 'xxx'
module.exports = xxx |
I'm getting this with For now, I've added This is all a bit over my head tbh, so apologies in advance if I'm not being clear 😅 |
|
@kzc in my case, it seems the trailing slash is being used to lookup a buffer polyfill for the browser so i am getting this error client-side when |
@jjenzz If you're seeing a bug you should probably open a separate issue for that with a reproducible test case, as it doesn't appear to be related to this issue. This issue is concerned with
Notice that The trailing backslash did not appear to matter for the browser platform if
|
@kzc thanks for all of that. I'll see if I can create a minimal repro of the issue I am experiencing and create a separate issue if so. |
Related: https://stackoverflow.com/questions/68423950/when-using-esbuild-with-external-react-i-get-dynamic-require-of-react-is-not-s?rq=1 It has no solution either |
To share some details about impact, it looks like this is:
|
FYI, I created a PR that addresses this issue for Node built-ins specifically: #2067 |
@eduardoboucas @jjenzz @kzc @enricoschaaf hey do you have any idea why this same
and the full error is Any ideas? |
This is breaking on AWS lambda with crypto module {
"errorType": "Error",
"errorMessage": "Dynamic require of \"crypto\" is not supported",
"trace": [
"Error: Dynamic require of \"crypto\" is not supported",
" at file:///var/task/dbStreamHandler.js:29:9",
" at ../node_modules/.pnpm/[email protected]/node_modules/uuid/dist/rng.js (file:///var/task/dbStreamHandler.js:10354:42)",
" at __require2 (file:///var/task/dbStreamHandler.js:44:50)",
" at ../node_modules/.pnpm/[email protected]/node_modules/uuid/dist/v1.js (file:///var/task/dbStreamHandler.js:10439:39)",
" at __require2 (file:///var/task/dbStreamHandler.js:44:50)",
" at ../node_modules/.pnpm/[email protected]/node_modules/uuid/dist/index.js (file:///var/task/dbStreamHandler.js:10822:37)",
" at __require2 (file:///var/task/dbStreamHandler.js:44:50)",
" at ../node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/middleware-retry/dist-cjs/StandardRetryStrategy.js (file:///var/task/dbStreamHandler.js:10930:18)",
" at __require2 (file:///var/task/dbStreamHandler.js:44:50)",
" at ../node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/middleware-retry/dist-cjs/AdaptiveRetryStrategy.js (file:///var/task/dbStreamHandler.js:11023:35)"
]
} any workarounds? |
@ShivamJoker we have a dependency that references
|
There were some weird issues with esbuild earlier resolving node_modules in cjs form incorrectly leading to require() statements inside the bundled index.js. Due to production issues, hacks were made to allow a deploy but those have been reverted and now this is an attempt to correctly fix the resolution issues. If this does not work, we should look at things like shimming require() as listed in evanw/esbuild#1921 (comment) and explained in: evanw/esbuild#3637 (comment)
There were some weird issues with esbuild earlier resolving node_modules in cjs form incorrectly leading to require() statements inside the bundled index.js. Due to production issues, hacks were made to allow a deploy but those have been reverted and now this is an attempt to correctly fix the resolution issues. If this does not work, we should look at things like shimming require() as listed in evanw/esbuild#1921 (comment) and explained in: evanw/esbuild#3637 (comment)
There were some weird issues with esbuild earlier resolving node_modules in cjs form incorrectly leading to require() statements inside the bundled index.js. Due to production issues, hacks were made to allow a deploy but those have been reverted and now this is an attempt to correctly fix the resolution issues. If this does not work, we should look at things like shimming require() as listed in evanw/esbuild#1921 (comment) and explained in: evanw/esbuild#3637 (comment)
There were some weird issues with esbuild earlier resolving node_modules in cjs form incorrectly leading to require() statements inside the bundled index.js. Due to production issues, hacks were made to allow a deploy but those have been reverted and now this is an attempt to correctly fix the resolution issues. If this does not work, we should look at things like shimming require() as listed in evanw/esbuild#1921 (comment) and explained in: evanw/esbuild#3637 (comment) This solution still uses external node_modules.
There were some weird issues with esbuild earlier resolving node_modules in cjs form incorrectly leading to require() statements inside the bundled index.js. Due to production issues, hacks were made to allow a deploy but those have been reverted and now this is an attempt to correctly fix the resolution issues. If this does not work, we should look at things like shimming require() as listed in evanw/esbuild#1921 (comment) and explained in: evanw/esbuild#3637 (comment) This solution still uses external node_modules.
@evanw do you have any idea what a solution to this would be? should there be one? we ran into this by simply bundling ( that seems like a very simple and common use case, for which esbuild currently produces invalid code (require in esm isn't valid) if you can let us know what the way forward for this is, maybe someone can contribute it. even if it turns out its to throw or something |
If you are bundling, presumably your target is the browser so
|
the target is probably the reason i haven't bumped into this yet is because most other projects i work on bundle for the browser but in this case we certainly are producing a node-only app (various reasons we are bundling, it is a valid use case) |
Hi @43081j have you tried the cjs shim solution? |
We did work around it using the banner in the end, as the inject workaround will mutate globalThis beyond the module so can affect other modules It seems like such a common use case though that esbuild really should handle this better or at least warn on build |
I tried every banner suggestion in here, and nothing works for me; I get this error:
Any help (please)? |
That is
|
There is no fix ex post facto. The fix is to write original source code that does not rely on Node.js internal modules. If that is not possible, e.g., you are using somebody elses code as a dependency, the only solution is to re-write the code by hand to not rely on Node.js internal modules. |
Post facto? |
The real fix is to not write code that depends on internal Post facto, the only fix is to hand re-write the source code. There is no programmatic way nor tool I am aware of that substitutes Web API's for Node.js-specific internal API's. |
No like, wdym post facto |
After the fact. Once a maintainer decides to write code using certain The given internal |
I'm still experiencing the issue today, thank you very much for the helpful snippets provided earlier. To resolve the issue with bundling in the AWS CDK NodejsFunction class, you can try the following configuration:
|
How can we use dependencies that still use require while shipping esm? Is that possible?
Minimal reproduction:
https://github.com/enricoschaaf/esbuild-issue
Just run node index.mjs. This was generated from running build which executes esbuild to bundle this file.
Problem:
The problem is that node internal modules can of course not bet bundled so I expected the require syntax would get written to import syntax if the target is esm but they are still require.
Error message:
Error: Dynamic require of "os" is not supported
at file:///home/enrico/work/esbuild-issue/index.mjs:1:450
at file:///home/enrico/work/esbuild-issue/index.mjs:1:906
at file:///home/enrico/work/esbuild-issue/index.mjs:1:530
at file:///home/enrico/work/esbuild-issue/index.mjs:1:950
at ModuleJob.run (node:internal/modules/esm/module_job:195:25)
at async Promise.all (index 0)
at async ESMLoader.import (node:internal/modules/esm/loader:337:24)
at async loadESM (node:internal/process/esm_loader:88:5)
at async handleMainPromise (node:internal/modules/run_main:65:12)
The text was updated successfully, but these errors were encountered: