Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node process on Windows 11 consumes memory continuously, increasing about 3mb a second without being released #3976

Open
scotscoder opened this issue Jan 1, 2025 · 16 comments
Labels
Bug Something isn't working

Comments

@scotscoder
Copy link

Description

When running wails dev, the node process gradually consumes more and more memory, slowing down the host system. As observed, it increases by about 2.5 to 6mb per second on my system. GIF attached.

This happens with a completely new Svelte project. I will test it with other JS frameworks as well and update this issue.

Screen.Recording.2025-01-01.133325.mp4

To Reproduce

Use Windows 11.

  1. Init a new wails project using Svelte
  2. Run wails dev
  3. Watch the memory increase in Task Manager

Expected behaviour

Memory use should stay constant or "breathe" slightly up and down.

Screenshots

Screencap attached above

Attempted Fixes

Found a closed bug on the SvelteKit repo that mentioned a similar problem. It may be upstream from even that, but I'd rather check before we dive deeper.

System Details

# Wails
Version | v2.9.2

# System
┌─────────────────────────────────────────────────────────────────────────────────────────────────┐
| OS           | Windows 10 Pro                                                                   |
| Version      | 2009 (Build: 26100)                                                              |
| ID           | 24H2                                                                             |
| Go Version   | go1.23.4                                                                         |
| Platform     | windows                                                                          |
| Architecture | amd64                                                                            |
| CPU          | AMD Ryzen 7 5800H with Radeon Graphics                                           |
| GPU 1        | NVIDIA GeForce RTX 3060 Laptop GPU (NVIDIA) - Driver: 32.0.15.5613               |
| GPU 2        | AMD Radeon(TM) Graphics (Advanced Micro Devices, Inc.) - Driver: 31.0.12002.1002 |
| Memory       | 16GB                                                                             |
└─────────────────────────────────────────────────────────────────────────────────────────────────┘

# Dependencies
┌────────────────────────────────────────────────────────┐
| Dependency | Package Name | Status    | Version        |
| WebView2   | N/A          | Installed | 131.0.2903.112 |
| Nodejs     | N/A          | Installed | 22.12.0        |
| npm        | N/A          | Installed | 10.9.0         |
| *upx       | N/A          | Installed | upx 4.2.4      |
| *nsis      | N/A          | Installed | v3.10          |
└─────────────── * - Optional Dependency ────────────────┘

# Diagnosis
 SUCCESS  Your system is ready for Wails development!

Additional context

No response

@scotscoder scotscoder added the Bug Something isn't working label Jan 1, 2025
@scotscoder
Copy link
Author

I only noticed this issue because it started increasing my laptop fan when it reached 5GB of RAM consumed.

@scotscoder
Copy link
Author

This is after 30 minutes.

Screen.Recording.2025-01-01.142119_stripped.mp4

@scotscoder
Copy link
Author

It does seem to partially garbage collect periodically, just not very much.

@leaanthony
Copy link
Member

Sounds like an issue with Vite. Have you tried upgrading it?

@scotscoder
Copy link
Author

How do I upgrade Vite?

@imthatwolf
Copy link

imthatwolf commented Jan 12, 2025

@scotscoder I also had this issue with vite using 25-32gb of ram out of 128gb, managed to fix it by setting this in vite.config.ts/js:

Edit:

Initially I thought this fixed the issue I was partially mistaken, it wasn't the setting at all it was a hot reload of the vite.config.ts/js that stops the cpu and ram usage, no other file edits seem to do the same only on the vite config.

vite.config.ts/js
import { sveltekit } from '@sveltejs/kit/vite';
import { defineConfig } from 'vite';

export default defineConfig({
  server:{
    hmr:{
      overlay: false
    }
  },
  plugins: [sveltekit()]
});

ref: vite.dev hmr docs

Its now only using around 600mb, even after 10mins and with updates.

Package versions:
"vite": "^6.0.7"
"svelte": "^5.17.3",
"@sveltejs/kit": "^2.15.2",
"@sveltejs/adapter-static": "^3.0.8",
"@sveltejs/vite-plugin-svelte": "^5.0.3",

To update your vite to the latest you will need to check your package manager documentation as npm/yarn/pnpm have slightly different commands for updating.

@imthatwolf
Copy link

imthatwolf commented Jan 14, 2025

After a lot of testing I have narrowed it down to wails causing the issue in nodejs, best guess is the ipc or runtime injection. Tested on vanilla template and custom sveltekit frontend with skeleton.

wails dev
node.exe = CPU ~10%, RAM ~250mb growing ~3-6mb a sec.

pnpm dev (in frontend)
node.exe = CPU ~0%, RAM ~180mb doesn't grow

If you hot reload the vite config when using wails dev, the CPU% drops and RAM stops increasing does not decrease though. Inspecting the node process in devtools shows the vm only being ~100mb or less in size, which is vastly different from the task manager view.

Using x64dbg attached to node I observed alot of :
return to ntdll.RtlAllocateHeap+C24 from ntdll.RtlAllocateHeap+11F0
and
return to node.inflateValidate+675F from node.inflateValidate+2397C

If I attached to the build go exe and pause the node process still keeps increasing,

Edit: Doesn't cause this issue in wails3

@leaanthony
Copy link
Member

Thanks for taking the time to deep dive this. The only difference that I can think of between a dev and prod build is Go flags so I'm more convinced it's a vite HMR issue. Someone on this thread mentioned it stopped when turning off an error option in Vite stopped it. If your project is a simple html page what does it do?

@BroNils
Copy link

BroNils commented Jan 20, 2025

This issue still happens even after i upgrade [email protected] and setup vite to disable hmr with { hmr: false } & { hmr: {overlay: false} }.

I have an ugly fix for this:

  1. Start your wails dev command and wait for your app window to show.
  2. Kill the Node.js process with the highest memory consumption from Task Manager.
  3. Start your frontend vite dev command from another command prompt.
  4. Done.

Alternatively, you can simply start your wails dev command with procgov: procgov -m 2G --recursive -- <your_wails_dev_cmd>. By using procgov, the Node process will kill itself when it exceeds the maximum memory limit (2G). After the process is killed, do step number 3.

If you encounter this error, just ignore it

Image

This method seems to be working for me (the memory usage of the Node process isn't too high, <500MB). I don't know why this works, but here’s a before and after.

Before

Image

After

Image

@scotscoder
Copy link
Author

Digging into this a little deeper, there are a lot of complaints about Vite using up excessive ram in development mode on Stack Overflow,

It also is being tracked by the Vite developers here, though they have locked the issue and it has been ongoing for nearly 4 years:
vitejs/vite#2433

They have recommended workarounds listed here, for Rollup:
https://rollupjs.org/troubleshooting/#error-javascript-heap-out-of-memory

Pretty obvious this is an upstream issue and nothing to do with Wails, but it still has a profound effect on any development. Would there be the possibility of switching from Vite to some other solution?

@scotscoder
Copy link
Author

It looks like Vite are working on an alternative bundler to Rollup called Rolldown rather than eliminating the problems with Rollup.

https://rolldown.rs/

@imthatwolf
Copy link

imthatwolf commented Jan 23, 2025

Unfortunately I must disagree, that issue (heap out of memory) is unrelated to this problem by testing and debugging further using a vanilla project.

If I run the frontend standalone with just "pnpm dev" the ram doesn't increase and it happily sits at 35mb and ~0% CPU.
If I run using "wails dev" node starts to consume more and more ram and is constantly using ~10% CPU.

So something wails is doing is triggering it to happen, using Process Monitor I managed to see that Vite when run using Wails gets stuck in a loop going over and over the "wailsjs" directory, it continually does this:

Image

When run standalone this doesn't happen.

If I use the js debug console in vscode I found Vite was looping with chokidar over those files in the wailsjs directory. Still doesn't explain why it only triggers when run with Wails. My initial thought was maybe the 2 file watchers where conflicting but its not that after testing fsnotify in a simple program and then running Vite.

When debugging Wails I found the exact point at which it starts to happen its the last time "wailsbindings.exe" gets run in bindings.go at line 86.

Now I don't know why generating the bindings triggers this to happen its a bit of a pain to debug, I stated it was the last time but it does happen before when it does the initial build its just the process is closed and then reopened again.

Things that stop it from happening:

  • Reload vite.config
  • add '/wailsjs/' to the server.watch.ignored in vite.config
  • run wails dev -skipbindings

It also like I said before doesn't cause an issue in wails3alpha, so for now I will use that instead. Curious as to why "wailsbindings.exe" triggers this to happen I'm unsure. And I knows it's a bit of a grey area as it isn't wails that's consuming the memory and cpu but vite but seen as wails causes it to happen where it wouldn't otherwise possibly there is a fix in wails. It would also be nice to know exact cause, but maybe that's just me.

@leaanthony
Copy link
Member

So I think what's happening is vite's filewatcher isn't cleaning up old file references in its watcher. When new binding are generated, it must be re-reading them or checksumming or something and not discarding the old ones.

@imthatwolf
Copy link

imthatwolf commented Jan 25, 2025

Yes so finally managed to find out, wails is removing and making both the "go" and "runtime" folders too quickly in app_binding.go:104-116, This causes chokidar to get stuck in a renaming loop event, I wrote a basic watcher with chokidar (v3 & v4) and a simple make and remove directory program in go and issue also occurs:

Raw event info: {
  event: 'rename',
  path: '\\\\?\\C:\\Users\\user\\projects\\vanilla\\frontend\\wailsjs\\go',
  details: {
    watchedPath: 'C:\\Users\\user\\projects\\vanilla\\frontend\\wailsjs\\go'
  }
} ...

This is why when debugging the issue doesn't occur as stepping through is slow so issue never occurs, if you add a 1 second sleep between the remove and make it stops the issue, or like I mentions before if you stop the binding generation or don't watch the wailsjs directory.

So its not really wails problem its chokidar but could be mitigated in wails as vite is a used in the templates and is quite common, vite did upgrade chokidar to v4 but rolled back to v3, either way I still had the issue which ever version I used in a simple test.

@leaanthony
Copy link
Member

Yes so finally managed to find out, wails is removing and making both the "go" and "runtime" folders too quickly in app_binding.go:104-116,

There's a Go vs JS statement in there somewhere 😅

I have to say, amazing sleuthing! 😎 Adding a second to every reload adds up and feels like the wrong default workaround. Can you think of a better way we could do something about this?

@imthatwolf
Copy link

imthatwolf commented Jan 29, 2025

Haha, yeah. Go: 'I cleaned up already.' JS: 'Wait, I wasn’t ready!' 😅.

Yes, I agree it's not an elegant way to deal with it; adding a delay as hardware and os differences could cause the issue to still occur.

Possible solutions:

  • Always launch the frontend last as well as on reload (large rewrite and problems occur)
  • Read vite.config.js/ts write newline and immediately write back the original (hacky fix but works)
  • Deal with 'go' and 'runtime' using atomic directory replacement (best solution)

I will post a pr today with a fix, I found the best way to deal with this is to actually use atomic directory replacement. We delete them as normal, then make a go-temp and runtime-temp extract/generate the files there, and then rename them to go and runtime.

It stops the high CPU usage and RAM munching. On the basic Wails frontend template, it stays at ~0% CPU and ~40mb ram even after changes and reloads both front and back ends.

P.S. also noticed in v2/cmd/wails/internal/ dev/dev.go:98 the variable "legacyUseDevServerInsteadofCustomScheme" is not used anymore.

Edit: After more testing the fix is hit or miss depending on project.

imthatwolf added a commit to imthatwolf/wails that referenced this issue Jan 29, 2025
* use atomic directory replace for binding generation
* remove unused 'legacyUseDevServerInsteadofCustomScheme' variable
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants