Memory leak building under WSL

[BrianMar]

Describe the bug
Trying to build a large repo on WSL2 (Ubuntu 22.04.4 LTS). When I invoke lage, the process seems to grow monotonically and slow down until one of the build threads runs out of memory in the JS heap.

To Reproduce
This occurs in a private repo within Microsoft that I can't share here. Please contact me internally and I can help you with the repro.

• Clone the repo in WSL2, yarn, yarn build

The build will start off transpiling packages very fast, but then the memory usage of the top level task will start growing. As it does, the overall build slows down.

Expected behavior
The build should complete.

Environment/Troubleshooting
I am running lage through node through yarn in our repo. The build environment is WSL2 running Ubuntu 22.04.4 LTS. I do not see this issue when I'm running the build inside of Windows

I hacked our internal lage tasks to dump a v8 Heap Snapshot at the time of the transpile starting. And there was definitely a leak visible between invocations of the next thread. Many of these appeared to be directly related to <graceful-fs.queue>.

I built my own local version of lage, replacing graceful-fs with stock fs. This reudce the rate of growth, but there was still a slowdown and eventual crash.

At which point I moved my instrumentation for Heap Snapshots to WorkerRunner.run. This slowed down the leak rate as well. I believe this was due to the v8.writeHeapSnapshot funciton invoking GC. However, this dump kept taking longer, so the build was still slow.

I then changed this code to a direct call to just invoke the GC before every run: 

This highly reduced the memory growth, and the build was about to complete in 7m34s, which is faster than building the same repo on Windows was.

So I'm not sure if there's an actual leak or just a need for some idle time for the GC to run. There definitely appears to be an issue with graceful-fs under WSL2 & Ubuntu 22.04.4 in lage.

Here's the output form "top" while the process was running. This is a 16-core 64GB machine, so I had 16 lage workers running with each limited to 2GB of JS heap.