Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EMFILE: too many open files when not using --no-cache #17

Open
straub opened this issue Aug 8, 2019 · 2 comments
Open

EMFILE: too many open files when not using --no-cache #17

straub opened this issue Aug 8, 2019 · 2 comments

Comments

@straub
Copy link

straub commented Aug 8, 2019

If the number of files in this.inputFileSet.files is large enough, readCache() reaches the user's limit and rejects with an error. The error is swallowed in getOutput(), which manifests in a very strange message: Error: Cannot find module 'collect-all', even though that package is installed. (This seems to be the next time something tries to access the filesystem.)

Is there any interest in exploring graceful-fs here or in another module to solve caching for very large numbers of files? Or any other suggested solutions? For now, my workaround is to use --no-cache, which works, but costs more time.

I'd be happy to throw a PR together, either way.

@75lb
Copy link
Member

75lb commented Oct 11, 2019

Hi, sorry for the delay.. which OS are you on? How many input files roughly causes the exception to be thrown? Obviously the error message Cannot find module 'collect-all' is highly misleading..

@75lb
Copy link
Member

75lb commented Aug 25, 2024

Hi, if you get time could you check if this is still an issue in v9.1.0 please. The codebase has been cleaned up and the sync functions abolished so I'm hoping this is no longer an issue. I have not been able to reproduce the issue locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants