Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Initial load of multiple home units is slow with lazy component loading #4381

Open
fendor opened this issue Aug 16, 2024 · 4 comments · May be fixed by #4439
Open

Initial load of multiple home units is slow with lazy component loading #4381

fendor opened this issue Aug 16, 2024 · 4 comments · May be fixed by #4439
Assignees
@soulomoon
Labels
component: ghcide multi-component Issues relating to multi-component support type: bug Something isn't right: doesn't work as intended, documentation is missing/outdated, etc..

Comments

@fendor
Copy link
Collaborator

fendor commented Aug 16, 2024

With a default hie.yaml of

cradle:
  cabal:

and "sessionLoading": "multiComponent", the initial load can be incredibly slow, as HLS starts asking for the GhcSession for each module that's currently open in the editor one by one. cabal needs to reconfigure the project each time the set of units to load changes, which can take a non-trivial amount of time on big projects (for example HLS itself).

We get logs such as these:

...
2024-08-16T08:59:17.665247Z | Debug | cabal exec -v0 -- ghc --print-libdir
2024-08-16T08:59:19.499261Z | Debug | cabal exec -v0 -- ghc -package-env=- -ignore-dot-ghci -e Control.Monad.join (Control.Monad.fmap System.IO.putStr System.Environment.getExecutablePath)
2024-08-16T08:59:19.500156Z | Info | cabal --builddir=/home/hugin/.cache/hie-bios/dist-hls-036ebb56a4cc135651b7510a90f3136d v2-repl --with-compiler /home/hugin/.cache/hie-bios/wrapper-b54f81dea4c0e6d1626911c526bc4e36 --with-hc-pkg /home/hugin/.cache/hie-bios/ghc-pkg-99f55999495b863a61d4f15d989eaab4 --keep-temp-files --enable-multi-repl /home/hugin/Documents/haskell/hls/ghcide/test/exe/HighlightTests.hs
...
2024-08-16T08:59:21.104638Z | Debug | Resolving dependencies...
2024-08-16T08:59:25.984647Z | Debug | Build profile: -w ghc-9.6.5 -O1
2024-08-16T08:59:25.984774Z | Debug | In order, the following will be built (use -v for more details):
2024-08-16T08:59:25.984835Z | Debug |  - hls-plugin-api-2.9.0.1 (lib) (configuration changed)
2024-08-16T08:59:25.984921Z | Debug |  - ghcide-2.9.0.1 (lib) (configuration changed)
2024-08-16T08:59:25.984989Z | Debug |  - hls-test-utils-2.9.0.1 (lib) (configuration changed)
2024-08-16T08:59:25.985049Z | Debug |  - ghcide-2.9.0.1 (exe:ghcide) (configuration changed)
2024-08-16T08:59:25.985105Z | Debug |  - haskell-language-server-2.9.0.1 (interactive) (test:ghcide-tests) (configuration changed)
...
2024-08-16T09:01:06.465845Z | Info | Load cabal cradle using all files (multi-components)
2024-08-16T09:01:08.269128Z | Debug | cabal exec -v0 -- ghc --print-libdir
2024-08-16T09:01:10.157674Z | Debug | cabal exec -v0 -- ghc -package-env=- -ignore-dot-ghci -e Control.Monad.join (Control.Monad.fmap System.IO.putStr System.Environment.getExecutablePath)
2024-08-16T09:01:10.158281Z | Info | cabal --builddir=/home/hugin/.cache/hie-bios/dist-hls-036ebb56a4cc135651b7510a90f3136d v2-repl --with-compiler /home/hugin/.cache/hie-bios/wrapper-b54f81dea4c0e6d1626911c526bc4e36 --with-hc-pkg /home/hugin/.cache/hie-bios/ghc-pkg-99f55999495b863a61d4f15d989eaab4 --keep-temp-files --enable-multi-repl /home/hugin/Documents/haskell/hls/ghcide/src/Development/IDE/Plugin/TypeLenses.hs /home/hugin/Documents/haskell/hls/ghcide/test/exe/HighlightTests.hs
...
2024-08-16T09:01:11.794655Z | Debug | Resolving dependencies...
2024-08-16T09:01:16.890247Z | Debug | Build profile: -w ghc-9.6.5 -O1
2024-08-16T09:01:16.890352Z | Debug | In order, the following will be built (use -v for more details):
2024-08-16T09:01:16.890393Z | Debug |  - ghcide-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:01:16.890695Z | Debug |  - hls-test-utils-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:01:16.890832Z | Debug |  - ghcide-2.9.0.1 (interactive) (exe:ghcide) (configuration changed)
2024-08-16T09:01:16.890906Z | Debug |  - haskell-language-server-2.9.0.1 (interactive) (test:ghcide-tests) (configuration changed)
...
2024-08-16T09:02:09.749233Z | Debug | cabal exec -v0 -- ghc -package-env=- -ignore-dot-ghci -e Control.Monad.join (Control.Monad.fmap System.IO.putStr System.Environment.getExecutablePath)
2024-08-16T09:02:09.749803Z | Info | cabal --builddir=/home/hugin/.cache/hie-bios/dist-hls-036ebb56a4cc135651b7510a90f3136d v2-repl --with-compiler /home/hugin/.cache/hie-bios/wrapper-b54f81dea4c0e6d1626911c526bc4e36 --with-hc-pkg /home/hugin/.cache/hie-bios/ghc-pkg-99f55999495b863a61d4f15d989eaab4 --keep-temp-files --enable-multi-repl /home/hugin/Documents/haskell/hls/hls-plugin-api/src/Ide/Types.hs /home/hugin/Documents/haskell/hls/ghcide/src/Development/IDE/Plugin/TypeLenses.hs /home/hugin/Documents/haskell/hls/ghcide/test/exe/HighlightTests.hs
...
2024-08-16T09:02:16.081585Z | Debug | Build profile: -w ghc-9.6.5 -O1
2024-08-16T09:02:16.081704Z | Debug | In order, the following will be built (use -v for more details):
2024-08-16T09:02:16.081758Z | Debug |  - hls-plugin-api-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:02:16.082405Z | Debug |  - ghcide-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:02:16.082488Z | Debug |  - hls-test-utils-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:02:16.082542Z | Debug |  - ghcide-2.9.0.1 (interactive) (exe:ghcide) (configuration changed)
2024-08-16T09:02:16.082591Z | Debug |  - haskell-language-server-2.9.0.1 (interactive) (test:ghcide-tests) (configuration changed)
...
2024-08-16T08:59:19.500156Z | Info | cabal --builddir=/home/hugin/.cache/hie-bios/dist-hls-036ebb56a4cc135651b7510a90f3136d v2-repl --with-compiler /home/hugin/.cache/hie-bios/wrapper-b54f81dea4c0e6d1626911c526bc4e36 --with-hc-pkg /home/hugin/.cache/hie-bios/ghc-pkg-99f55999495b863a61d4f15d989eaab4 --keep-temp-files --enable-multi-repl /home/hugin/Documents/haskell/hls/ghcide/test/exe/HighlightTests.hs
...
2024-08-16T08:59:21.104638Z | Debug | Resolving dependencies...
2024-08-16T08:59:25.984647Z | Debug | Build profile: -w ghc-9.6.5 -O1
2024-08-16T08:59:25.984774Z | Debug | In order, the following will be built (use -v for more details):
2024-08-16T08:59:25.984835Z | Debug |  - hls-plugin-api-2.9.0.1 (lib) (configuration changed)
2024-08-16T08:59:25.984921Z | Debug |  - ghcide-2.9.0.1 (lib) (configuration changed)
2024-08-16T08:59:25.984989Z | Debug |  - hls-test-utils-2.9.0.1 (lib) (configuration changed)
2024-08-16T08:59:25.985049Z | Debug |  - ghcide-2.9.0.1 (exe:ghcide) (configuration changed)
2024-08-16T08:59:25.985105Z | Debug |  - haskell-language-server-2.9.0.1 (interactive) (test:ghcide-tests) (configuration changed)
...
2024-08-16T09:01:06.465845Z | Info | Load cabal cradle using all files (multi-components)
2024-08-16T09:01:08.269128Z | Debug | cabal exec -v0 -- ghc --print-libdir
2024-08-16T09:01:10.157674Z | Debug | cabal exec -v0 -- ghc -package-env=- -ignore-dot-ghci -e Control.Monad.join (Control.Monad.fmap System.IO.putStr System.Environment.getExecutablePath)
2024-08-16T09:01:10.158281Z | Info | cabal --builddir=/home/hugin/.cache/hie-bios/dist-hls-036ebb56a4cc135651b7510a90f3136d v2-repl --with-compiler /home/hugin/.cache/hie-bios/wrapper-b54f81dea4c0e6d1626911c526bc4e36 --with-hc-pkg /home/hugin/.cache/hie-bios/ghc-pkg-99f55999495b863a61d4f15d989eaab4 --keep-temp-files --enable-multi-repl /home/hugin/Documents/haskell/hls/ghcide/src/Development/IDE/Plugin/TypeLenses.hs /home/hugin/Documents/haskell/hls/ghcide/test/exe/HighlightTests.hs
2024-08-16T09:01:11.794655Z | Debug | Resolving dependencies...
2024-08-16T09:01:16.890247Z | Debug | Build profile: -w ghc-9.6.5 -O1
2024-08-16T09:01:16.890352Z | Debug | In order, the following will be built (use -v for more details):
2024-08-16T09:01:16.890393Z | Debug |  - ghcide-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:01:16.890695Z | Debug |  - hls-test-utils-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:01:16.890832Z | Debug |  - ghcide-2.9.0.1 (interactive) (exe:ghcide) (configuration changed)
2024-08-16T09:01:16.890906Z | Debug |  - haskell-language-server-2.9.0.1 (interactive) (test:ghcide-tests) (configuration changed)
...
2024-08-16T09:02:09.749233Z | Debug | cabal exec -v0 -- ghc -package-env=- -ignore-dot-ghci -e Control.Monad.join (Control.Monad.fmap System.IO.putStr System.Environment.getExecutablePath)
2024-08-16T09:02:09.749803Z | Info | cabal --builddir=/home/hugin/.cache/hie-bios/dist-hls-036ebb56a4cc135651b7510a90f3136d v2-repl --with-compiler /home/hugin/.cache/hie-bios/wrapper-b54f81dea4c0e6d1626911c526bc4e36 --with-hc-pkg /home/hugin/.cache/hie-bios/ghc-pkg-99f55999495b863a61d4f15d989eaab4 --keep-temp-files --enable-multi-repl /home/hugin/Documents/haskell/hls/hls-plugin-api/src/Ide/Types.hs /home/hugin/Documents/haskell/hls/ghcide/src/Development/IDE/Plugin/TypeLenses.hs /home/hugin/Documents/haskell/hls/ghcide/test/exe/HighlightTests.hs
...
2024-08-16T09:02:12.266032Z | Debug | eval: fp /home/hugin/Documents/haskell/hls/ghcide/src/Development/IDE/Plugin/TypeLenses.hs
2024-08-16T09:02:12.266264Z | Debug | Finished: codeLens.config Took: 0.00s
2024-08-16T09:02:16.081585Z | Debug | Build profile: -w ghc-9.6.5 -O1
2024-08-16T09:02:16.081704Z | Debug | In order, the following will be built (use -v for more details):
2024-08-16T09:02:16.081758Z | Debug |  - hls-plugin-api-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:02:16.082405Z | Debug |  - ghcide-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:02:16.082488Z | Debug |  - hls-test-utils-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:02:16.082542Z | Debug |  - ghcide-2.9.0.1 (interactive) (exe:ghcide) (configuration changed)
2024-08-16T09:02:16.082591Z | Debug |  - haskell-language-server-2.9.0.1 (interactive) (test:ghcide-tests) (configuration changed)
...

Note, how this takes three minutes.

Potential solution

Load all modules that are currently open immediately if loadStyle == multiComponent.
@fendor fendor added type: bug Something isn't right: doesn't work as intended, documentation is missing/outdated, etc.. status: needs triage component: ghcide multi-component Issues relating to multi-component support and removed status: needs triage labels Aug 16, 2024
@soulomoon
Copy link
Collaborator

soulomoon commented Aug 24, 2024 •
edited
Loading

I suppose, we need a batch loader in hie-bois ? So we can generate deps for all the foi files at once.

@soulomoon soulomoon mentioned this issue Sep 25, 2024
Closed
@soulomoon soulomoon self-assigned this Oct 24, 2024
@soulomoon soulomoon linked a pull request Oct 24, 2024 that will close this issue
Open
@soulomoon
Copy link
Collaborator

soulomoon commented Oct 28, 2024 •
edited
Loading

After implementing a demo for batch load mode and play with it, I’ve encountered a significant challenge: it’s difficult to effectively distribute errors when consulting cradles, as we’re batch loading a large number of new files. 🤔

@fendor
Copy link
Collaborator Author

fendor commented Oct 29, 2024

Yeah, that makes sense... can't we just display the errors in all files that we failed to load?

@soulomoon
Copy link
Collaborator

soulomoon commented Oct 29, 2024 •
edited
Loading

can't we just display the errors in all files that we failed to load?

Yes we can, but the cons is a bit high.
Say we opened x1 .... xn. x1 is the one can't be loaded. But we can load x2 ... xn perfectly.
In batch mode, we are left with x1 ... xn all in loading errors, which is really annoying.

Pertential workaround would be to load these files one by one when batch load failed.
Trade off with load times.

soulomoon added a commit to soulomoon/haskell-language-server that referenced this issue Nov 2, 2024
@soulomoon
Refactor session loading to manage pending files so we can batch load…
58b8b68 
… them to improve performance fix haskell#4381
@soulomoon soulomoon mentioned this issue Nov 9, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@soulomoon soulomoon

Labels
component: ghcide multi-component Issues relating to multi-component support type: bug Something isn't right: doesn't work as intended, documentation is missing/outdated, etc..
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

SessionLoader: batch load soulomoon/haskell-language-server
2 participants
@fendor @soulomoon