Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timeout when installing GHC #2254

Open
nmattia opened this issue Oct 7, 2024 · 0 comments
Open

Timeout when installing GHC #2254

nmattia opened this issue Oct 7, 2024 · 0 comments

Comments

@nmattia
Copy link
Contributor

nmattia commented Oct 7, 2024

Describe the bug

I'm seeing timeouts when installing the GHC bindist with rules_haskell:

ERROR: An error occurred during the fetch of repository 'rules_haskell_ghc_linux_amd64':
   Traceback (most recent call last):
	File "/var/tmp/bazel-output/external/rules_haskell/haskell/ghc_bindist.bzl", line 192, column 31, in _ghc_bindist_impl
		execute_or_fail_loudly(
	File "/var/tmp/bazel-output/external/rules_haskell/haskell/private/workspace_utils.bzl", line 25, column 13, in execute_or_fail_loudly
		fail("\n".join(["Command failed: " + " ".join(arguments), exec_result.stderr]))
Error in fail: Command failed: make install -f relocatable.mk
Timed out

For now I'm patching the rules to bump the execute timeout to 3 times the default (3 * 600 seconds). The command is installing (copying) several Gigabytes of data and it seems to take longer than ten minutes on machines with slow IO.

diff --git a/haskell/private/workspace_utils.bzl b/haskell/private/workspace_utils.bzl
index df851329..980b3e16 100644
--- a/haskell/private/workspace_utils.bzl
+++ b/haskell/private/workspace_utils.bzl
@@ -19,6 +19,8 @@ def execute_or_fail_loudly(
         environment = environment,
         quiet = True,
         working_directory = working_directory,
+        # triple the default timeout because e.g. copying GHC is slow (1.5G)
+        timeout = 3 * 600,
     )
     if exec_result.return_code != 0:
         arguments = [_as_string(x) for x in arguments]

Would it make sense to add timeout as an argument to execute_or_fail_loudly and specify a greater timeout in the make install call? Or is there a reason why it would get completely stuck and throw a timeout error? Unfortunately I have not managed to reproduce this reliably and can't confirm that increasing the timeout fixes the issue for good (i.e. can't rule out there's another cause beyond GHC being absurdly large)

Environment

  • OS name + version: Linux, Ubuntu Focal 20.04.6 LTS
  • Bazel version: 7.3.1
  • Version of the rules: 1.0
github-merge-queue bot pushed a commit to dfinity/ic that referenced this issue Oct 7, 2024
This patches our rules_haskell version to increase the timeout for the
`make install` command installing GHC. The default of 10 minutes seems
to be too short in some scenarios, though this has to be confirmed after
the workaround has been in use for a couple of days. We will also need
to investigate ways to avoid burning 10+minutes installing GHC.

See also: tweag/rules_haskell#2254
nikolay-komarevskiy pushed a commit to dfinity/ic that referenced this issue Oct 11, 2024
This patches our rules_haskell version to increase the timeout for the
`make install` command installing GHC. The default of 10 minutes seems
to be too short in some scenarios, though this has to be confirmed after
the workaround has been in use for a couple of days. We will also need
to investigate ways to avoid burning 10+minutes installing GHC.

See also: tweag/rules_haskell#2254
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant