Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Toolchainize //scala:toolchain_type #1633

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

mbland
Copy link
Contributor

@mbland mbland commented Oct 23, 2024

Description

Moves the toolchain targets for //scala:toolchain_type to a new @io_bazel_rules_scala_toolchains repository as a step towards Bzlmodification. Part of #1482.

Motivation

Instantiating toolchains in their own repository enables module extensions to define the the repositories required by those toolchains within the extension's own namespace. Bzlmod users can then register the toolchains from this repository without having to import all the toolchains' dependencies into their own namespace via use_repo().


The scala_toolchains_repo() macro wraps the underlying repository rule and assigns it the standard name io_bazel_rules_scala_toolchains. Right now it's only instantiating the main Scala toolchain via the default scala = True parameter. Future changes will expand this macro and repository rule with more boolean parameters to instantiate other toolchains, specifically:

  • scalatest
  • junit
  • specs2
  • twitter_scrooge
  • jmh
  • scala_proto and scala_proto_enable_all_options
  • testing (includes all of scalatest, junit, and specs2)
  • scalafmt

WORKSPACE users will now have to import and call the scala_toolchains_repo() macro to instantiate
@io_bazel_rules_scala_toolchains.

load("@io_bazel_rules_scala//:scala_config.bzl", "scala_config")

scala_config()

load(
    "//scala:scala.bzl",
    "rules_scala_setup",
    "rules_scala_toolchain_deps_repositories",
    "scala_toolchains_repo",
)

rules_scala_setup()

rules_scala_toolchain_deps_repositories()

scala_toolchains_repo()

register_toolchains("@io_bazel_rules_scala_toolchains//...:all")

Or, they can use the new single scala_toolchains() macro to elide most of the calls:

load("@io_bazel_rules_scala//:scala_config.bzl", "scala_config")

scala_config()

load("//scala:scala.bzl", "scala_toolchains")

scala_toolchains()

register_toolchains("@io_bazel_rules_scala_toolchains//...:all")

This is what the corresponding MODULE.bazel setup would look like:

module(name = "rules_scala", version = "7.0.0")

scala_config = use_extension(
    "//scala/extensions:config.bzl", "scala_config"
)
scala_config.settings(scala_version = "2.13.14")

scala_deps = use_extension("//scala/extensions:deps.bzl", "scala_deps")
scala_deps.toolchains()

The register_toolchains() call in WORKSPACE isn't strictly required at this point, but is recommended. All the WORKSPACE files in this repo already registered their required toolchains using existing macros. However, they've been updated to use register_toolchains(...) instead.

These register_toolchains() calls maintain the order of previous toolchain registrations to avoid the following breakages:

  • test_compilation_fails_with_plus_one_deps_undefined from test/shell/test_compilation.sh depends on //scala:unused_dependency_checker_error_toolchain resolving first. This toolchains sets the scala_toolchain() parameters dependency_tracking_method = "ast-plus" and unused_dependency_checker_mode = "error", and the @io_bazel_rules_scala_toolchains//scala toolchains don't.

  • test_scala_binary_allows_opt_in_to_use_of_argument_file_in_runner_for_improved_performance from test/shell/test_scala_binary.sh depends on the use_argument_file_in_runner parameter of scala_toolchain being False. This is the default, but the @io_bazel_rules_scala_toolchains//scala toolchains explicitly set this to True instead.

In the Bzlmod case, the register_toolchains() call isn't necessary at all. This is because @io_bazel_rules_scala_toolchains includes one package per set of toolchains, and the rules_scala MODULE.bazel calls register_toolchains("@io_bazel_rules_scala_toolchains//...:all"). This will automatically register all configured rules_scala toolchains, while allowing users to override any of them using register_toolchains() in their own MODULE.bazel files.

Technically, the scala_deps.toolchains() call isn't required when only using the default scala = True parameter; the rules_scala MODULE.bazel will instantiate this automatically, too.

Moves the `toolchain` targets for `//scala:toolchain_type` to a new
`@io_bazel_rules_scala_toolchains` repository as a step towards
Bzlmodification. Part of bazelbuild#1482.

Instantiating toolchains in their own repository enables module
extensions to define the the repositories required by those toolchains
within the extension's own namespace. Bzlmod users can then register the
toolchains from this repository without having to import all the
toolchains' dependencies into their own namespace via `use_repo()`.

---

The `scala_toolchains_repo()` macro wraps the underlying repository rule
and assigns it the standard name `io_bazel_rules_scala_toolchains`.
Right now it's only instantiating the main Scala toolchain via the
default `scala = True` parameter. Future changes will expand this
macro and repository rule with more boolean parameters to instantiate
other toolchains, specifically:

- `scalatest`
- `junit`
- `specs2`
- `twitter_scrooge`
- `jmh`
- `scala_proto` and `scala_proto_enable_all_options`
- `testing` (includes all of `scalatest`, `junit`, and `specs2`)
- `scalafmt`

---

`WORKSPACE` users will now have to import and call the
`scala_toolchains_repo()` macro to instantiate
`@io_bazel_rules_scala_toolchains`.

```py
load("@io_bazel_rules_scala//:scala_config.bzl", "scala_config")

scala_config()

load(
    "//scala:scala.bzl",
    "rules_scala_setup",
    "rules_scala_toolchain_deps_repositories",
    "scala_toolchains_repo",
)

rules_scala_setup()

rules_scala_toolchain_deps_repositories()

scala_toolchains_repo()

register_toolchains("@io_bazel_rules_scala_toolchains//...:all")
```

This is what the corresponding `MODULE.bazel` setup would look like:

```py
module(name = "rules_scala", version = "7.0.0")

scala_config = use_extension(
    "//scala/extensions:config.bzl", "scala_config"
)
scala_config.settings(scala_version = "2.13.14")

scala_deps = use_extension("//scala/extensions:deps.bzl", "scala_deps")
scala_deps.toolchains()
```

The `register_toolchains()` call in `WORKSPACE` isn't strictly required
at this point, but is recommended. However, all the `WORKSPACE` files in
this repo already register their required toolchains using existing
macros, which have been updated in this change.

In fact, calling `register_toolchains()` right after
`scala_toolchains_repo()` as shown above breaks two tests that depend on
the existing `WORKSPACE` toolchain registration:

- `test_compilation_fails_with_plus_one_deps_undefined` from
  `test/shell/test_compilation.sh` depends on
  `scala_register_unused_deps_toolchains()` setting up its toolchain to
  resolve first. `//scala:unused_dependency_checker_error_toolchain`
  sets the `scala_toolchain()` parameters `dependency_tracking_method =
  "ast-plus"` and `unused_dependency_checker_mode = "error"`, and the
  `@io_bazel_rules_scala_toolchains//scala` toolchains don't.

- `test_scala_binary_allows_opt_in_to_use_of_argument_file_in_runner_for_improved_performance`
  from `test/shell/test_scala_binary.sh` depends on the
  `use_argument_file_in_runner` parameter of `scala_toolchain` being
  `False`. This is the default, but the
  `@io_bazel_rules_scala_toolchains//scala` toolchains explicitly set
  this to `True` instead.

In the Bzlmod case, the `register_toolchains()` call isn't necessary at
all. This is because `@io_bazel_rules_scala_toolchains` includes one
package per set of toolchains, and the rules_scala `MODULE.bazel` calls
`register_toolchains("@io_bazel_rules_scala_toolchains//...:all")`. This
will automatically register all configured rules_scala toolchains, while
allowing users to override any of them using `register_toolchains()` in
their own `MODULE.bazel` files.

Technically, the `scala_deps.toolchains()` call isn't required when only
using the default `scala = True` parameter; the rules_scala
`MODULE.bazel` will instantiate this automatically, too.
Extracted a single `scala_toolchains` macro to share between `WORKSPACE`
and the `deps.bzl` module extension. This will make it easier to ensure
`WORKSPACE` compatibility, and to add a Bazel module extension as a thin
layer on top. Part of bazelbuild#1482.

This change includes updates to `rules_scala_setup` and
`scala_repositories` to support this, while preserving compatibility
with existing `WORKSPACE` calls. The next commit will replace many
existing `WORKSPACE` calls with `scala_toolchains`.
Also added `@io_bazel_rules_scala_toolchains//...:all` to
`register_toolchains()` calls everywhere, even when not specifically
necessary. This proves the mechanism is safe and works with `WORKSPACE`
now, and will make future updates to consolidate other toolchains less
noisy.
@mbland
Copy link
Contributor Author

mbland commented Oct 25, 2024

I've rebased this PR onto master after #1635 landed, updated it substantially with two new commits, and adjusted the PR description.

The first new commit introduces the new scala_toolchains() macro, which will be the common funnel point between WORKSPACE and Bzlmod. However, that first commit doesn't add the macro to existing WORKSPACE calls. This demonstrates that the changes to rules_scala_setup and scala_repositories are compatible with existing API usage.

The second commit does replace as many WORKSPACE calls as possible with scala_toolchains() and register_toolchains(). This shows that the new API produces equivalent results, and proves a clear path towards adding the Bzlmod layer.

If you'd like to see the end state I have in mind for this macro, the scala_toolchains_repo() macro/rule, and the Bzlmod extension, check out the equivalent files in my Bzlmod working branch:

I've tested that all the following work with both the first commit and the second:

Once this change lands, toolchainizing the remaining toolchains should prove very straightforward. Then Bzlmod is possibly only PR away after that.

Should've been included in the previous commit. All tests still pass.
Missed this one in third_party/test/proto earlier. Also removed
unnecessary `USE_BAZEL_VERSION` expression in
test_scala_proto_library.sh. If `USE_BAZEL_VERSION` is set, it takes
precedence to begin with, and third_party/test/proto/.bazelversion
exists now after bazelbuild#1629.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant