Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add exec_group_compatible_with attribute #24964

Draft
wants to merge 14 commits into
base: master
Choose a base branch
from

Conversation

fmeum
Copy link
Collaborator

@fmeum fmeum commented Jan 18, 2025

Fixes #23202
Fixes #23802

@fmeum fmeum force-pushed the 23802-add-exec-group-compatible-with branch 2 times, most recently from 3df50ad to b54072a Compare January 19, 2025 10:32
fmeum added 2 commits January 20, 2025 21:32
This makes `exec_properties` such as `no-remote-exec` effective for the main test spawn, where it previously only affected post-processing spawns such as test XML generation.
@keith
Copy link
Member

keith commented Jan 22, 2025

on this reproducer: #23202 (comment) w/ this change (and the exec_properties one) I get this crash:

FATAL: bazel crashed due to an internal error. Printing stack trace:
java.lang.RuntimeException: Unrecoverable error while evaluating node '' (requested by nodes ':all, excludedSubdirs=[], filteringPolicy=com.google.devtools.build.lib.pkgcache.FilteringPolicies$FilterManual@7dab797e')
        at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:551)
        at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:435)
        at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1577)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:507)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1489)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:2071)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:2033)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:187)
Caused by: net.starlark.java.eval.Starlark$UncheckedEvalException: InvalidStarlarkValueException thrown during Starlark evaluation (an initializer)
        at <starlark>.cc_test(<builtin>:0)
        at <starlark>.<toplevel>(/tmp/repro/BUILD.bazel:30)
Caused by: net.starlark.java.eval.Starlark$InvalidStarlarkValueException: invalid Starlark value: class java.util.ArrayList
        at net.starlark.java.eval.Starlark.checkValid(Starlark.java:136)
        at net.starlark.java.eval.Dict.lambda$copyOf$0(Dict.java:454)
        at com.google.common.collect.SingletonImmutableBiMap.forEach(SingletonImmutableBiMap.java:70)
        at net.starlark.java.eval.Dict.copyOf(Dict.java:451)
        at net.starlark.java.eval.Starlark.fromJava(Starlark.java:217)
        at com.google.devtools.build.lib.packages.Attribute$StarlarkComputedDefaultTemplate.computeValue(Attribute.java:1471)
        at com.google.devtools.build.lib.packages.Attribute$StarlarkComputedDefaultTemplate$1.compute(Attribute.java:1425)
        at com.google.devtools.build.lib.packages.Attribute$ComputationStrategy.computeValuesForAllCombinations(Attribute.java:1228)
        at com.google.devtools.build.lib.packages.Attribute$StarlarkComputedDefaultTemplate.computePossibleValues(Attribute.java:1446)
        at com.google.devtools.build.lib.packages.RuleClass.populateDefaultRuleAttributeValues(RuleClass.java:2438)
        at com.google.devtools.build.lib.packages.RuleClass.populateRuleAttributeValues(RuleClass.java:2208)
        at com.google.devtools.build.lib.packages.RuleClass.createRule(RuleClass.java:2158)
        at com.google.devtools.build.lib.packages.RuleFactory.createRule(RuleFactory.java:106)
        at com.google.devtools.build.lib.packages.RuleFactory.createAndAddRule(RuleFactory.java:141)
        at com.google.devtools.build.lib.analysis.starlark.StarlarkRuleClassFunctions$StarlarkRuleFunction.call(StarlarkRuleClassFunctions.java:1754)
        at net.starlark.java.eval.StarlarkCallable.fastcall(StarlarkCallable.java:87)
        at net.starlark.java.eval.Starlark.fastcall(Starlark.java:804)
        at net.starlark.java.eval.Eval.evalFastcall(Eval.java:821)
        at net.starlark.java.eval.Eval.evalCall(Eval.java:678)
        at net.starlark.java.eval.Eval.eval(Eval.java:547)
        at net.starlark.java.eval.Eval.exec(Eval.java:288)
        at net.starlark.java.eval.Eval.execStatements(Eval.java:83)
        at net.starlark.java.eval.Eval.execFunctionBody(Eval.java:67)
        at net.starlark.java.eval.StarlarkFunction.callWithArguments(StarlarkFunction.java:225)
        at net.starlark.java.eval.StarlarkFunction.positionalOnlyCall(StarlarkFunction.java:209)
        at net.starlark.java.eval.Starlark.positionalOnlyCall(Starlark.java:872)
        at net.starlark.java.eval.Starlark.execFileProgram(Starlark.java:1195)
        at com.google.devtools.build.lib.packages.PackageFactory.executeBuildFileImpl(PackageFactory.java:416)
        at com.google.devtools.build.lib.packages.PackageFactory.executeBuildFile(PackageFactory.java:377)
        at com.google.devtools.build.lib.skyframe.PackageFunction.loadPackage(PackageFunction.java:1188)
        at com.google.devtools.build.lib.skyframe.PackageFunction.compute(PackageFunction.java:539)
        at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:471)
        at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:435)
        at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1577)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:507)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1489)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:2071)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:2033)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:187)

i looked into this a bit last week and I believe it's because this function

return Dict.copyOf(mutability, (Map<?, ?>) x);
needs some sort of "deep copy" in this case, but I couldn't figure that out so I can't say for sure

@fmeum fmeum force-pushed the 23802-add-exec-group-compatible-with branch from 8528676 to c943de3 Compare January 22, 2025 12:30
@keith
Copy link
Member

keith commented Jan 22, 2025

verified on the latest commits here it works with this repro case now!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants