-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support discovery of suites with failed initialization by UniqueId
#109
Comments
I also prepared a PR that should fix the issue, as it seemed pretty straight-forward. But let me know if that's not how you see it :). |
I was also wondering, why did you decide to report such failures during the execution phase instead of making it a hard failure during the discovery phase? |
Hi @pshevche, thank you for the report and especially for the PR. As far as I remember I did that weird workaround, because if it failed, the exception was somehow lost up in the Junit5 system during the discovery. As the result, no tests were "discovered" and the test run was "passing". I should investigate if it is still the issue. |
@giurim , I see, it makes sense. Another benefit of this approach is that you could still execute as many test suites as possible and get at least some feedback instead of failing early, which might be a good thing. If you decide to stick with the current strategy, then we would need to handle these custom descriptors one way or another, as it is documented as an engine requirement here: https://junit.org/junit5/docs/current/user-guide/#test-engines-requirements-mandatory (see the second point). |
Describe the bug
Currently, the engine will create a custom
TestDescriptor
if the test suite initialization fails (ref). This descriptor will have aUniqueId
of form[engine:scalatest]/[failed:<SUITE_NAME>]
. However, the engine does not support the suite discovery based on this ID. This limits the ways to interact with the engine, since the consumer can use the discovery result to further limit the tests to be executed and then pass a smaller set ofUniqueIdDescriptor
s toexecute
method.To Reproduce
This branch has a test that shows the empty discovery result if the
UniqueId
of this syntheticTestDescriptor
is used as input.Expected behavior
The engine detects that its synthetic descriptor is used and returns a valid discovery result.
The text was updated successfully, but these errors were encountered: