Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running tests in parallel #153

Closed
mixedCase opened this issue Oct 26, 2021 · 6 comments
Closed

Running tests in parallel #153

mixedCase opened this issue Oct 26, 2021 · 6 comments
Assignees
Labels
improvement Not a bug but an improvement to overall user experience

Comments

@mixedCase
Copy link

Is your feature request related to a problem? Please describe.

I'd like to parallelize the execution of most of my tests in order to speed up their execution.

To be more specific I've a series of tests that heavily bottleneck on IO not only during execution but also in their fixtures. While an asynchronous IO runtime on the Java-side would be best I don't think that's going to be viable for a few years at least, so having multiple threads to nanny the syscalls is the next best thing and the only alternative to projects who are actually bottlenecking on CPU instead.

Describe the solution you'd like

To be able to set metadata to specific deftests as well as ns declarations (which should be equivalent to marking all of its deftests as parallel), to indicate that these tests can become part of a grand pool of tests that can be executed in parallel by a group of workers before the synchronized tests run.

Nice to haves:

  • Being able to specify a custom worker cap as a system property, with a sensible default like the system's virtual core count.
  • Buffered output when running Poly CLI for extra clarity, meaning results would only be printed once all tests for a given namespace are done in case they're all parallel, or waiting until the beginning of the namespace's synchronous execution phase to print the results of its parallel tests right before executing the synchronous ones. Failures could be shown when they occur and releasing the buffer for its namespace then and there, stopping execution just like poly test currently does for synchronous tests.

Describe alternatives you've considered

  • Plug-in API for custom test runners. -- Feels overkill, and likely to introduce more complexity than having a single test runner with even the superset of all popular test runner features.
  • Testing-specific CLI alternative to poly. -- It would likely lag behind Polylith if maintained out of tree and cause a lot more of maintenance effort.
@svdo
Copy link

svdo commented Oct 28, 2021

Maybe only slightly related, but I'd also love to easily be able to use kaocha for example.

@tengstrand tengstrand self-assigned this Oct 31, 2021
@tengstrand tengstrand added the improvement Not a bug but an improvement to overall user experience label Oct 31, 2021
@furkan3ayraktar
Copy link
Collaborator

@mixedCase This should be solved by #196. Once that PR ready and merged, you could write your own custom test runner to
parallelize the tests.

@svdo There will be a custom koacha test runner that @imrekoszo is developing. Once the custom test runner support is there, you can plug it in.

@svdo
Copy link

svdo commented Apr 7, 2022

Awesome! 😎

@imrekoszo
Copy link
Contributor

With parallel execution support coming to kaocha and #196 coming to Polylith, this should be a possibility soon.

@tengstrand
Copy link
Collaborator

Hi guys !

Now when it's possible to run tests using Kaocha in the latest 0.2.15-alpha release, can we close this issue?
@mixedCase @imrekoszo @svdo

@mixedCase
Copy link
Author

I believe so. Can't test myself as I don't find myself working in Clojure anymore but I think that does it. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Not a bug but an improvement to overall user experience
Projects
None yet
Development

No branches or pull requests

5 participants