-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Stop a training job #147
Stop a training job #147
Conversation
for more information, see https://pre-commit.ci
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #147 +/- ##
==========================================
+ Coverage 89.29% 94.31% +5.02%
==========================================
Files 24 24
Lines 2411 2570 +159
Branches 138 149 +11
==========================================
+ Hits 2153 2424 +271
+ Misses 258 146 -112 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A couple of small comments on my end. I can't review the front-end stuff, but the back end components seem sound.
|
||
# Start the server training listener and client training listeners as threads to update | ||
# the job's metrics and status once the training is done | ||
server_listener_thread = Thread(target=asyncio.run, args=(server_training_listener(job),)) | ||
server_listener_thread.daemon = True |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Out of curiosity, why do we need this to be True now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I should have added an explanation here, but daemon threads will not prevent the main thread to terminate if they are still running. This will solve an issue I have seen with shutting down the servers when there are listeners still running.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah...that makes sense. Thanks for the background!
This reverts commit fa3e587.
PR Type
Feature
Short Description
Clickup Ticket(s): Link(s) if applicable.
Adding a button on in-progress jobs to stop it. Clicking on it will call an API that will ultimately kill the clients' processes and the server processes.
In this PR I am:
/start
endpoint failsTests Added
Fully unit and integration tested