You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
However, it also contains some data that is specific to particilar job submissions:
platform - The platform that the current/next job submission is using / will use.
submit_num - The submit num of the current / next job.
is_manual_submit - Were any(?) of the submissions the result of a manual trigger?
local_job_file_path - Path of the current submission.
Additionally, the task proxy has two separate objects containing specific attributes of job submissions:
jobs - (potentially incomplete) list of job selected configuration settings.
mode_settings - Transient job state pertaining to the current/next submission.
This is a bit of a jumble, and code interfaces can easily end up relying on interfaces which are not stable for the intended purposes (e.g. #6326).
Proposal:
Create a Job object to take over from TaskProxy.job[item] and TaskProxy.mode_settings. Use an efficient store, e.g. a class with __slots__ or a protobuf serialisation.
Migrate other job-specific TaskProxy fields into the job.
Ensure that these fields are either repopulated from the DB on task-re-spawn/workflow-reload/workflow-restart OR recomputed as required.
Manage job data lifecycle for efficiency (i.e. don't hang onto data we no longer need).
The text was updated successfully, but these errors were encountered:
Separate out the task and job specific data into separate objects and ensure it is persisted correctly.
The
TaskProxy
object contains some data that is specific to the task, e.g:tdef
- The task definition.tokens
- The parsed task ID.clock_trigger_time
- External wallclock dependency.state
- Aggregate task state.However, it also contains some data that is specific to particilar job submissions:
platform
- The platform that the current/next job submission is using / will use.submit_num
- The submit num of the current / next job.is_manual_submit
- Were any(?) of the submissions the result of a manual trigger?local_job_file_path
- Path of the current submission.Additionally, the task proxy has two separate objects containing specific attributes of job submissions:
jobs
- (potentially incomplete) list of job selected configuration settings.mode_settings
- Transient job state pertaining to the current/next submission.This is a bit of a jumble, and code interfaces can easily end up relying on interfaces which are not stable for the intended purposes (e.g. #6326).
Proposal:
Job
object to take over fromTaskProxy.job[item]
andTaskProxy.mode_settings
. Use an efficient store, e.g. a class with__slots__
or a protobuf serialisation.TaskProxy
fields into the job.The text was updated successfully, but these errors were encountered: