Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consolidate Transform gwms_aws_config_data into AWSSpotPrice and AWSJobInstances #386

Open
StevenCTimm opened this issue Nov 3, 2021 · 0 comments
Labels
operations Issue filed by Operations team

Comments

@StevenCTimm
Copy link
Contributor

We have noticed that the intermediate steps currently taken in the gwms_aws_config_data transform
of the channel_aws_config_data channel could actually be done by routines that are downstream from them.

WE propose the following arrangement of code. Once this is done we will then not have any sources (or source proxies)
that depend on the output of transforms.

[root@hepcsvc03 sources]# more /root/proposed_plan
Now

channel_aws_config_data
source: factory_entries -> Transform gwms_aws_config_data -> aws_instance_limits datablock -> AwsJobLimits source pro
xy->FigureOfMerit transform

Change to

source: factory_entries -> combined transform [ code from gwms_to_aws.py that makes Job_limits] ->FigureOfMerit
transform
Note would have to add a factory entries source proxy to aws_calculations_with_source_proxy temporarily
to make this work

source: factory_entries -> transform gwms_aws_config_data -> spot_occupancy_config datablock -> hybrid source_proxy (
AWSSpotPrice) -> FigureOfMerit transform

Change to

source:factory_entries -> combined transform [ code that makes spot_occupancy_config datablock + code that queries fo
r spot_occupancy ] -> AWS FigureOfMerit transform

@StevenCTimm StevenCTimm added the operations Issue filed by Operations team label Nov 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
operations Issue filed by Operations team
Projects
None yet
Development

No branches or pull requests

1 participant