Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added node index to current_context #2812

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

Atharva1723
Copy link

Tracking issue

Closes flyteorg/flyte#5534

Why are the changes needed?

In Flyte, when working with map tasks, outputs often risk being overwritten if they share the same static name. For example, if multiple nodes generate output files with the same name, only the last file would remain, as previous ones would be overwritten. By adding the array node index, it becomes possible to uniquely name these outputs, avoiding conflicts.

What changes were proposed in this pull request?

The key goal of this PR is to include the index of an array node in map_tasks within the current_context() returned value. This feature is especially useful when multiple nodes within a map task could potentially overwrite outputs

Copy link

welcome bot commented Oct 13, 2024

Thank you for opening this pull request! 🙌

These tips will help get your PR across the finish line:

  • Most of the repos have a PR template; if not, fill it out to the best of your knowledge.
  • Sign off your commits (Reference: DCO Guide).

Copy link
Member

@pingsutw pingsutw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link

codecov bot commented Oct 14, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 53.11%. Comparing base (ac7b7e8) to head (ad31d7e).
Report is 1 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (ac7b7e8) and HEAD (ad31d7e). Click for more details.

HEAD has 9 uploads less than BASE
Flag BASE (ac7b7e8) HEAD (ad31d7e)
18 9
Additional details and impacted files
@@             Coverage Diff             @@
##           master    #2812       +/-   ##
===========================================
- Coverage   81.34%   53.11%   -28.24%     
===========================================
  Files         196      226       +30     
  Lines       20324    21818     +1494     
  Branches     2619     2613        -6     
===========================================
- Hits        16533    11588     -4945     
- Misses       3063     9704     +6641     
+ Partials      728      526      -202     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@pingsutw pingsutw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you run make fmt locally to fix the lint error?

@Atharva1723
Copy link
Author

Will write an unit test and will run formatter too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Array Node Index To current_context [Core feature]
2 participants