Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MOD-2454: add async_chain #2418

Open
wants to merge 26 commits into
base: main
Choose a base branch
from

Conversation

kramstrom
Copy link
Contributor

@kramstrom kramstrom commented Oct 28, 2024

Adds an async_chain which allows you to concat different async generators and iterate through them sequentially

Depends on #2421 for some general fixes for async_utils methods

@mwaskom mwaskom changed the title MOD-2454: add async_cocat MOD-2454: add async_concat Oct 28, 2024
@kramstrom kramstrom changed the title MOD-2454: add async_concat [WIP] MOD-2454: add async_concat Oct 29, 2024
@kramstrom kramstrom marked this pull request as draft October 29, 2024 09:37
@kramstrom kramstrom marked this pull request as ready for review October 29, 2024 15:21
@kramstrom kramstrom changed the title [WIP] MOD-2454: add async_concat MOD-2454: add async_concat Oct 29, 2024
modal/_utils/async_utils.py Outdated Show resolved Hide resolved
modal/_utils/async_utils.py Outdated Show resolved Hide resolved


async def async_concat(*iterables):
for it in iterables:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: In some of our other generators we "take ownership" of the input iterables and make sure to cancel/close them if the generator itself is cancelled/closed. Might want to be consistent in how we do this across all of these utils

modal/_utils/async_utils.py Outdated Show resolved Hide resolved
…rom/mod-2454-remove-aiostream-dependency-cancellation-handling
…tion-handling' of github.com:modal-labs/modal-client into kramstrom/mod-2454-remove-aiostream-dependency-async-concat
@kramstrom kramstrom changed the title MOD-2454: add async_concat MOD-2454: add async_chain Oct 30, 2024
…tion-handling' of github.com:modal-labs/modal-client into kramstrom/mod-2454-remove-aiostream-dependency-async-concat
…tion-handling' of github.com:modal-labs/modal-client into kramstrom/mod-2454-remove-aiostream-dependency-async-concat
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants