Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Align sandbox exec arguments with subprocess #2419

Merged
merged 1 commit into from
Oct 31, 2024

Conversation

pawalt
Copy link
Member

@pawalt pawalt commented Oct 28, 2024

Describe your changes

This commit adds new sandbox exec arguments, allowing the user to specify if they want to consume a stream directly, ignore the result, or consume it via a pipe.

Resolves WRK-422
Resolves WRK-413

Backward/forward compatibility checks

Check these boxes or delete any item (or this section) if not relevant for this PR.

  • Client+Server: this change is compatible with old servers
  • Client forward compatibility: this change ensures client can accept data intended for later versions of itself

Note on protobuf: protobuf message changes in one place may have impact to
multiple entities (client, server, worker, database). See points above.


I've tested this behavior out locally:

import modal
from modal.io_streams import StreamType

app = modal.App("peyton-sb-test")

@app.local_entrypoint()
async def main():
    sandbox = modal.Sandbox.create(
        timeout=1000,
        app=app,
    )

    exc = sandbox.exec("echo", "hello", stdout=StreamType.PIPE)
    print(exc.stdout.read(), end="")

    exc = sandbox.exec("echo", "hi2", stdout=StreamType.DEVNULL)
    try:
        print(exc.stdout.read())
    except modal.exception.InvalidError as e:
        print(e)

    exc = sandbox.exec("echo", "hi3", stdout=StreamType.STDOUT)
    exc.wait()
    try:
        print(exc.stdout.read())
    except modal.exception.InvalidError as e:
        print(e)

    sandbox.terminate()

Changelog

Output from Sandbox.exec can now be directed to /dev/null, stdout, or stored for consumption. This behavior can be controlled via the new StreamType arguments.

@pawalt pawalt requested a review from freider October 28, 2024 19:29
modal/io_streams.py Outdated Show resolved Hide resolved
modal/io_streams.py Outdated Show resolved Hide resolved
Copy link
Contributor

@freider freider left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments and major realizations :D

modal/io_streams.py Outdated Show resolved Hide resolved
modal/io_streams.py Outdated Show resolved Hide resolved
modal/io_streams.py Outdated Show resolved Hide resolved
modal/sandbox.py Show resolved Hide resolved
@pawalt pawalt force-pushed the pawalt/subprocccy branch 5 times, most recently from bca1027 to 0c38119 Compare October 29, 2024 17:42
Copy link
Contributor

@freider freider left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! didn't look to closely at the internals, but the public API makes sense to me! wdyt @mwaskom ?

modal/sandbox.py Show resolved Hide resolved
This commit adds new sandbox exec arguments, allowing the user to
speicfy if they want to consume a stream directly, ignore the result, or
consume it via a pipe.
@pawalt
Copy link
Member Author

pawalt commented Oct 31, 2024

@prbot automerge

Copy link

@modal-pr-review-automation modal-pr-review-automation bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some required workflows are pending. Will automerge once they complete successfully. Please bear with us in this difficult time.

@modal-pr-review-automation modal-pr-review-automation bot merged commit 8f8e31a into main Oct 31, 2024
21 checks passed
@modal-pr-review-automation modal-pr-review-automation bot deleted the pawalt/subprocccy branch October 31, 2024 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants