Support port-forwarding or tunnels natively

[vanpelt]

What problem or use case are you trying to solve?

It would be very helpful to interact with a web service when creating one with Open Hands. Think streamlit app, or even a rest api.

Describe the UX of the solution you'd like

Ideally the sandbox could detect when a port is opened and forward it like Colab, Codespaces, or Jupyter Hub does.

Do you have thoughts on the technical implementation?

There are many possible approaches here. Here are some idea's:

1. The quickest solution would be to get VSCode to do forwarding via MSoft Dev Tunnels. This requires a Github or MSoft login, but seems like it would do the trick. I've not been able to figure out how to get the openvscode-server to expose ports, but VSCode's official docs say it's supported.
2. Other implementations of VSCode like the Coder implementation which supports forwarding via subdomain or a /proxy path. Speaking from experience, forwarding from a path is unfortunate and can cause a lot of pain. It's much better if each opened port can just be served from its own subdomain.
3. The first two solutions don't really impact the runtime or the agent as it can just operate on localhost still. Certain runtimes like Modal etc. do have api's to tunnel new ports after the sandbox is started. Unfortunately as far as I can tell, there's no way to expose a new port from a docker container after starting it so a more universal tunneling solution is likely better.

Describe alternatives you've considered

I can ask the agent to use it's browser tool to visit the application, but I want to interact with whatever it's built.

Additional context

[mamoodi]

From what I've been told, this is already available in main. We are hoping to release this soon.

[Niklas080]

What options are there currently to directly try out what the agent built without having to download and set up everything locally?

[mamoodi]

@Niklas080 if you would look to try it out, you could try the main version and see how it works. We are hoping to release it as beta soon and get some feedback. We know there are some issues with it. I've tried it and it works well with localhost. Even if it doesn't show in the App tab, it becomes accessible by going to the URL directly.

docker run -it --rm --pull=always \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:main-nikolaik \
    -e LOG_ALL_EVENTS=true \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v ~/.openhands-state:/.openhands-state \
    -p 3000:3000 \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.all-hands.dev/all-hands-ai/openhands:main

[Niklas080]

I tried it, this only worked once and now I can't reach any page on localhost again. thanks for your help anyway

[mamoodi]

The first version of this has been released as part of 0.20.0. If this version does not quite resolve this issue, please explain what is missing and I'll reopen.