Container snapshots/restore feature and other critical bug fixes

[kripper]

PR Summary

This PR introduces several important fixes and features to enhance the overall functionality of the system.

Key Changes:

• Fix for re-joining existing conversations/sandboxes when using use_host_network = 1.
• Fix for parsing config docker_runtime_kwargs = { privileged = true }
  [Feature] Support running docker-in-docker for OpenHands runtime #5569
• Fix for reattaching to Docker when using host network, addressing [Bug]: KeyError: 'ExposedPorts' #6440
• Snapshot/restore of container storage feature added, resolving [Feature] Container snapshots and restore #6163
• Dev Containers support added (stable, and recommended as the official development approach).

New Feature: Container Snapshot and Restore

This feature introduces the ability to snapshot and restore container storage. It mounts Docker's storage directory (/var/lib/docker) on a virtual Btrfs volume using a loop device within the OpenHands (OH) environment. This approach has been tested on Windows using VS Code and Dev Containers, with OH running in a Docker container and creating sandboxes in nested containers.

Motivation:

The primary goal of this feature is to enable reverting changes made by OH across the entire system. This allows users to effectively undo any system modifications introduced during interactions within OH, promoting a safer and more predictable development and testing environment.

Important Notes:

• The workspace is mounted separately and is not included in the container storage, meaning it is excluded from snapshots. Since the workspace is shared across containers or conversations, it would not be appropriate to roll back changes that affect other sessions or containers. See: [Proposal] Exclusive Workspaces for Snapshot Isolation Across Conversations #6457.

Current Status:

• No additional features will be added at this moment.
• Frontend developers are welcome to implement a button to allow users to revert snapshots (one per user message).

Usage Instructions:

To enable the snapshot and restore functionality, add the following configuration to your sandbox settings:

[sandbox]
docker_snapshots = true

If disabled, the Docker configuration will revert upon restarting OH.

Snapshot Creation:

• Every time a user sends a message, a new snapshot of the container storage will be created.
• The logs will display the command required to restore each snapshot.

Testing

• The file openhands/storage/docker_snapshots.py contains a test case (unit-tests in comments) to debug and understand how it works.

---

Feel free to reach out if there are any issues or further questions.

[enyst]

This is a great feature, thank you for working on it!

It might be good to try to keep the bug fixes separate from new feature(s). One smaller fixes PR and one large feature PR would mean that the fixes PR is faster to review and merge. If you need the fixes too, you can create/rebase the feature branch on top on the fixes branch. It shouldn't cause much trouble when merged, then.

Up to you, it just would be better IMHO. Because I think you are correct that the fixes are needed (independently), so it would be nice if we can take them in before the snapshots feature is ready.

[kripper]

It might be good to try to keep the bug fixes separate from new feature(s).

Here I isolated the critical bug fixes:
#6460