diff --git a/README.md b/README.md index d89ee8d..45ebe57 100644 --- a/README.md +++ b/README.md @@ -109,6 +109,8 @@ In this section, we will demonstrate our real-world deployment/evaluation system * Set center of gravity to (2, -6, 37)mm, CX/CY/CZ. * TCP will be set automatically by the eval script. * On UR5e, switch control mode to remote. + + If you are using Franka, follow this [instruction](franka_instruction.md). 3. Setup WSG50 gripper with web interface: * Obtain IP address and update [eval_robots_config.yaml](example/eval_robots_config.yaml)/grippers/gripper_ip. * In Settings > Command Interface diff --git a/assets/umi_franka.png b/assets/umi_franka.png new file mode 100644 index 0000000..3ad2902 Binary files /dev/null and b/assets/umi_franka.png differ diff --git a/franka_instruction.md b/franka_instruction.md new file mode 100644 index 0000000..18bed1d --- /dev/null +++ b/franka_instruction.md @@ -0,0 +1,30 @@ +# Deploy UMI Policy on Franka Robot + +## System Overview + + +* FrankaInterface (Server): [scripts_real/launch_franka_interface_server.py](scripts_real/launch_franka_interface_server.py) (L7) + +* FrankaInterface (Client): [umi/real_world/franka_interpolation_controller.py](umi/real_world/franka_interpolation_controller.py) (L36) + +* FrankaInterpolationController: [umi/real_world/franka_interpolation_controller.py](umi/real_world/franka_interpolation_controller.py) (L71) + + +## Instructions +* Follow the [documentation](https://facebookresearch.github.io/fairo/polymetis/prereq.html#franka-panda-hardware-setup) to install *Polymetis* on a computer with realtime kernel (i.e., NUC in our example). +* Launch FrankaInterface Server on NUC. + + `python scripts_real/launch_franka_interface_server.py` +* (optional) Now you should be able to control the Franka arm using a space mouse on another desktop, the one you are going to run the robot policy on. + + `python scripts_real/control_franka.py` +* Change [eval_robots_config.yaml](example/eval_robots_config.yaml)/robots/robot_type to `'franka'`. This will change the robot controller in [umi_env](umi/real_world/bimanual_umi_env.py) (L233). + +* Obtain IP address and update [eval_robots_config.yaml](example/eval_robots_config.yaml)/robots/robot_ip. +* On the Franka interface website + * Set mass to 1.8 kg + * Set Flange to Center of Mass of Load Vector to (0.064, -0.06, 0.03)m. + +* Then you should be able to launch the evaluation on the franka arm. + + `python eval_real.py --robot_config=example/eval_robots_config.yaml -i cup_wild_vit_l.ckpt -o data/eval_cup_wild_example` \ No newline at end of file