A reference implementation for the Affordance-based Robot Manipulation with Flow Matching.
- Paper page: Affordance-based Robot Manipulation with Flow Matching. https://arxiv.org/abs/2409.01083
- Project page: https://hri-eu.github.io/flow-matching-policy/
- Code: https://github.com/HRI-EU/flow_matching
- Author: Fan Zhang ([email protected]), Michael Gienger
We are in process of integrating flow matching into the Hugging Face 🤗 LeRobot PushT task.
🔬 This repo contains
Training and evaluation examples of using flow matching on PushT and Franka Kitchen benchmarks.
🌷 Getting Started
- Clone this repo and change into it:
git clone [email protected]:HRI-EU/flow-matching-policy.git && cd flow_matching\ - Install the Python dependencies:
python -m venv venv_fm && source venv_fm/bin/activate && pip install --no-cache-dir -r requirements.txt - Enjoy!
🏆 Some Results
Pretrained weights with flow matching: Push-T, Franka Kitchen, Robomimic
| Methods | Push-T1 | Push-T2 | Franka Kitchen | Robomimic3 |
|---|---|---|---|---|
| Flow Matching | 0.9035/0.7519 | 0.7363/0.6218 | 0.9960/0.7425 | 0.9360/0.7289 |
sampling range1: [rs.randint(50, 450), rs.randint(50, 450), rs.randint(200, 300), rs.randint(200, 300), rs.randn() * 2 * np.pi - np.pi]
sampling range2: [rs.randint(50, 450), rs.randint(50, 450), rs.randint(100, 400), rs.randint(100, 400), rs.randn() * 2 * np.pi - np.pi]
Robomimic3: transport task ph
📝 Acknowledgements
- The model structure implementation is modified from Cheng Chi's diffusion_policy repo. The code is under external/diffusion_policy (MIT license). Some code that we modified is located under external/models.
- We use some functions from Alexander Tong's TorchCFM repo (MIT license). It is installed through pip.
- Please download the PushT and robomimic demonstration data from Cheng Chi's diffusion_policy repo.
- Please download the Franka Kitchen demonstration data from Nur Muhammad Shafiullah's Behavior Transformers repo (MIT license).
This project is licensed under the BSD 3-clause license - see the LICENSE.md file for details