Skip to content

Reinforcement learning environments for robot-assisted laparoscopic surgery

License

Notifications You must be signed in to change notification settings

ScheiklP/sofa_env

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

72 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SOFA_ENV

Gym environments for reinforcement learning based on the SOFA Simulation Framework. This repository is part of "LapGym - An Open Source Framework for Reinforcement Learning in Robot-Assisted Laparoscopic Surgery". See LapGym for the paper and lap_gym for the top level repository.

Getting Started

Getting the right python version

  1. Install pyenv
  2. Make sure you also installed the build dependencies
  3. Install python 3.10 through pyenv
pyenv install 3.10.15
  1. Create a virtualenv with python 3.10 for sofa and activate it
pyenv virtualenv 3.10.15 sofa
pyenv activate sofa

Option A (recommended): Using prebuilt SOFA binaries

  1. Clone this repository
git clone https://github.com/ScheiklP/sofa_env.git 
  1. Install the repository
cd sofa_env
pip install -e .
  1. Test the installation
python3 sofa_env/scenes/controllable_object_example/controllable_env.py

Option B (for the reckless): Manually setting up SOFA and SofaPython3

If you want to set up SOFA and SofaPython3 manually, you can follow the instructions to install SOFA and SofaPython3. And then install the sofa_env package with an environment variable to tell the setup script to skip the SOFA installation.

git clone https://github.com/ScheiklP/sofa_env.git 
cd sofa_env
SKIP_SOFA=1 pip install -e .
python3 sofa_env/scenes/controllable_object_example/controllable_env.py

Environments

This repository currently contains 14 reinforcement learning environments.

Reach Deflect Spheres Search for Point
Tissue Manipulation Pick and Place Grasp Lift and Touch
Rope Cutting Precision Cutting Tissue Dissection
Thread in Hole Rope Threading Ligating Loop
Tissue Retraction Magnetic Continuum Robot Bimanual Tissue Manipulation

Adding new Environments

A SofaEnv base class and sofa_templates are provided to simplify implementing new reinforcement environments. Please refer to the included documentation for a tutorial on how to build new scenes in SOFA and implement reinforcement environments with sofa_env. Also visit sofa_godot for SofaGodot, a Godot plugin to visually create new SOFA scenes.

Documentation

You can either access the documentation here or build it locally with

cd docs
make html
firefox build/html/index.html

Citing

If you use the project in your work, please consider citing it with:

@article{JMLR:v24:23-0207,
  author  = {Paul Maria Scheikl and Balázs Gyenes and Rayan Younis and Christoph Haas and Gerhard Neumann and Martin Wagner and Franziska Mathis-Ullrich},
  title   = {LapGym - An Open Source Framework for Reinforcement Learning in Robot-Assisted Laparoscopic Surgery},
  journal = {Journal of Machine Learning Research},
  year    = {2023},
  volume  = {24},
  number  = {368},
  pages   = {1--42},
  url     = {http://jmlr.org/papers/v24/23-0207.html}
}

Notes

The Freemotion Animation Loop of SOFA solves collisions and other constraints with Lagrange Multipliers. This method requires the computation of a compliance matrix for each object. This compliance matrix can be precomputed and is valid as long as the object does not change too much for example through cutting. Precomputing, handling, and loading the compliance matrix is done by the PrecomputedConstraintCorrection object. If your scene has one or more of them, the object will try to load the matrix from storage, and compute it, when it cannot find the appropriate file. This can be seen by the terminal output of SOFA when creating the scene. You may see a [ERROR] [FileRepository] File tissue-1479-0.1.comp NOT FOUND in: ... output which is actually not an error but SOFA telling you it is now generating the precomputed compliance matrix. This may take up to a few minutes.

Getting Started with SOFA - Reading List

  • To get a general understanding of how SOFA works, read all pages under Simulation Principles in the official documentation
  • To get a deeper understanding of the underlying principles, like mappings, read the SOFA Paper. But be aware, that some terms like MechanicalState (now called MechanicalObject) in the paper have changed in the actual API. Use it more as a "how does SOFA work?" not as a "how is SOFA implemented?"
  • Look through the SOFA Tutorials. Some of the Tutorials might not work. That is generally not your fault. You can run them via the command line. For example
$FOLDER_TARGET/install/bin/runSofa $FOLDER_SRC/examples/Tutorials/Basic/TutorialBasicCube.scn

Acknowledgements

This work is supported by the Helmholtz Association under the joint research school "HIDSS4Health – Helmholtz Information and Data Science School for Health".

About

Reinforcement learning environments for robot-assisted laparoscopic surgery

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages