diff --git a/README.md b/README.md index 2cb96c58..ec69f01e 100644 --- a/README.md +++ b/README.md @@ -16,15 +16,26 @@ Authors :: Vikash Kumar (vikashplus@gmail.com), Vittorio Caggiano (caggiano@gma `MyoSuite` is a collection of musculoskeletal environments and tasks simulated with the [MuJoCo](http://www.mujoco.org/) physics engine and wrapped in the OpenAI ``gym`` API to enable the application of Machine Learning to bio-mechanic control problems. - [Full task details](https://github.com/myohub/myosuite/blob/main/docs/source/suite.rst#tasks) | [Baselines](https://github.com/myohub/myosuite/tree/main/myosuite/agents/baslines_NPG) | [Documentation](https://myosuite.readthedocs.io/en/latest/) -| [Tutorials](https://github.com/myohub/myosuite/tree/main/docs/source/tutorials) + + + + + +[Documentation](https://myosuite.readthedocs.io/en/latest/) | [Tutorials](https://github.com/myohub/myosuite/tree/main/docs/source/tutorials) | [Task specifications](https://github.com/myohub/myosuite/blob/main/docs/source/suite.rst#tasks) + Below is an overview of the tasks in the MyoSuite. TasksALL -## Getting Started + + +## Before you start +It is highly recommended to familiarize your self with the OpenAI ``gym`` (also known as [Gymnasium](https://gymnasium.farama.org/)) API. The tasks are defined through the ``env`` API and a introductory level understanding of Reinforcement Learning in general is also recommened. + +## Installations You will need Python 3.8 or later versions. It is recommended to use [Miniconda](https://docs.conda.io/en/latest/miniconda.html#latest-miniconda-installer-links) and to create a separate environment with: @@ -43,6 +54,8 @@ Test your installation using the following command (this will return also a list ``` bash python -m myosuite.tests.test_myo ``` + + You can also visualize the environments with random controls using the command below: ``` bash @@ -53,10 +66,14 @@ python -m myosuite.utils.examine_env --env_name myoElbowPose1D6MRandom-v0 mjpython -m myosuite.utils.examine_env --env_name myoElbowPose1D6MRandom-v0 ``` + + ## Examples It is possible to create and interface with MyoSuite environments just like any other OpenAI gym environments. For example, to use the `myoElbowPose1D6MRandom-v0` environment, it is possible simply to run: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1zFuNLsrmx42vT4oV8RbnEWtkSJ1xajEo) + + ```python from myosuite.utils import gym env = gym.make('myoElbowPose1D6MRandom-v0') @@ -70,6 +87,14 @@ env.close() You can find [tutorials](https://github.com/myohub/myosuite/tree/main/docs/source/tutorials#tutorials) and the **ICRA2023 Colab Tutorial** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1KGqZgSYgKXF-vaYC33GR9llDsIW9Rp-q) on how to load MyoSuite models/tasks, train them, and visualize their outcome. Also, you can find [baselines](https://github.com/myohub/myosuite/tree/main/myosuite/agents) to test some pre-trained policies. + + +## Projects with Myosuite + +TODO: Add lattice paper and other works using myosuite +TODO: add Myodex paper + + ## License diff --git a/docs/source/api.rst b/docs/source/api.rst new file mode 100644 index 00000000..adc8a668 --- /dev/null +++ b/docs/source/api.rst @@ -0,0 +1,6 @@ +API +========================================== + +.. _documentations: + +To be developed \ No newline at end of file diff --git a/docs/source/images/test_env.png b/docs/source/images/test_env.png new file mode 100644 index 00000000..47707a38 Binary files /dev/null and b/docs/source/images/test_env.png differ diff --git a/docs/source/index.rst b/docs/source/index.rst index 343c325a..190a03ee 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -1,19 +1,54 @@ Welcome to MyoSuite's documentation! ===================================== -`MyoSuite `_ is a collection of musculoskeletal environments and tasks simulated with the `MuJoCo `_ physics engine and wrapped in the OpenAI ``gym`` API to enable the application of Machine Learning to bio-mechanic control problems +`MyoSuite `_ is a collection of musculoskeletal environments and tasks simulated with the `MuJoCo `_ physics engine and wrapped in the OpenAI ``gym`` API to enable the application of Machine Learning to bio-mechanic control problems. + +Check our `github repository `__ for more technical details. + +Our paper can be found at: `https://arxiv.org/abs/2205.13600 `__ + +Before you start, we highly recommended to have a basic understanding of `OpenAI Gym API `__. Principle knowledge in Reinforcement Learning is also expected for a best experience. + .. note:: This project is under active development. + +Main features +============================================ + * Bio-mechanic based simulation of musculoskeletal + * Task-driven controller for motion synthesis + +.. TODO: confirm features for myosuite + + .. toctree:: :maxdepth: 1 :caption: Get started install - suite tutorials - baselines + +.. toctree:: + :maxdepth: 1 + :caption: Advanced Features + + suite + + +.. toctree:: + :maxdepth: 1 + :caption: Documentations + + api + +.. toctree:: + :maxdepth: 1 + :caption: Projects with Myosuite + + projects + baselines + .. toctree:: :maxdepth: 1 diff --git a/docs/source/install.rst b/docs/source/install.rst index 4707cfea..914be454 100644 --- a/docs/source/install.rst +++ b/docs/source/install.rst @@ -67,3 +67,8 @@ For example, to use the ``myoElbowPose1D6MRandom-v0`` environment it is possible env.mj_render() env.step(env.action_space.sample()) # take a random action env.close() + +By default it will activate the simulator and the following visualization is expected: + +.. image:: images/test_env.png + :width: 300 \ No newline at end of file diff --git a/docs/source/projects.rst b/docs/source/projects.rst new file mode 100644 index 00000000..fb6759c8 --- /dev/null +++ b/docs/source/projects.rst @@ -0,0 +1,107 @@ +Projects with Myosuite +######################################### + +.. _projects: + +* :ref:`myochallenge` + * :ref:`myo_challenge_22` + * :ref:`myo_challenge_23` + * :ref:`myo_challenge_24` +* :ref:`pub_with_myosuite` + * :ref:`ref_myodex` + * :ref:`ref_deprl` + * :ref:`ref_lattic` + * :ref:`ref_sar` + + +.. _myochallenge: + +MyoChallenge +======================================== +The Myosuite Team organised MyoChallenge to tackle difficult problems in top-level machine learning conference. +Our latest challenge is accepted to NeuIPs 2024. + +.. _myo_challenge_22: + +MyoChallenge-2022: Learning Physiological Dexterity +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +Introducing `MyoChallenge - a NeurIPS 2022 `__ competition track on learning contact-rich manipulation skills for a physiologically +realistic musculo-skeletal hand. The goal of MyoChallenge is to push our understanding of physiological motor-control responsible +for nimble and agile movements of the human body. In the current edition of MyoChallenge, +we are focusing on developing controllers for contact rich dexterous manipulation behaviors. +This challenge builds upon the MyoSuite ecosystem -- a fast (>4000x faster) and contact-rich framework +for musculoskeletal motor control. + + + + +Competition Tracks +The MyoChallenge consists of two tracks: + +.. _myo_challenge_23: + +MyoChallenge-2023: Towards Human-Level Dexterity and Agility +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +Introducing `MyoChallenge 2023 `__: Towards Human-Level Dexterity and Agility + +Humans effortlessly grasp objects of diverse shapes and properties and execute +agile locomotion without overwhelming their cognitive capacities. This ability was acquired +through millions of years of evolution, which honed the symbiotic relationship between the central and +peripheral nervous systems and the musculoskeletal structure. Consequently, it is not surprising that +uncovering the intricacies of these complex, evolved systems underlying human movement remains a formidable +challenge. Advancements in neuromechanical simulations and data driven methods offer promising avenues to +overcome these obstacles. This year’s competition will feature two tracks: the manipulation track and the locomotion track. + +.. _myo_challenge_24: + +Incoming Myochallenge This Year: ++++++++++++++++++++++++++++++++++++++ +About to be released + + + + +.. _pub_with_myosuite: + +Publications with Myosuite +======================================== + + +Please feel free to create a PR for your own project with Myosuite + +.. _ref_myodex: + +MyoDex: A Generalizable Prior for Dexterous Manipulation +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +Link avaiable at `here `__ + + + +.. _ref_deprl: + +DEP-RL: Embodied Exploration for Reinforcement Learning in Overactuated and Musculoskeletal Systems +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +Link avaiable at `here `__ + + + +.. _ref_lattic: + +Lattice: Latent Exploration for Reinforcement Learning +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +Link avaiable at `here `__ + + + +.. _ref_sar: + +SAR: Generalization of Physiological Agility and Dexterity via Synergistic Action Representation +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + + +Link avaiable at `here `__ \ No newline at end of file diff --git a/docs/source/suite.rst b/docs/source/suite.rst index 4ddad10f..f297893a 100644 --- a/docs/source/suite.rst +++ b/docs/source/suite.rst @@ -1,8 +1,24 @@ -Suite -################## +Models and Tasks +############################## .. _suite: +Models and tasks are the most important aspects of Myosuite. To provide further, + + +* :ref:`models` + * :ref:`myoFinger` + * :ref:`myoElbow` + * :ref:`myoHand` + * :ref:`myoLeg` + * :ref:`myoArm` +* :ref:`tasks` +* :ref:`task_and_variations` + + + + +.. _models: Models ******** @@ -16,14 +32,29 @@ contact-rich movements involving object-manipulation such as pen-twirling and ba myoFinger ========== -Simplified and intuitive model (based on Xu et al. (2012)) of a 4 Degree of Freedom (DoF) finger +Simplified and intuitive model (based on Xu et al. (2012)[https://ieeexplore.ieee.org/document/6290710]) of a 4 Degree of Freedom (DoF) finger which is actuated through a series of 5 simplified antagonistic muscle-tendon units. We also provide its robotic counterpart with simple torque actuators to facilitate the comparative investigation. +Details of this model can be found at `myoFinger `__ + + .. image:: images/myoFinger.png :height: 200 +======== ============================= +Acronym Muscle +======== ============================= +EXTN Central Extensor +adabR Abduction Tendon(Right) +adabL Abduction Tendon(left) +mflx Pip Flexor +dflx DIP Flexor +======== ============================= + + + .. _myoElbow: myoElbow @@ -31,6 +62,8 @@ myoElbow A model of 2 DoF human elbow joint -- based on OpenSim’s default testing arm model (Delp et al. (2007); Seth et al. (2018)) -- and actuated using multiple agonist/antagonist pairs (3 flexors and 3 extensors). +Details of this model can be found at `myoElbow `__ + .. image:: images/myoElbow.png :height: 200 @@ -52,6 +85,8 @@ myoHand The dexterous human hand requires coordination of multiple highly redundant muscles, which have complementary and antagonistic effects on various joints. This musculoskeletal model is comprised of 29 bones, 23 joints, and 39 muscles-tendon units. +Details of this model can be found at `myoHand `__ + .. image:: images/myoHand.png :height: 200 @@ -93,9 +128,52 @@ Rajagopal's full body gait model [https://github.com/opensim-org/opensim-models/ Joints and muscle details can be found in Rajagopal's paper [https://ieeexplore.ieee.org/document/7505900] +The model implementation details can be found on our github repo at `myoLeg `__ + .. image:: images/MyoLeg.png :height: 200 +.. ============= ================================= +.. Acronym Muscle +.. ============= ================================= +.. addbrev_r Adductor brevis +.. addlong_r Adductor longus +.. addmagDist_r Adductor magnus (distal) +.. addmagIsch_r Adductor magnus (ischial) +.. addmagMid_r Adductor magnus (middle) +.. addmagProx_r Adductor magnus (proximal) +.. bflh_r Biceps femoris long head +.. bfsh_r Biceps femoris short head +.. edl_r Extensor digitorum longusc +.. ehl_r Extensor hallucis longusc +.. fdl_r Flexor digitorum longus +.. fhl_r Flexor hallucis longus +.. gaslat_r Gastrocnemius lateral head +.. gasmed_r Gastrocnemius medial head +.. glmax1_r Gluteus maximus (superior) +.. glmax2_r Gluteus maximus (middle) +.. glmax3_r Gluteus maximus (inferior) +.. glmed1_r Gluteus medius (anterior) +.. glmed2_r Gluteus medius (middle) +.. glmed3_r Gluteus medius (posterior) +.. glmin1_r Gluteus minimus (anterior) +.. glmin2_r Gluteus minimus (middle) +.. glmin3_r Gluteus minimus (posterior) + + +.. fhl Flexor hallucis longus +.. fhl Flexor hallucis longus +.. fhl Flexor hallucis longus +.. fhl Flexor hallucis longus +.. fhl Flexor hallucis longus +.. fhl Flexor hallucis longus +.. fhl Flexor hallucis longus +.. fhl Flexor hallucis longus +.. fhl Flexor hallucis longus +.. fhl Flexor hallucis longus +.. fhl Flexor hallucis longus +.. ============= ================================= + .. _myoArm: myoArm @@ -104,10 +182,11 @@ The 3D dexterous human arm reaching and manipulation. This musculoskeletal model is comprised of 27 Degree of Freedom, and 63 muscles-tendon units. -.. image:: images/myoArm.png - :height: 200 +Model specification can be found on our github repo of `myoArm `__ +.. image:: images/myoArm.png + :height: 200 .. _tasks: @@ -246,7 +325,7 @@ Variants: :width: 200 Hand Multiobject Reorientation -============== +====================================== A :ref:`myoHand` model reorients a set of parameterized geometries to a given target orientation without dropping it. As with before, the complexity of this task is due to the intermittent contacts between the object and multiple fingers while trying to stabilize the object, as well as the multiobject nature of the task. @@ -320,142 +399,150 @@ Objective: Variants: - **Easy**: Achieve a forward velocity in the y-direction without moving in the x-direction. ``myoLegWalk-v0`` - **Hard**: Achieve a forward velocity in the y-direction without moving in the x-direction on uneven terrain. ``myoLegWalk-v0`` + .. image:: images/myoLeg_walk.png :width: 200 -Relocate Mobjects -======================================== -A :ref:`myoArm` model - - - -Non-stationarities task variations -*********************************** - +.. Relocate Mobjects +.. ======================================== +.. A :ref:`myoArm` model -.. _sarcopenia: - -Sarcopenia -============== -Sarcopenia is a muscle disorder that occurs commonly in the elderly population (Cruz-Jentoft and Sayer (2019)) -and characterized by a reduction in muscle mass or volume. -The peak in grip strength can be reduced up to 50% from age 20 to 40 (Dodds et al. (2016)). -We modeled sarcopenia for each muscle as a reduction of 50% of its maximal isometric force. +.. Non-stationarities task variations +.. *********************************** -.. _fatigue: -Fatigue -============================ -Muscle Fatigue is a short-term (second to minutes) effect that happens after sustained or repetitive voluntary movement -and it has been linked to traumas e.g. cumulative trauma disorder (Chaffin et al. (2006)). -A dynamic muscle fatigue model (Ma et al. (2009)) was integrated into the modeling framework. -This model was based on the idea that different types of muscle fiber that have different contributions -and resistance to fatigue (Vøllestad (1997)). -The current implementation is simplified to consider the same fatigue factor for all muscles and -that muscle can be completely fatigued. - -.. image:: images/Fatigue.png - :width: 800 - - -.. _ttransfer: - -Tendon transfer -================================= -Contrary to muscle fatigue or sarcopenia that occurs to all muscles, tendon transfer surgery can target a single -muscle-tendon unit. Tendon transfer surgery allows redirecting the application point of muscle forces from one joint -DoF to another (see below). It can be used to regain functional control of a joint or limb motion after injury. -One of the current procedures in the hand is the tendon transfer of Extensor Indicis Proprius (EIP) to replace the -Extensor Pollicis Longus (EPL) (Gelb (1995)). Rupture of the EPL can happen after a broken wrist and create a loss of control -of the Thumb extension. We introduce a physical tendon transfer where the EIP application point of the tendon was moved -from the index to the thumb and the EPL was removed (see Figure 3). - -.. image:: images/tendon_transfer.png - :width: 400 - -.. _exo: - -Exoskeleton assistance -====================== -Exoskeleton assisted rehabilitation is becoming more and more common practice (Jezernik et al. (2003)) due to its multiple benefit (Nam et al. (2017)). -Modeling of an exoskeleton for the elbow was done via an ideal actuator and the addition of two supports with a weight of 0.101 Kg for the upper arm and 0.111 Kg on the forearm. The assistance given by the exoskeleton was a percentage of the biological joint torque, this was based on the neuromusculoskeletal controller presented in Durandau et al. (2019). - -.. image:: images/elbow_exo.png - :width: 200 +.. .. _sarcopenia: -Summary of task and variantions -*********************************** - - - - -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -|:ref:`tasks` | **Environment** | **Difficulty** |:ref:`sarcopenia` |:ref:`fatigue` | :ref:`ttransfer` | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Finger Joint Pose | ``myoFingerPoseFixed-v0`` | Easy | ✓ | ✓ | | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Finger Joint Pose | ``myoFingerPoseRandom-v0`` | Hard | ✓ | ✓ | | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Finger Tip Reach | ``myoFingerReachFixed-v0`` | Easy | ✓ | ✓ | | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Finger Tip Reach | ``myoFingerReachRandom-v0`` | Hard | ✓ | ✓ | | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Elbow Joint Pose | ``myoElbowPose1D6MRandom-v0`` | Hard | ✓ | ✓ | | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Hand Joints Pose | ``myoHandPoseFixed-v0`` | Easy | ✓ | ✓ | ✓ | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Hand Joints Pose | ``myoHandPoseRandom-v0`` | Hard | ✓ | ✓ | ✓ | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Hand Tips Reach | ``myoHandReachFixed-v0`` | Easy | ✓ | ✓ | ✓ | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Hand Tips Reach | ``myoHandReachRandom-v0`` | Hard | ✓ | ✓ | ✓ | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Hand Key Turn | ``myoHandKeyTurnFixed-v0`` | Easy | ✓ | ✓ | ✓ | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Hand Key Turn | ``myoHandKeyTurnRandom-v0`` | Hard | ✓ | ✓ | ✓ | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Hand Object Hold | ``myoHandObjHoldFixed-v0`` | Easy | ✓ | ✓ | ✓ | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Hand Object Hold | ``myoHandObjHoldRandom-v0`` | Hard | ✓ | ✓ | ✓ | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Hand Pen Twirl | ``myoHandPenTwirlFixed-v0`` | Easy | ✓ | ✓ | ✓ | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Hand Pen Twirl | ``myoHandPenTwirlRandom-v0`` | Hard | ✓ | ✓ | ✓ | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Die Rotation | ``myoChallengeDieReorientP1-v1`` | Easy | ✓ | ✓ | ✓ | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Die Rotation | ``myoChallengeDieReorientP2-v1`` | Hard | ✓ | ✓ | ✓ | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Hand Baoding Balls | ``myoChallengeBaodingP1-v1`` | Easy | ✓ | ✓ | ✓ | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Hand Baoding Balls | ``myoChallengeBaodingP2-v1`` | Hard | ✓ | ✓ | ✓ | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| 8 Obj. Rotation | ``myoHandReorient8-v0`` | Easy | ✓ | ✓ | ✓ | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| 100 Obj. Rotation | ``myoHandReorient100-v0`` | Easy | ✓ | ✓ | ✓ | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| 1000 Obj. Rotation | ``myoHandReorientID-v0`` | Hard | ✓ | ✓ | ✓ | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| 1000 Obj. Rotation | ``myoHandReorientOOD-v0`` | Hard | ✓ | ✓ | ✓ | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Leg walk | ``myoLegWalk-v0`` | Easy | ✓ | ✓ | | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Leg walk Rough Ground | ``myoLegRoughTerrainWalk-v0`` | Hard | ✓ | ✓ | | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Leg walk Hilly Ground | ``myoLegHillyTerrainWalk-v0`` | Hard | ✓ | ✓ | | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Leg walk Stair Ground | ``myoLegStairTerrainWalk-v0`` | Hard | ✓ | ✓ | | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Grasping & Placing | ``myoChallengeRelocateP1-v0`` | Easy | ✓ | ✓ | | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ -| Chase Tag | ``myoChallengeChaseTagP1-v0`` | Easy | ✓ | ✓ | | -+------------------------+----------------------------------+-----------------+------------------+---------------+---------------------+ - - -Variations: +.. Sarcopenia +.. ============== + +.. Sarcopenia is a muscle disorder that occurs commonly in the elderly population (Cruz-Jentoft and Sayer (2019)) +.. and characterized by a reduction in muscle mass or volume. +.. The peak in grip strength can be reduced up to 50% from age 20 to 40 (Dodds et al. (2016)). +.. We modeled sarcopenia for each muscle as a reduction of 50% of its maximal isometric force. + + +.. .. _fatigue: + +.. Fatigue +.. ============================ +.. Muscle Fatigue is a short-term (second to minutes) effect that happens after sustained or repetitive voluntary movement +.. and it has been linked to traumas e.g. cumulative trauma disorder (Chaffin et al. (2006)). +.. A dynamic muscle fatigue model (Ma et al. (2009)) was integrated into the modeling framework. +.. This model was based on the idea that different types of muscle fiber that have different contributions +.. and resistance to fatigue (Vøllestad (1997)). +.. The current implementation is simplified to consider the same fatigue factor for all muscles and +.. that muscle can be completely fatigued. + +.. .. image:: images/Fatigue.png +.. :width: 800 + + +.. .. _ttransfer: + +.. Tendon transfer +.. ================================= +.. Contrary to muscle fatigue or sarcopenia that occurs to all muscles, tendon transfer surgery can target a single +.. muscle-tendon unit. Tendon transfer surgery allows redirecting the application point of muscle forces from one joint +.. DoF to another (see below). It can be used to regain functional control of a joint or limb motion after injury. +.. One of the current procedures in the hand is the tendon transfer of Extensor Indicis Proprius (EIP) to replace the +.. Extensor Pollicis Longus (EPL) (Gelb (1995)). Rupture of the EPL can happen after a broken wrist and create a loss of control +.. of the Thumb extension. We introduce a physical tendon transfer where the EIP application point of the tendon was moved +.. from the index to the thumb and the EPL was removed (see Figure 3). + +.. .. image:: images/tendon_transfer.png +.. :width: 400 + +.. .. _exo: + +.. Exoskeleton assistance +.. ====================== +.. Exoskeleton assisted rehabilitation is becoming more and more common practice (Jezernik et al. (2003)) due to its multiple benefit (Nam et al. (2017)). +.. Modeling of an exoskeleton for the elbow was done via an ideal actuator and the addition of two supports with a weight of 0.101 Kg for the upper arm and 0.111 Kg on the forearm. The assistance given by the exoskeleton was a percentage of the biological joint torque, this was based on the neuromusculoskeletal controller presented in Durandau et al. (2019). + +.. .. image:: images/elbow_exo.png +.. :width: 200 + + +.. _task_and_variations: + + +Task and Variantions +************************************ + +For convenience, Myosuite has implemented a set of muscle decifiencies that currently includes: :ref:`test_muscle_fatigue` , sarcopenia and tendon transfer alongside with the tasks. + +Here are a list of the tasks currently implement in Myosuite. See :ref:`task_variation` on how to include the additional features. + + ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +|:ref:`tasks` | **Environment** | **Difficulty** |:ref:`test_sarcopenia` |:ref:`test_muscle_fatigue` | :ref:`test_tendon_transfer` | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Finger Joint Pose | ``myoFingerPoseFixed-v0`` | Easy | ✓ | ✓ | | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Finger Joint Pose | ``myoFingerPoseRandom-v0`` | Hard | ✓ | ✓ | | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Finger Tip Reach | ``myoFingerReachFixed-v0`` | Easy | ✓ | ✓ | | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Finger Tip Reach | ``myoFingerReachRandom-v0`` | Hard | ✓ | ✓ | | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Elbow Joint Pose | ``myoElbowPose1D6MRandom-v0`` | Hard | ✓ | ✓ | | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Hand Joints Pose | ``myoHandPoseFixed-v0`` | Easy | ✓ | ✓ | ✓ | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Hand Joints Pose | ``myoHandPoseRandom-v0`` | Hard | ✓ | ✓ | ✓ | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Hand Tips Reach | ``myoHandReachFixed-v0`` | Easy | ✓ | ✓ | ✓ | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Hand Tips Reach | ``myoHandReachRandom-v0`` | Hard | ✓ | ✓ | ✓ | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Hand Key Turn | ``myoHandKeyTurnFixed-v0`` | Easy | ✓ | ✓ | ✓ | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Hand Key Turn | ``myoHandKeyTurnRandom-v0`` | Hard | ✓ | ✓ | ✓ | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Hand Object Hold | ``myoHandObjHoldFixed-v0`` | Easy | ✓ | ✓ | ✓ | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Hand Object Hold | ``myoHandObjHoldRandom-v0`` | Hard | ✓ | ✓ | ✓ | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Hand Pen Twirl | ``myoHandPenTwirlFixed-v0`` | Easy | ✓ | ✓ | ✓ | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Hand Pen Twirl | ``myoHandPenTwirlRandom-v0`` | Hard | ✓ | ✓ | ✓ | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Die Rotation | ``myoChallengeDieReorientP1-v1`` | Easy | ✓ | ✓ | ✓ | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Die Rotation | ``myoChallengeDieReorientP2-v1`` | Hard | ✓ | ✓ | ✓ | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Hand Baoding Balls | ``myoChallengeBaodingP1-v1`` | Easy | ✓ | ✓ | ✓ | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Hand Baoding Balls | ``myoChallengeBaodingP2-v1`` | Hard | ✓ | ✓ | ✓ | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| 8 Obj. Rotation | ``myoHandReorient8-v0`` | Easy | ✓ | ✓ | ✓ | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| 100 Obj. Rotation | ``myoHandReorient100-v0`` | Easy | ✓ | ✓ | ✓ | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| 1000 Obj. Rotation | ``myoHandReorientID-v0`` | Hard | ✓ | ✓ | ✓ | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| 1000 Obj. Rotation | ``myoHandReorientOOD-v0`` | Hard | ✓ | ✓ | ✓ | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Leg walk | ``myoLegWalk-v0`` | Easy | ✓ | ✓ | | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Leg walk Rough Ground | ``myoLegRoughTerrainWalk-v0`` | Hard | ✓ | ✓ | | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Leg walk Hilly Ground | ``myoLegHillyTerrainWalk-v0`` | Hard | ✓ | ✓ | | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Leg walk Stair Ground | ``myoLegStairTerrainWalk-v0`` | Hard | ✓ | ✓ | | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Grasping & Placing | ``myoChallengeRelocateP1-v0`` | Easy | ✓ | ✓ | | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ +| Chase Tag | ``myoChallengeChaseTagP1-v0`` | Easy | ✓ | ✓ | | ++------------------------+----------------------------------+-----------------+-----------------------+---------------------------+--------------------------------+ + +.. _task_variation: + +Variations - **Sarcopenia**: myoSarc e.g. myoSarcHandPoseFixed-v0 - **Fatigue**: myoFati e.g. myoFatiElbowPose1D6MRandom-v0 - **TTransfer / Reafferentation**: myoReaf e.g. myoReafHandPoseFixed-v0 diff --git a/docs/source/tutorials.rst b/docs/source/tutorials.rst index e38ef55e..823c0e6e 100644 --- a/docs/source/tutorials.rst +++ b/docs/source/tutorials.rst @@ -5,17 +5,30 @@ Tutorials Here a set of examples on how to use different MyoSuite models and non-stationarities. -Jupyter-Notebooks can be found `here `__ +It is highly recommended to read through the `OpenAI Gym API `__ before you start. * :ref:`run_myosuite` * :ref:`run_visualize_index_movements` * :ref:`run_trained_policy` -* :ref:`test_muscle_fatigue` -* :ref:`test_sarcopenia` -* :ref:`test_tendon_transfer` -* :ref:`resume_training` -* :ref:`load_deprl_baseline` -* :ref:`load_MyoReflex_baseline` +* :ref:`advanced_muscle_conditions` + * :ref:`test_muscle_fatigue` + * :ref:`test_sarcopenia` + * :ref:`test_tendon_transfer` + * :ref:`exoskeleton` +* :ref:`use_reinforcement_learning` + * :ref:`resume_training` + * :ref:`load_deprl_baseline` + * :ref:`load_MyoReflex_baseline` +* :ref:`customizing_tasks` + +.. _jupyter_notebook: + +Tutorials on Jupyter-Notebook +======================================== +Please refer to our tutorials on Jupyter-Notebook `here `__ + +There are also tutorials for our ICRA workshops: `ICRA-2023 `__, `ICRA-2024 `__ + .. _run_myosuite: @@ -71,12 +84,30 @@ Example on using a policy e.g. elbow flexion, and change non-stationaries env.step(env.action_space.sample()) # take a random action +.. _advanced_muscle_conditions: + +Advanced Muscle Conditions +========================================= + +Besides from the simulation of healthy muscle conditions, Myosuite also provides features to simulate a number of muscle deficiencies. We aim provides a safe and trust-worthy environment for healthcare or rehabilitation strategies development leveraging the help of a simulator. .. _test_muscle_fatigue: -Test Muscle Fatigue -====================== -This example shows how to add fatigue to a model. It tests random actions on a model without and then with muscle fatigue. +Muscle Fatigue ++++++++++++++++++++++++++++++++++++++ +Muscle Fatigue is a short-term (second to minutes) effect that happens after sustained or repetitive voluntary movement +and it has been linked to traumas e.g. cumulative trauma disorder (Chaffin et al. (2006)). +A dynamic muscle fatigue model (Ma et al. (2009)) was integrated into the modeling framework. +This model was based on the idea that different types of muscle fiber that have different contributions +and resistance to fatigue (Vøllestad (1997)). +The current implementation is simplified to consider the same fatigue factor for all muscles and +that muscle can be completely fatigued. + +.. image:: images/Fatigue.png + :width: 800 + + +This example shows how to add fatigue to a model. The muscle force will gradually decrease as a result of repeated actions. It tests random actions on a model without and then with muscle fatigue. .. code-block:: python @@ -99,9 +130,15 @@ More advanced examples as well as detailed explanations can be found in `this tu .. _test_sarcopenia: -Test Sarcopenia -====================== -This example shows how to add sarcopenia or muscle weakness to a model. It tests random actions on a model without and then with muscle weakness. +Sarcopenia ++++++++++++++++++++++++++++++++++++++ + +Sarcopenia is a muscle disorder that occurs commonly in the elderly population (Cruz-Jentoft and Sayer (2019)) +and characterized by a reduction in muscle mass or volume. +The peak in grip strength can be reduced up to 50% from age 20 to 40 (Dodds et al. (2016)). +We modeled sarcopenia for each muscle as a reduction of 50% of its maximal isometric force. + +This example shows how to add sarcopenia or muscle weakness to a model. The maximum muscle force will be reduced. It tests random actions on a model without and then with muscle weakness. .. code-block:: python @@ -123,10 +160,20 @@ This example shows how to add sarcopenia or muscle weakness to a model. It tests .. _test_tendon_transfer: -Test Physical tendon transfer -============================== +Physical tendon transfer ++++++++++++++++++++++++++++++++++++++ +Contrary to muscle fatigue or sarcopenia that occurs to all muscles, tendon transfer surgery can target a single +muscle-tendon unit. Tendon transfer surgery allows redirecting the application point of muscle forces from one joint +DoF to another (see below). It can be used to regain functional control of a joint or limb motion after injury. +One of the current procedures in the hand is the tendon transfer of Extensor Indicis Proprius (EIP) to replace the +Extensor Pollicis Longus (EPL) (Gelb (1995)). Rupture of the EPL can happen after a broken wrist and create a loss of control +of the Thumb extension. We introduce a physical tendon transfer where the EIP application point of the tendon was moved +from the index to the thumb and the EPL was removed (see Figure 3). + +.. image:: images/tendon_transfer.png + :width: 400 -This example shows how load a model with physical tendon transfer. +This example shows how load a model with physical tendon transfer. This simulates a redirected muscle actuations .. code-block:: python @@ -145,11 +192,31 @@ This example shows how load a model with physical tendon transfer. env.step(env.action_space.sample()) # take a random action env.close() +.. _exoskeleton: + +Exoskeleton assistance ++++++++++++++++++++++++++++++++++++++ +Exoskeleton assisted rehabilitation is becoming more and more common practice (Jezernik et al. (2003)) due to its multiple benefit (Nam et al. (2017)). +Modeling of an exoskeleton for the elbow was done via an ideal actuator and the addition of two supports with a weight of 0.101 Kg for the upper arm and 0.111 Kg on the forearm. The assistance given by the exoskeleton was a percentage of the biological joint torque, this was based on the neuromusculoskeletal controller presented in Durandau et al. (2019). + + +The models and code will be released soon. + +.. image:: images/elbow_exo.png + :width: 200 + +.. _use_reinforcement_learning: + +Using Reinforcement Learning +============================================= +Myosuite provdies features to support RL training. Here are examples of using different RL libraries on Myosuite. +TODO: add contents to the sites of projects + .. _resume_training: Resume Learning of policies -============================== ++++++++++++++++++++++++++++++++++++++ When using ``mjrl`` it might be needed to resume training of a policy locally. It is possible to use the following instruction .. code-block:: bash @@ -159,7 +226,7 @@ When using ``mjrl`` it might be needed to resume training of a policy locally. I .. _load_deprl_baseline: Load DEP-RL Baseline -==================== ++++++++++++++++++++++++++++++++++++++ See `here `__ for more detailed documentation of ``deprl``. If you want to load and execute the pre-trained DEP-RL baseline. Make sure that the ``deprl`` package is installed. @@ -182,15 +249,16 @@ If you want to load and execute the pre-trained DEP-RL baseline. Make sure that .. _load_MyoReflex_baseline: Load MyoReflex Baseline -======================= ++++++++++++++++++++++++++++++++++++++ To load and execute the MyoReflex controller with baseline parameters. Run the MyoReflex tutorial `here `__ +.. _customizing_tasks: Customizing Tasks -================= +====================== In order to create a new customized task, there are two places where you need to act: @@ -201,7 +269,7 @@ In order to create a new customized task, there are two places where you need to Set up a new environment +++++++++++++++++++++++++ -Environment classes are developed according to the `OpenAI Gym definition `__ +Environment classes are developed according to the `OpenAI Gym definition `__ and contain all the information specific for a task, to interact with the environment, to observe it and to act on it. In addition, each environment class contains diff --git a/myosuite/version.py b/myosuite/version.py index ee42e181..54374234 100644 --- a/myosuite/version.py +++ b/myosuite/version.py @@ -1 +1 @@ -__version_tuple__ = (2, 3, 0) +__version_tuple__ = (2, 4, 0)