Merge pull request #142 from MyoHub/dev

Gymnasium Upgrade
MyoHub · May 1, 2024 · 4e05093 · 4e05093
2 parents 4d146e1 + a0bd8de
commit 4e05093
Show file tree

Hide file tree

Showing 59 changed files with 1,396 additions and 296 deletions.
diff --git a/.github/CHANGELOG.md b/.github/CHANGELOG.md
@@ -5,6 +5,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
+[FEATURE] Support for both Gym/Gymnasium (#142)
+[FEATURE] Add support for TorchRL by @vmoens (5efdf93)
+
+## [2.2.0] - 2024-01-20
+[FEATURE] Inverse dynamics tutorial. Thanks to @andreh1111 #121
+[RELEASE] MyoArm and MyoLeg models (4c01023, cd9a25e)
+[RELEASE] MyoChallenge'23 environments release (#128)
+[BUGFIX] Fixed heightfield collisions for myoleg scenes #132
+[BUGFIX] Fixed names of data keys from _int to _init in myodm by @andreh1111 in (#119)
 
 ## [1.3.0] - 2023-01-11
 - Rebase and building on RoboHive v0.3

diff --git a/.github/workflows/python-app.yml b/.github/workflows/python-app.yml
@@ -42,7 +42,7 @@ jobs:
       run: echo "SYSTEM_VERSION_COMPAT=0" >> $GITHUB_ENV
 
     - name: Checkout repository and submodules
-      uses: actions/checkout@v3
+      uses: actions/checkout@v4
       with:
         submodules: recursive
 
@@ -60,14 +60,14 @@ jobs:
 
     - name: Enable Conda (macOS)
       if: ${{ runner.os == 'macOS' }}
-      uses: conda-incubator/setup-miniconda@v2
+      uses: conda-incubator/setup-miniconda@v3
       with:
         auto-activate-base: false
         python-version: 3.8
 
     - name: Enable Conda (window/ubuntu)
       if: ${{ runner.os != 'macOS' }}
-      uses: conda-incubator/setup-miniconda@v2
+      uses: conda-incubator/setup-miniconda@v3
       with:
         auto-activate-base: false
 
@@ -114,5 +114,9 @@ jobs:
     - name: Run Test jupyter-notebooks tutorials
       if: ${{ runner.os == 'Linux' }}
       run: |
-        conda activate $CONDA_DEFAULT_ENV
-        source myosuite/tests/test_tutorials.sh
+        source myosuite/tests/test_tutorials.sh
+
+    - name: Run Test gym/gymnasium + stable-baselines3
+      if: ${{ runner.os == 'Linux' }}
+      run: |
+        source myosuite/tests/test_versions.sh
diff --git a/README.md b/README.md
@@ -11,7 +11,6 @@ Authors  :: Vikash Kumar ([email protected]), Vittorio Caggiano (caggiano@gma
 [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/myohub/myosuite/blob/main/docs/CONTRIBUTING.md)
 [![Downloads](https://static.pepy.tech/badge/myosuite)](https://pepy.tech/project/myosuite)
 [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1zFuNLsrmx42vT4oV8RbnEWtkSJ1xajEo)
-[![Supporting MyoChallenge](https://img.shields.io/static/v1?label=Supporting&message=NeurIPS-MyoChallenge2023&color=informational&link=https://github.com/MyoHub/myochallenge_2023eval)](https://github.com/MyoHub/myochallenge_2023eval)
 [![Slack](https://img.shields.io/badge/Slack-4A154B?style=for-the-badge&logo=slack&logoColor=white)](https://join.slack.com/t/myosuite/shared_invite/zt-1zkpw2zzk-NhVhVlSDxhoMHbzROD8gMA)
 [![Twitter Follow](https://img.shields.io/twitter/follow/MyoSuite?style=social)](https://twitter.com/MyoSuite)
 
@@ -59,8 +58,7 @@ It is possible to create and interface with MyoSuite environments just like any
 
 
 ```python
-import myosuite
-import gym
+from myosuite.utils import gym
 env = gym.make('myoElbowPose1D6MRandom-v0')
 env.reset()
 for _ in range(1000):
@@ -69,7 +67,8 @@ for _ in range(1000):
 env.close()
 ```
 
-You can find [tutorials](https://github.com/myohub/myosuite/tree/main/docs/source/tutorials#tutorials) on how to load MyoSuite models/tasks, train them, and visualize their outcome. Also, you can find [baselines](https://github.com/myohub/myosuite/tree/main/myosuite/agents) to test some pre-trained policies.
+You can find [tutorials](https://github.com/myohub/myosuite/tree/main/docs/source/tutorials#tutorials) and the **ICRA2023 Colab Tutorial** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1KGqZgSYgKXF-vaYC33GR9llDsIW9Rp-q)
+on how to load MyoSuite models/tasks, train them, and visualize their outcome. Also, you can find [baselines](https://github.com/myohub/myosuite/tree/main/myosuite/agents) to test some pre-trained policies.
 
 
 ## License

diff --git a/docs/source/install.rst b/docs/source/install.rst
@@ -60,8 +60,7 @@ For example, to use the ``myoElbowPose1D6MRandom-v0`` environment it is possible
 
 .. code-block:: python
 
-   import myosuite
-   import gym
+   from myosuite.utils import gym
    env = gym.make('myoElbowPose1D6MRandom-v0')
    env.reset()
    for _ in range(1000):

diff --git a/docs/source/tutorials.rst b/docs/source/tutorials.rst
@@ -25,8 +25,7 @@ Example on how to use an environment e.g. send random movements
 
 .. code-block:: python
 
-    import myosuite
-    import gym
+    from myosuite.utils import gym
     env = gym.make('myoElbowPose1D6MRandom-v0')
     env.reset()
     for _ in range(1000):
@@ -43,8 +42,7 @@ Example on how to generate and visualize a movement e.g. index flexion, and visu
 
 .. code-block:: python
 
-    import myosuite
-    import gym
+    from myosuite.utils import gym
     env = gym.make('myoHandPoseRandom-v0')
     env.reset()
     for _ in range(1000):
@@ -60,8 +58,7 @@ Example on using a policy e.g. elbow flexion, and change non-stationaries
 
 .. code-block:: python
 
-    import myosuite
-    import gym
+    from myosuite.utils import gym
     policy = "iterations/best_policy.pickle"
 
     import pickle
@@ -83,8 +80,7 @@ This example shows how to add fatigue to a model. It tests random actions on a m
 
 .. code-block:: python
 
-    import myosuite
-    import gym
+    from myosuite.utils import gym
     env = gym.make('myoElbowPose1D6MRandom-v0')
     env.reset()
     for _ in range(1000):
@@ -108,8 +104,7 @@ This example shows how to add sarcopenia or muscle weakness to a model. It tests
 
 .. code-block:: python
 
-    import myosuite
-    import gym
+    from myosuite.utils import gym
     env = gym.make('myoElbowPose1D6MRandom-v0')
     env.reset()
     for _ in range(1000):
@@ -134,8 +129,7 @@ This example shows how load a model with physical tendon transfer.
 
 .. code-block:: python
 
-    import myosuite
-    import gym
+    from myosuite.utils import gym
     env = gym.make('myoHandKeyTurnFixed-v0')
     env.reset()
     for _ in range(1000):
@@ -171,8 +165,7 @@ If you want to load and execute the pre-trained DEP-RL baseline. Make sure that
 
 .. code-block:: python
 
-    import gym
-    import myosuite
+    from myosuite.utils import gym
     import deprl
 
     # we can pass arguments to the environments here
@@ -192,3 +185,79 @@ Load MyoReflex Baseline
 
 To load and execute the MyoReflex controller with baseline parameters.
 Run the MyoReflex tutorial `here <https://github.com/facebookresearch/myosuite/tree/main/docs/source/tutorials/4b_reflex>`__
+
+
+
+Customizing Tasks
+=================
+
+In order to create a new customized task, there are two places where you need to act:
+
+1. Set up a new environment class for the new task
+
+2. Register the new task
+
+Set up a new environment
++++++++++++++++++++++++++
+
+Environment classes are developed according to the `OpenAI Gym definition <https://www.gymlibrary.dev/content/environment_creation/>`__
+and contain all the information specific for a task,
+to interact with the environment, to observe it and to
+act on it. In addition, each environment class contains
+a reward function which converts the observation into a
+number that establishes how good the observation is with
+respect to the task objectives. In order to create a new
+task, a new environment class needs to be generated eg.
+reach2_v0.py (see for example how `reach_v0.py <https://github.com/MyoHub/myosuite/blob/main/myosuite/envs/myo/myobase/reach_v0.py>`__ is structured).
+In this file, it is possible to specify the type of observation (eg. joint angles, velocities, forces), actions (e.g. muscle, motors), goal, and reward.
+
+
+.. code-block:: python
+
+    from myosuite.envs.myo.base_v0 import BaseV0
+
+    # Class extends Basev0
+    class NewReachEnvV0(BaseV0):
+        ....
+
+    # defines the observation
+    def get_obs_dict(self, sim):
+        ....
+
+    # defines the rewards
+    def get_reward_dict(self, obs_dict):
+        ...
+
+    #reset condition that
+    def reset(self):
+        ...
+
+.. _setup_base_class:
+
+
+Register the new environment
+++++++++++++++++++++++++++++++
+
+Once defined the task `reach2_v0.py`, the new environment needs to be registered to be
+visible when importing `myosuite`. This is achieved by introducing the new environment in
+the `__init__.py` (called when the library is imported) where the registration routine happens.
+The registration of the new enviornment is obtained adding:
+
+.. code-block:: python
+
+    from gym.envs.registration import register
+
+    register(id='newReachTask-v0',
+        entry_point='myosuite.envs.myo.myobase.reach_v0:NewReachEnvV0', # where to find the new Environment Class
+        max_episode_steps=200, # duration of the episode
+        kwargs={
+            'model_path': curr_dir+'/../assets/hand/myohand_pose.xml', # where the xml file of the environment is located
+            'target_reach_range': {'IFtip': ((0.1, 0.05, 0.20), (0.2, 0.05, 0.20)),}, # this is used in the setup to define the goal e.g. rando position of the team between 0.1 and 0.2 in the x coordinates
+            'normalize_act': True, # if to use normalized actions using a sigmoid function.
+            'frame_skip': 5, # collect a sample every 5 iteration step
+        }
+    )
+
+
+.. _register_new_environment:
+
diff --git a/docs/source/tutorials/1_Get_Started.ipynb b/docs/source/tutorials/1_Get_Started.ipynb
@@ -22,8 +22,7 @@
     }
    ],
    "source": [
-    "import myosuite\n",
-    "import gym\n",
+    "from myosuite.utils import gym\n",
     "import skvideo.io\n",
     "import numpy as np\n",
     "import os"

diff --git a/docs/source/tutorials/2_Load_policy.ipynb b/docs/source/tutorials/2_Load_policy.ipynb
@@ -6,8 +6,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import myosuite\n",
-    "import gym\n",
+    "from myosuite.utils import gym\n",
     "import skvideo.io\n",
     "import numpy as np\n",
     "import os"
@@ -79,7 +78,7 @@
     "        frames.append(frame)\n",
     "        o = env.get_obs()\n",
     "        a = pi.get_action(o)[0]\n",
-    "        next_o, r, done, ifo = env.step(a) # take an action based on the current observation\n",
+    "        next_o, r, done, *_, ifo = env.step(a) # take an action based on the current observation\n",
     "env.close()\n",
     "\n",
     "os.makedirs('videos', exist_ok=True)\n",

diff --git a/docs/source/tutorials/3_Analyse_movements.ipynb b/docs/source/tutorials/3_Analyse_movements.ipynb
@@ -6,8 +6,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import myosuite\n",
-    "import gym\n",
+    "from myosuite.utils import gym\n",
     "import skvideo.io\n",
     "import numpy as np\n",
     "import os"
@@ -45,7 +44,7 @@
     "    for _ in range(100): # 100 samples for each episode\n",
     "        o = env.get_obs()\n",
     "        a = pi.predict(o)[0]\n",
-    "        next_o, r, done, ifo = env.step(a) # take a random action\n",
+    "        next_o, r, done, *_, ifo = env.step(a) # take a random action\n",
     "                    \n",
     "        data_store.append({\"action\":a.copy(), \n",
     "                            \"jpos\":env.sim.data.qpos.copy(), \n",

diff --git a/docs/source/tutorials/4_Train_policy.ipynb b/docs/source/tutorials/4_Train_policy.ipynb
@@ -6,8 +6,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import myosuite\n",
-    "import gym\n",
+    "from myosuite.utils import gym\n",
     "import skvideo.io\n",
     "import numpy as np\n",
     "import os"
@@ -122,7 +121,7 @@
     "        frames.append(frame[::-1,:,:])\n",
     "        o = env.get_obs()\n",
     "        a = pi.get_action(o)[0]\n",
-    "        next_o, r, done, ifo = env.step(a) # take an action based on the current observation\n",
+    "        next_o, r, done, *_, ifo = env.step(a) # take an action based on the current observation\n",
     "env.close()\n",
     "\n",
     "os.makedirs('videos', exist_ok=True)\n",

diff --git a/docs/source/tutorials/4a_deprl.ipynb b/docs/source/tutorials/4a_deprl.ipynb
@@ -29,10 +29,8 @@
     }
    ],
    "source": [
-    "\n",
-    "import gym, myosuite\n",
-    "import deprl\n",
-    "import gym"
+    "from myosuite.utils import gym\n",
+    "import deprl"
    ]
   },
   {
@@ -75,10 +73,10 @@
    "source": [
     "policy = deprl.load_baseline(env)\n",
     "\n",
-    "obs = env.reset()\n",
+    "obs = env.reset()[0]\n",
     "for _ in range(T):\n",
     "    action = policy(obs)\n",
-    "    obs, rew, done, info = env.step(action)\n",
+    "    obs, rew, done, *_, info = env.step(action)\n",
     "    if done:\n",
     "        break\n",
     "env.close()\n",
@@ -95,13 +93,6 @@
    "source": [
     "If you want to load your own policy, replace `deprl.load_baseline(env)` with `deprl.load(path, env)`. This will automatically load it for you. We also provide more advanced commands like `python -m deprl.play --path /folder/` to setup a visualization script."
    ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": []
   }
  ],
  "metadata": {

diff --git a/docs/source/tutorials/4c_Train_SB_policy.ipynb b/docs/source/tutorials/4c_Train_SB_policy.ipynb
@@ -6,8 +6,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import myosuite\n",
-    "import gym\n",
+    "from myosuite.utils import gym\n",
     "import skvideo.io\n",
     "import numpy as np\n",
     "import os"
@@ -92,7 +91,7 @@
     "        frames.append(frame[::-1,:,:])\n",
     "        o = env.get_obs()\n",
     "        a = pi.predict(o)[0]\n",
-    "        next_o, r, done, ifo = env.step(a) # take an action based on the current observation\n",
+    "        next_o, r, done, *_, ifo = env.step(a) # take an action based on the current observation\n",
     "env.close()\n",
     "\n",
     "os.makedirs('videos', exist_ok=True)\n",

diff --git a/docs/source/tutorials/5_Move_Hand_Fingers.ipynb b/docs/source/tutorials/5_Move_Hand_Fingers.ipynb
@@ -6,8 +6,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import myosuite\n",
-    "import gym\n",
+    "from myosuite.utils import gym\n",
     "import skvideo.io\n",
     "import numpy as np\n",
     "import os"