Hand-E Robotiq Gripper in MuJoCo - Control of Position and Force #1988

vmstavens · 2024-05-15T08:27:04Z

vmstavens
May 15, 2024

Hi MuJoCo community,

I'm a research assistant and I'm trying to use MuJoCo for grasping and manipulation. Specifically, I am attempting to grasp objects using the Hand-E Robotiq gripper such that the simulated gripper closely follows the real gripper's behavior. My current implementation is based on this URDF and uses a tendon and equalities for moving both fingers with one actuator. The meshes can be found here.

I'm looking for some help with building the MJCF file such that the gripper fulfills the following criteria:

I need the sliding fingers to be within the joint range (0, 0.025), and this constraint should be respected, meaning very little if any overshoot. This position control should be mapped to the control signal (0, 255). This means that at
$$ctrl = 0 \rightarrow qpos = 0 \quad \text{and} \quad ctrl = 255 \rightarrow qpos = 0.025 \quad \text{and} \quad ctrl = 127 \rightarrow qpos = 0.0125$$
The joints should be very robust to external forces, meaning when I send $ctrl = 255$ and the gripper is closed, a large force should be necessary to pry the gripper open
Third and finally, if an object causes the gripper to not reach its desired position, the force exerted by the motor should increase until the forcerange max is achieved.

The scene file can be found below. As you can see, I have attempted to replicate the behavior from the franka panda gripper, since they in principle are similar, however, only my position and not my force criteria are met in this model.

minimal XML

<mujoco model="scene">
  <compiler angle="radian"/>
  <option gravity="0 0 -9.82" integrator="RK4" cone="elliptic"/>
  <visual>
    <global azimuth="120" elevation="-20"/>
    <headlight diffuse="0.6 0.6 0.6" specular="0 0 0"/>
    <rgba haze="0.15 0.25 0.35 1"/>
  </visual>
  <statistic meansize="0.1" extent="1" center="4 0 0"/>
  <default class="main">
    <default class="hande">
      <default class="hande_right_finger">
        <joint type="slide" range="0 0.025" actuatorfrcrange="-100 100"/>
      </default>
      <default class="hande_left_finger">
        <joint type="slide" range="0 0.025" actuatorfrcrange="-100 100"/>
      </default>
    </default>
  </default>
  <asset>
    <texture type="skybox" builtin="gradient" rgb1="0.3 0.5 0.7" rgb2="0 0 0" width="512" height="3072"/>
    <texture type="2d" name="groundplane" builtin="checker" mark="edge" rgb1="0.2 0.3 0.4" rgb2="0.1 0.2 0.3" markrgb="0.8 0.8 0.8" width="300" height="300"/>
    <material name="groundplane" texture="groundplane" texuniform="true" texrepeat="5 5" reflectance="0.2"/>
    <mesh name="hande" file="../assets/hande/assets/hande.stl"/>
    <mesh name="coupler" file="../assets/hande/assets/coupler.stl"/>
    <mesh name="right_finger" file="../assets/hande/assets/finger_1.stl"/>
    <mesh name="left_finger" file="../assets/hande/assets/finger_2.stl"/>
  </asset>
  <worldbody>
    <geom name="floor" size="0 0 0.05" type="plane" material="groundplane"/>
    <light pos="0 0 1.5" dir="0 0 -1" directional="true"/>
    <body name="hande">
      <joint type="free" limited="false" actuatorfrclimited="false"/>
      <geom name="hande_coupler" type="mesh" contype="0" conaffinity="0" group="1" rgba="0.1 0.1 0.1 1" mesh="coupler"/>
      <geom name="hande_base" type="mesh" contype="0" conaffinity="0" group="1" rgba="0.1 0.1 0.1 1" mesh="hande"/>
      <site name="tcp" pos="0 0 0.135"/>
      <body name="hande_left_finger" pos="0 0 0.099">
        <inertial pos="0.02262 -0.00759 0.00738" quat="0.871952 0.0369732 -0.211073 0.440205" mass="0.03804" diaginertia="1.22763e-05 1.22763e-05 1.22763e-05"/>
        <joint name="hande_left_finger_joint" class="hande_left_finger" pos="0 0 0" axis="1 0 0"/>
        <geom type="mesh" contype="0" conaffinity="0" group="1" rgba="0.4 0.4 0.4 1" mesh="left_finger"/>
      </body>
      <body name="hande_right_finger" pos="0 0 0.099">
        <inertial pos="-0.02262 0.00759 0.00738" quat="0.871952 -0.0369732 0.211073 0.440205" mass="0.03804" diaginertia="1.22763e-05 1.22763e-05 1.22763e-05"/>
        <joint name="hande_right_finger_joint" class="hande_right_finger" pos="0 0 0" axis="-1 0 0"/>
        <geom type="mesh" contype="0" conaffinity="0" group="1" rgba="0.4 0.4 0.4 1" mesh="right_finger"/>
      </body>
    </body>
    <body name="mocap" mocap="true">
      <geom name="mocap" size="0.02 0.02 0.02" type="box" contype="0" conaffinity="0"/>
    </body>
  </worldbody>
  <contact>
    <exclude body1="hande" body2="hande_left_finger"/>
    <exclude body1="hande" body2="hande_right_finger"/>
  </contact>
  <equality>
    <joint joint1="hande_left_finger_joint" joint2="hande_right_finger_joint" polycoef="0 1 0 0 0"/>
    <weld body1="hande" body2="mocap" anchor="0 0 0" torquescale="1" relpose="0 0 0 1 0 0 0" solref="0.001" solimp="1 1 1 1 5"/>
  </equality>
  <tendon>
    <fixed name="split">
      <joint joint="hande_left_finger_joint" coef="0.5"/>
      <joint joint="hande_right_finger_joint" coef="0.5"/>
    </fixed>
  </tendon>
  <actuator>
	<!-- Remap original ctrlrange (0, 0.025) to (0, 255): 0.025 * 100 / 255 = 0.009803922 -->
	<!-- https://assets.robotiq.com/website-assets/support_documents/document/online/Hand-E_Aubo_InstructionManual_HTML5_20190501.zip/Hand-E_Aubo_InstructionManual_HTML5/Content/6.%20Specifications.htm -->
	<!-- https://mujoco.readthedocs.io/en/stable/XMLreference.html#actuator -->
	<!-- scalar_force = gain_term * (act or ctrl) + bias_term -->
	<!-- gain_term = gain_prm[0] + gain_prm[1]*length + gain_prm[2]*velocity -->
	<!-- bias_term = biasprm[0] + biasprm[1]*length + biasprm[2]*velocity -->
    <general name="hande_fingers_actuator" class="hande" tendon="split" ctrlrange="0 255" forcerange="-200 200" gaintype="affine" biastype="affine" gainprm="0.00980392" biasprm="0 -100 -10"/>
  </actuator>
</mujoco>

Here is a GIF, illustrating the current behavior of my gripper:

As you can see the position criteria are fulfilled quite nicely, however when obstacles are introduced, the gripper exerts about 0.8N of force (measured from data.actuator("hande_fingers_actuator").force)

All feedback and/or ideas for how I could progress/solve this problem of mine are very much appreciated!

kevinzakka · 2024-05-27T02:04:33Z

kevinzakka
May 27, 2024
Collaborator

Hi @vmstavens, how about you submit a PR to MuJoCo Menagerie and we can work on the model together?

0 replies

vmstavens · 2024-06-03T21:38:03Z

vmstavens
Jun 3, 2024
Author

Hi @kevinzakka, sorry for the late response, that sounds great I will make a PR to Menagerie :)

0 replies

vmstavens · 2024-06-04T08:22:14Z

vmstavens
Jun 4, 2024
Author

The fork can now be found here

0 replies

kevinzakka · 2024-06-04T21:36:38Z

kevinzakka
Jun 4, 2024
Collaborator

Hi @vmstavens, there's nothing in that fork?

0 replies

vmstavens · 2024-06-05T07:58:00Z

vmstavens
Jun 5, 2024
Author

My apologies, thank you for the correction it was a mistake on my part, does it work correctly now? I have added this folder with the mesh files, scene.xml and hande.xml.

1 reply

wukongwong Feb 7, 2025

Is this problem solved? I'm having the same problem.

wukongwong · 2025-02-07T03:18:26Z

wukongwong
Feb 7, 2025

Is this problem solved? I'm having the same problem.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hand-E Robotiq Gripper in MuJoCo - Control of Position and Force #1988

{{title}}

Replies: 6 comments 1 reply

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Hand-E Robotiq Gripper in MuJoCo - Control of Position and Force #1988

vmstavens May 15, 2024

Replies: 6 comments · 1 reply

kevinzakka May 27, 2024 Collaborator

vmstavens Jun 3, 2024 Author

vmstavens Jun 4, 2024 Author

kevinzakka Jun 4, 2024 Collaborator

vmstavens Jun 5, 2024 Author

wukongwong Feb 7, 2025

wukongwong Feb 7, 2025

vmstavens
May 15, 2024

Replies: 6 comments 1 reply

kevinzakka
May 27, 2024
Collaborator

vmstavens
Jun 3, 2024
Author

vmstavens
Jun 4, 2024
Author

kevinzakka
Jun 4, 2024
Collaborator

vmstavens
Jun 5, 2024
Author

wukongwong
Feb 7, 2025