eus_vive

Robot remote control program with Vive/Oculus

Note

This package depends on these branches below:

For PR2 and Baxter
For Baxter only
- knorth55/baxter_interface@baxter-vive-control
- pazeshun/dynamixel_motor@gripper-v6-devel

Tested Environment

Build environment

Ubuntu 16.04 + ROS Kinetic
- NVidia driver: 396.37
- OpenVR: 1.3.22
- Steam VR: 1.6.10
Ubuntu 18.04 + ROS Melodic
- NVidia driver: 390.116
- OpenVR: 1.3.22
- Steam VR: 1.6.10

VR Devices

Vive
- Arm motion tracking
- Controller button interface
- HMD visual interface
- Vibration interface
- Sound interface
Oculus
- Arm motion tracking
- Controller button interface
- HMD visual interface
- Vibration interface
- Sound interface

Installation

Dependency installation

Install dependencies (for Vive)

sudo apt-get install --reinstall xserver-xorg-video-intel libgl1-mesa-glx libgl1-mesa-dri xserver-xorg-core
sudo dpkg-reconfigure xserver-xorg

Install nvidia-driver (for Vive)

# for melodic, run command below
sudo apt install nvidia-driver-390
# for kinetic, install cuda-9.2 deb (local) manually and run command below.
sudo apt install nvidia-396

Install `OpenVR`, `steam` and `steamVR` (for Vive)

Follow here

Install kodak 4k pro camera and ELP usb camera udev (for Baxter)

sudo cp udev/99-kodak.rules /etc/udev/rules.d/
sudo udevadm control --reload-rules && sudo udevadm trigger

ROS Workspace build

Build `jsk_apc` workspace (Only for baxter users)

source /opt/ros/$ROS_DISTRO/setup.bash
mkdir ~/jsk_apc_ws/src -p
cd ~/jsk_apc_ws/src
wstool init . https://raw.githubusercontent.com/start-jsk/jsk_apc/master/fc.rosinstall.${ROS_DISTRO}
wstool up
rosdep install -y -r --from-paths .

cd ~/jsk_apc_ws 
catkin config
catkin build

Build `eus_vive` workspace

source /opt/ros/$ROS_DISTRO/setup.bash
mkdir ~/vive_ws/src -p
cd ~/vive_ws/src
wstool init . https://raw.githubusercontent.com/knorth55/eus_vive/master/fc.rosinstall

# Only for baxter users
# wstool merge https://raw.githubusercontent.com/knorth55/eus_vive/master/baxter.rosinstall
# wstool merge https://raw.githubusercontent.com/knorth55/eus_vive/master/baxter.rosinstall.$ROS_DISTRO

wstool up
rosdep install --ignore-src --from-path . -y -r -i

# Only for baxter users
# source ~/jsk_apc_ws/devel/setup.bash

cd ~/vive_ws
catkin config
catkin build

How to start

Start procedure

Connect Vive HMD and Lighthouse and power on the controller.
Place Lighthouse B in front of you.
Start one of launch files below.
If you don't use HMD, press Calibrate button and do arm calibration.
Press Enable button in GUI to start teleoperation.

Arm calibration (Only for no HMD mode)

The arm calibration is required for no HMD mode in order to measure your position and your arm length.

Press Calibrate l/rarm button in GUI
Listen to the voice instruction.
Stretch your left/right arm and Press Trigger after the first instruction.
Listen to the voice instruction again.
Fold your left/right arm and Press Trigger after the second instruction.
If calibration failed, please try again.

PR2 + Vive

PR1040 in JSK 73B2

rossetip
rossetmaster pr1040
# HMD mode
roslaunch eus_vive pr1040_vive.launch
# No HMD mode
roslaunch eus_vive pr1040_vive.launch head:=false

PR1012 in JSK 610

rossetip
rossetmaster pr1012
# HMD mode
roslaunch eus_vive pr1012_vive.launch
# No HMD mode
roslaunch eus_vive pr1012_vive.launch head:=false

Gazebo

roslaunch eus_vive pr2_vive_gazebo.launch

Baxter + Vive

Real Robot in JSK 73B2

rossetip
rossetmaster baxter
# HMD mode
roslaunch eus_vive baxter_vive.launch
# No HMD mode
roslaunch eus_vive baxter_vive.launch head:=false

Gazebo

roslaunch baxter_gazebo baxter_world.launch
roslaunch eus_vive vive.launch
roslaunch eus_vive baxter_vive_gazebo.launch

Baxter + MoveIt!

Real Robot in JSK 73B2

rossetip
rossetmaster baxter
roslaunch eus_vive baxter_moveit.launch

Demo & Experiments

Miraikan Demo 2019/08/23-24

rossetip
rossetmaster baxter
roslaunch eus_vive baxter_vive_mirror.launch

Miraikan Demo 2020/09/11-13

Robot control PC (Robot side)

rossetip
rossetmaster baxter
roslaunch eus_vive baxter_remote.launch

Vive control PC (Pilot side)

rossetip
rossetmaster baxter
roslaunch eus_vive baxter_vive_remote.launch

Visualization, display and feedback PC (Pilot side)

rossetip
rossetmaster baxter
roslaunch eus_vive baxter_display_remote.launch

How to use Vive controller

PR2

Button	Usage
1 / Menu	Control toggle: base/arm (Default: base)
3 / Stream Menu	Steam Menu
8 / Grip	Not used

Arm mode

You can enable arm mode of right and left arm separately.

Command	Usage
7 / Trigger	Gripper toggle: open/close in Toggle grasp mode (Default: open)
7 / Trigger	Gripper toggle: open only when trigger is pressed in Hold grasp mode
Controller pose	robot end effector's pose

Base mode

Base mode is enabled when both arms are disabled in Arm mode.

Command	Usage
2 / Trackpad	Torso control: right: down / left: up
2 / Trackpad + 7 / Trigger (right)	Safe base control: right: x, y / left: w
2 / Trackpad + 7 / Trigger (right + left)	Unsafe base control: right: x, y / left: w

Baxter

Button	Usage
2 / Trackpad	Control toggle: stop / arm (Default: stop)
3 / Stream Menu	Steam Menu
8 / Grip	Not used

Arm mode

You can enable arm mode of right and left arm separately.

Command	Usage
7 / Trigger	Gripper toggle: open/close (Default: open)
Controller pose	robot end effector's pose

How to use GUI control

Teleoperation control buttons

Button	Usage
Reset	Diable and reset robot to initial posture
Enable	Enable robot
Disable	Disable robot
Calibrate larm/rarm	Calibrate with left/right arm

Arm & Gripper control buttons

Button	Usage
Reset larm/rarm	Reset only left/right arm
Enable larm/rarm	Enable only left/right arm
Disable larm/rarm	Disable only left/right arm
Start grasp lgripper/rgripper	Start grasp only left/right gripper
Stop grasp lgripper/rgripper	Stop grasp only left/right gripper
Toggle grasp mode	Change to Toggle grasp mode
Hold grasp mode	Change to Hold grasp mode

Tips

Baxter Network Configuration

Open Field Service Menu and change network configuration

VPNC Install

sudo apt install libgcrypt20-dev libgnutls28-dev
git clone https://github.com/streambinder/vpnc.git
cd vpnc
git checkout 1cf24ed6aa4a04b4b01cc9ebfacbad723eed04f5
make
sudo make install
cd ..
git clone git://git.infradead.org/users/dwmw2/vpnc-scripts.git
cd vpnc-scripts
sudo cp vpnc-script /etc/vpnc

VPNC Command for Fortigate

sudo vpnc --local-port 0 --gateway <gateway> --id ipsecvpn --username <username> --pfs dh5 --dh dh5 --auth-mode psk --no-detach --vendor fortigate --dh dh5

Name		Name	Last commit message	Last commit date
Latest commit History 720 Commits
.github		.github
config		config
euslisp		euslisp
launch		launch
msg		msg
node_scripts		node_scripts
resource		resource
rvizconfig		rvizconfig
sample		sample
scripts		scripts
sounds		sounds
udev		udev
urdf		urdf
.ci.rosinstall		.ci.rosinstall
CHANGELOG.rst		CHANGELOG.rst
CMakeLists.txt		CMakeLists.txt
README.md		README.md
baxter.rosinstall		baxter.rosinstall
baxter.rosinstall.kinetic		baxter.rosinstall.kinetic
baxter.rosinstall.melodic		baxter.rosinstall.melodic
fc.rosinstall		fc.rosinstall
package.xml		package.xml

tohirose/eus_vive

Folders and files

Latest commit

History

Repository files navigation

eus_vive

Note

Tested Environment

Build environment

VR Devices

Installation

Dependency installation

Install dependencies (for Vive)

Install nvidia-driver (for Vive)

Install OpenVR, steam and steamVR (for Vive)

Install kodak 4k pro camera and ELP usb camera udev (for Baxter)

ROS Workspace build

Build jsk_apc workspace (Only for baxter users)

Build eus_vive workspace

How to start

Start procedure

Arm calibration (Only for no HMD mode)

PR2 + Vive

PR1040 in JSK 73B2

PR1012 in JSK 610

Gazebo

Baxter + Vive

Real Robot in JSK 73B2

Gazebo

Baxter + MoveIt!

Real Robot in JSK 73B2

Demo & Experiments

Miraikan Demo 2019/08/23-24

Miraikan Demo 2020/09/11-13

Robot control PC (Robot side)

Vive control PC (Pilot side)

Visualization, display and feedback PC (Pilot side)

How to use Vive controller

PR2

Arm mode

Base mode

Baxter

Arm mode

How to use GUI control

Teleoperation control buttons

Arm & Gripper control buttons

Tips

Baxter Network Configuration

VPNC Install

VPNC Command for Fortigate

Demo Video

PR2 Fridge demo

Baxter APC demo

About

Resources

Stars

Watchers

Forks

Releases

Packages 0

Languages

Install `OpenVR`, `steam` and `steamVR` (for Vive)

Build `jsk_apc` workspace (Only for baxter users)

Build `eus_vive` workspace

Packages