Bipedal Walker

This is how to teach a bot to walk on two feet.

Bipedal Walker is an OpenAI Gym environment where an agent learns to control a bipedal 2D character to reach the end of an obstacle course. What makes this challenging is that the agent only receives limbs coordinates along with the Lidar information. The agent has to learn to balance, walk, run, jump on its own without any human intervention.

There are two versions:

• Normal, with slightly uneven terrain.

• Hardcore, with ladders, stumps, pitfalls.

To solve the normal version, you need to get 300 points in 1600 time steps. To solve the hardcore version, you need 300 points in 2000 time steps.

The code in this repo solves Bipedal Walker V3 in normal mode (hardcore=False).

Prerequisites and Recommended Tools

• Conda installed on your system
• Visual Studio Code

Setup Instructions

Follow the steps below to set up your environment and install the necessary dependencies.

1. Create a Conda Environment

First, create a new Conda environment with Python 3.11:

conda create -n gym-env python=3.11 -y

2. Activate the newly created environment

conda activate gym-env

3. Install the required packages

conda install \
    swig gymnasium-box2d \
    stable-baselines3 tensorboard \
    plotly scikit-learn optuna -y

4. Clone this github repository.

Once the environment is set up and dependencies are installed, you can proceed to train the agent and evaluate its performance.

Project Structure

• bipedal_train.py: Train the agent with PPO default parameters.
• bipedal_eval.py: Evaluate the agent using a previously saved model.
• optuna_ppo.py: Auto-tune PPO using OPTUNA.
• optuna_ddpg.py: Auto-tune DDPG using OPTUNA.
• bipedal_wandb_ppo.py: Train / evaluate the agent with tuned hyper-parameters + W&B integration.
• bipedal_wandb_ddpg.py: Train / evaluate the agent with tuned hyper-parameters + W&B integration.
• record_video.py: Record a video of the trained agent.

Tensorboard Visualization

tensorboard --logdir=./training/logs

Additional Information

• Stable Baselines3 Documentation
• Gymnasium Documentation
• TensorBoard Documentation

License

This project is licensed under the MIT License.