HotWheelsRL (wip)

Zack Beucler

Use RL to train an agent to competitively complete a race on the first level in the GBA game 'Hot Wheels Stunt Track Challenge'
Agent should be be able to complete a lap

Resources

stable_baselines
stable-baselines3
stable-baselines3-contrib
Exploration Strategies for Deep Reinforcement Learning
stable-retro
NOTE: actual_playing.state is a save file of other maps and challenges in the game.

	discrete	multidiscrete	multibinary
PPO	✅	✅	✅
A2C	✅	✅	✅
DQN	✅	❌	❌
HER	✅	❌	❌
QR-DQN	✅	❌	❌
RecurrentPPO	✅	✅	✅
TRPO	✅	✅	✅
Maskable PPO	✅	✅	✅
ARS	✅	❌	❌

Reward function

math is probably formatted wrong but idc
speed reward:
- +/- 0.1 if mean speed increases/decreases

$$\sum_{i=1}^{n} \delta progress$$

n : Total time steps in episode
In my mind, this should encourage the bot to make forward progress and score points

Experimental reward function

train 3 laps
+10 for completing a lap
+0.1 or +0.01 for increasing speed
bigger score reward

Hyperparameters

Using PPO hyperparameters from Proximal Policy Optimization Algorithms paper

learning_rate=2.5e-4,
n_steps=128,
n_epochs=3,
batch_size=32,
ent_coef=0.01,
vf_coef=1.0,
num_envs=8