Additional fix while retraining policies #629

Cadene · 2025-01-11T17:07:37Z

What this does

Retrain policies

How it was tested

act aloha insertion

python lerobot/scripts/train.py \
--policy.type=act \
--dataset.repo_id=lerobot/aloha_sim_insertion_human \
--env.type=aloha \
--wandb.enable=true

https://wandb.ai/rcadene/lerobot/runs/1mfzmkyg?nw=nwuserrcadene

python lerobot/scripts/eval.py \
--policy.path=outputs/train/2025-01-09/17-59-06_aloha_act/checkpoints/last/pretrained_model \
--env.type=aloha \
--env.task=AlohaTransferCube-v0 \
--eval.n_episodes=50 \
--eval.batch_size=50 \
--device=cuda \
--use_amp=false

{'avg_sum_reward': 218.12, 'avg_max_reward': 2.34, 'pc_success': 20.0, 'eval_s': 92.44307279586792, 'eval_ep_s': 1.8488614654541016}

python lerobot/scripts/eval.py \
--policy.path=outputs/train/2025-01-09/17-59-06_aloha_act/checkpoints/last/pretrained_model \
--output_dir=outputs/train/2025-01-09/17-59-06_aloha_act/full_eval/last/AlohaTransferCube-v0 \
--env.type=aloha \
--env.task=AlohaTransferCube-v0 \
--eval.n_episodes=50 \
--eval.batch_size=50 \
--device=cuda \
--use_amp=false

{'avg_sum_reward': 1.7, 'avg_max_reward': 0.1, 'pc_success': 0.0, 'eval_s': 87.0343189239502, 'eval_ep_s': 1.740686388015747}

python lerobot/scripts/eval.py \
--policy.path=outputs/train/2025-01-09/17-59-06_aloha_act/checkpoints/last/pretrained_model \
--output_dir=outputs/train/2025-01-09/17-59-06_aloha_act/full_eval/last/AlohaInsertion-v0 \
--env.type=aloha \
--env.task=AlohaInsertion-v0 \
--eval.n_episodes=50 \
--eval.batch_size=50 \
--device=cuda \
--use_amp=false

{'avg_sum_reward': 218.08, 'avg_max_reward': 2.34, 'pc_success': 20.0, 'eval_s': 89.83276915550232, 'eval_ep_s': 1.796655387878418}

act aloha transfer cube

python lerobot/scripts/train.py \
--policy.type=act \
--dataset.repo_id=lerobot/aloha_sim_transfer_cube_human \
--env.type=aloha \
--env.task=AlohaTransferCube-v0 \
--wandb.enable=true

https://wandb.ai/rcadene/lerobot/runs/neuu3olc?nw=nwuserrcadene

python lerobot/scripts/eval.py \
--policy.path=outputs/train/2025-01-10/11-41-03_aloha_act/checkpoints/last/pretrained_model \
--output_dir=outputs/train/2025-01-10/11-41-03_aloha_act/full_eval/last \
--env.type=aloha \
--env.task=AlohaTransferCube-v0 \
--eval.n_episodes=50 \
--eval.batch_size=50 \
--device=cuda \
--use_amp=false

{'avg_sum_reward': 212.4, 'avg_max_reward': 3.38, 'pc_success': 76.0, 'eval_s': 86.73920726776123, 'eval_ep_s': 1.7347841548919678}

**diffusion pusht**
```bash
python lerobot/scripts/train.py \
--policy.type=diffusion \
--dataset.repo_id=lerobot/pusht \
--seed=100000 \
--env.type=pusht \
--batch_size=64 \
--offline.steps=200000 \
--eval_freq=25000 \
--save_freq=25000 \
--wandb.enable=true

https://wandb.ai/rcadene/lerobot/runs/7yovun9s

python lerobot/scripts/eval.py \
--policy.path=outputs/train/2025-01-11/15-12-08_pusht_diffusion/checkpoints/200000/pretrained_model \
--output_dir=outputs/train/2025-01-11/15-12-08_pusht_diffusion/full_eval/200000 \
--env.type=pusht \
--eval.n_episodes=50 \
--eval.batch_size=50 \
--device=cuda \
--use_amp=false

{'avg_sum_reward': 121.85938595512995, 'avg_max_reward': 0.9644504711735705, 'pc_success': 56.00000000000001, 'eval_s': 47.386802196502686, 'eval_ep_s': 0.9477360534667969}

```bash
python lerobot/scripts/eval.py \
--policy.path=outputs/train/2025-01-11/15-12-08_pusht_diffusion/checkpoints/100000/pretrained_model \
--output_dir=outputs/train/2025-01-11/15-12-08_pusht_diffusion/full_eval/100000 \
--env.type=pusht \
--eval.n_episodes=50 \
--eval.batch_size=50 \
--device=cuda \
--use_amp=false

{'avg_sum_reward': 113.42846335694817, 'avg_max_reward': 0.9828476584918505, 'pc_success': 78.0, 'eval_s': 47.40688681602478, 'eval_ep_s': 0.9481377410888672}

python lerobot/scripts/eval.py \
--policy.path=outputs/train/2025-01-11/15-12-08_pusht_diffusion/checkpoints/050000/pretrained_model \
--output_dir=outputs/train/2025-01-11/15-12-08_pusht_diffusion/full_eval/050000 \
--env.type=pusht \
--eval.n_episodes=50 \
--eval.batch_size=50 \
--device=cuda \
--use_amp=false

tdmpc xarm

python lerobot/scripts/train.py \
--policy.type=tdmpc \
--dataset.repo_id=lerobot/xarm_lift_medium \
--seed=1 \
--env.type=xarm \
--batch_size=256 \
--offline.steps=200000 \
--online.steps=50000 \
--online.env_seed=10000 \
--online.buffer_capacity=80000 \
--online.steps_between_rollouts=50 \
--eval_freq=5000 \
--save_freq=10000 \
--log_freq=100 \
--wandb.enable=true

https://wandb.ai/rcadene/lerobot/runs/65b0rxz7

python lerobot/scripts/eval.py \
--policy.path=outputs/train/2025-01-11/15-12-08_pusht_diffusion/checkpoints/last/pretrained_model \
--output_dir=outputs/train/2025-01-11/15-12-08_pusht_diffusion/full_eval/last \
--env.type=pusht \
--eval.n_episodes=50 \
--eval.batch_size=50 \
--device=cuda \
--use_amp=false

17-38-52_xarm_tdmpc

vqbet pusht

python lerobot/scripts/train.py \
--policy.type=vqbet \
--dataset.repo_id=lerobot/pusht \
--seed=100000 \
--env.type=pusht \
--batch_size=64 \
--offline.steps=250000 \
--eval_freq=25000 \
--save_freq=25000 \
--wandb.enable=true

https://wandb.ai/rcadene/lerobot/runs/sgkstbls

python lerobot/scripts/eval.py \
--policy.path=outputs/train/2025-01-11/18-03-47_pusht_vqbet/checkpoints/250000/pretrained_model \
--output_dir=outputs/train/2025-01-11/18-03-47_pusht_vqbet/full_eval/250000 \
--env.type=pusht \
--eval.n_episodes=50 \
--eval.batch_size=50 \
--device=cuda \
--use_amp=false

{'avg_sum_reward': 96.32497890276665, 'avg_max_reward': 0.7956230464645369, 'pc_success': 46.0, 'eval_s': 27.269179582595825, 'eval_ep_s': 0.5453836011886597}

python lerobot/scripts/eval.py \
--policy.path=outputs/train/2025-01-11/18-03-47_pusht_vqbet/checkpoints/100000/pretrained_model \
--output_dir=outputs/train/2025-01-11/18-03-47_pusht_vqbet/full_eval/100000 \
--env.type=pusht \
--eval.n_episodes=50 \
--eval.batch_size=50 \
--device=cuda \
--use_amp=false

{'avg_sum_reward': 97.06195423096551, 'avg_max_reward': 0.8539270621245656, 'pc_success': 52.0, 'eval_s': 27.543201208114624, 'eval_ep_s': 0.5508640289306641}

python lerobot/scripts/eval.py \
--policy.path=outputs/train/2025-01-11/18-03-47_pusht_vqbet/checkpoints/150000/pretrained_model \
--output_dir=outputs/train/2025-01-11/18-03-47_pusht_vqbet/full_eval/150000 \
--env.type=pusht \
--eval.n_episodes=50 \
--eval.batch_size=50 \
--device=cuda \
--use_amp=false

{'avg_sum_reward': 113.66729212298688, 'avg_max_reward': 0.844645479041044, 'pc_success': 44.0, 'eval_s': 26.88631582260132, 'eval_ep_s': 0.5377263259887696}

TODO

Cadene · 2025-01-11T17:08:19Z

lerobot/common/logger.py

@@ -121,7 +121,7 @@ def __init__(self, cfg: TrainPipelineConfig):
                notes=cfg.wandb.notes,
                tags=cfg_to_group(cfg, return_list=True),
                dir=self.log_dir,
-                config=OmegaConf.to_container(cfg, resolve=True),
+                config=draccus.encode(cfg),


TODO: remove

…DENTITY

Cadene added 2 commits January 9, 2025 16:10

Fix bug when wandb.enable=True

aa93aa1

Fix wandb log + RL

0d0f290

Cadene changed the title ~~Additional fix~~ Additional fix while retraining policies Jan 11, 2025

Cadene commented Jan 11, 2025

View reviewed changes

Cadene requested a review from aliberts January 11, 2025 17:11

Fix decoding with None not found for NormalizationMode. Replaced by I…

fced457

…DENTITY

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Additional fix while retraining policies #629

Additional fix while retraining policies #629

Cadene commented Jan 11, 2025 •

edited

Loading

Cadene Jan 11, 2025

Additional fix while retraining policies #629

Are you sure you want to change the base?

Additional fix while retraining policies #629

Conversation

Cadene commented Jan 11, 2025 • edited Loading

What this does

How it was tested

Cadene Jan 11, 2025

Choose a reason for hiding this comment

Cadene commented Jan 11, 2025 •

edited

Loading