-
Notifications
You must be signed in to change notification settings - Fork 723
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[PPO2] problems resuming training #781
Comments
Related: #301 #692 Also, you should use a Did you try using the rl zoo? |
I looked at that issues but didn't find the solution. That's not critical anyway. |
@k0rean any solution to this problem? |
@njanirudh I do not have direct answer, but if possible, try out stable-baselines3 and see if it helps with your issues. It is more actively maintained so we can discuss and fix bugs there :) |
I'm trying to resume the model training and I'm getting some strange results. Using SubProcVecEnv and VecNormalize on a custom environment:
Firstly, don't know why it doesn't continue the current tensorboard training curve if I passed reset_num_timesteps=False. Already updated tensorboard to the last version and verified the same behaviour.
But the bigger problem is the discontinuity verified between the two runs. Already tried a single run with more timesteps (10e6) and got a continuous improving curve but without reaching a reward of 2.5 as the 2nd run got in this case. The 2nd run reached a bigger reward almost in the beginning but didn't improve anymore.
Am I doing some mistake loading the previous model?
System Info
The text was updated successfully, but these errors were encountered: