[Bug]: Load Trained Policy

[zlw21gxy]

🐛 Bug

Even the example code in Doc runs into bugs, loading is the problem

To Reproduce

import gymnasium as gym

from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env

# Parallel environments
vec_env = make_vec_env("CartPole-v1", n_envs=4)

model = PPO("MlpPolicy", vec_env, verbose=1)
model.learn(total_timesteps=25000)
model.save("ppo_cartpole")

del model # remove to demonstrate saving and loading

model = PPO.load("ppo_cartpole")

obs = vec_env.reset()
while True:
    action, _states = model.predict(obs)
    obs, rewards, dones, info = vec_env.step(action)
    vec_env.render("human")

Relevant log output / Error message

Traceback (most recent call last):
  File "test_ppo.py", line 15, in <module>
    model = PPO.load("ppo_cartpole")
  File "/project/stable-baselines3/stable_baselines3/common/base_class.py", line 680, in load
    data, params, pytorch_variables = load_from_zip_file(
  File "/project/stable-baselines3/stable_baselines3/common/save_util.py", line 450, in load_from_zip_file
    th_object = th.load(file_content, map_location=device, weights_only=True)
  File "/anaconda3/envs/test/lib/python3.8/site-packages/torch/serialization.py", line 788, in load
    raise pickle.UnpicklingError(UNSAFE_MESSAGE + str(e)) from None
_pickle.UnpicklingError: Weights only load failed. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution.Do it only if you get the file from a trusted source. WeightsUnpickler error: Unsupported operand 71

System Info

• OS: Linux-5.15.0-102-generic-x86_64-with-glibc2.17 # 112~20.04.1-Ubuntu SMP Thu Mar 14 14:28:24 UTC 2024
• Python: 3.8.18
• Stable-Baselines3: 2.3.1
• PyTorch: 1.13.1
• GPU Enabled: True
• Numpy: 1.24.3
• Cloudpickle: 1.2.1
• Gymnasium: 0.29.1
• OpenAI Gym: 0.15.7

Checklist

• My issue does not relate to a custom gym environment. (Use the custom gym env template instead)
• I have checked that there is no similar issue in the repo
• I have read the documentation
• I have provided a minimal and working example to reproduce the bug
• I've used the markdown code blocks for both code and stack traces.

[araffin]

Hello,
I cannot reproduce the error (and the CI either).
You are also using a unsupported gym version (maybe try uninstall it, or even better, start with a fresh virtual env).
You could also try upgrading pytorch and cloudpickle.

Could you share a google colab where you can reproduce the error?

[CAI23sbP]

🐛 Bug

Even the example code in Doc runs into bugs, loading is the problem

To Reproduce

import gymnasium as gym

from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env

# Parallel environments
vec_env = make_vec_env("CartPole-v1", n_envs=4)

model = PPO("MlpPolicy", vec_env, verbose=1)
model.learn(total_timesteps=25000)
model.save("ppo_cartpole")

del model # remove to demonstrate saving and loading

model = PPO.load("ppo_cartpole")

obs = vec_env.reset()
while True:
    action, _states = model.predict(obs)
    obs, rewards, dones, info = vec_env.step(action)
    vec_env.render("human")

Relevant log output / Error message

Traceback (most recent call last):
  File "test_ppo.py", line 15, in <module>
    model = PPO.load("ppo_cartpole")
  File "/project/stable-baselines3/stable_baselines3/common/base_class.py", line 680, in load
    data, params, pytorch_variables = load_from_zip_file(
  File "/project/stable-baselines3/stable_baselines3/common/save_util.py", line 450, in load_from_zip_file
    th_object = th.load(file_content, map_location=device, weights_only=True)
  File "/anaconda3/envs/test/lib/python3.8/site-packages/torch/serialization.py", line 788, in load
    raise pickle.UnpicklingError(UNSAFE_MESSAGE + str(e)) from None
_pickle.UnpicklingError: Weights only load failed. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution.Do it only if you get the file from a trusted source. WeightsUnpickler error: Unsupported operand 71

System Info

* OS: Linux-5.15.0-102-generic-x86_64-with-glibc2.17 # 112~20.04.1-Ubuntu SMP Thu Mar 14 14:28:24 UTC 2024

* Python: 3.8.18

* Stable-Baselines3: 2.3.1

* PyTorch: 1.13.1

* GPU Enabled: True

* Numpy: 1.24.3

* Cloudpickle: 1.2.1

* Gymnasium: 0.29.1

* OpenAI Gym: 0.15.7

Checklist

* [x]  My issue does not relate to a custom gym environment. (Use the custom gym env template instead)

* [x]  I have checked that there is no similar [issue](https://github.com/DLR-RM/stable-baselines3/issues) in the repo

* [x]  I have read the [documentation](https://stable-baselines3.readthedocs.io/en/master/)

* [x]  I have provided a [minimal and working](https://github.com/DLR-RM/stable-baselines3/issues/982#issuecomment-1197044014) example to reproduce the bug

* [x]  I've used the [markdown code blocks](https://help.github.com/en/articles/creating-and-highlighting-code-blocks) for both code and stack traces.

Same to me.
I just installed stable-baselines3 and ran that example.

[araffin]

I just installed stable-baselines3 and ran that example.

@CAI23sbP could you share your python environment?

python -c 'import stable_baselines3 as sb3; sb3.get_system_info()'

and
pip freeze

#1852

cc @kit1980

[CAI23sbP]

Hello! @araffin.
Sorry to late reply.

sb3.get_system_info()

• OS: Linux-5.15.0-101-generic-x86_64-with-glibc2.29 # 111~20.04.1-Ubuntu SMP Mon Mar 11 15:44:43 UTC 2024
• Python: 3.8.10
• Stable-Baselines3: 2.3.0
• PyTorch: 1.13.1+cu116
• GPU Enabled: True
• Numpy: 1.22.0
• Cloudpickle: 1.2.2
• Gymnasium: 0.29.1
• OpenAI Gym: 0.15.7

Freeze

absl-py==1.4.0
actionlib==1.14.0
aiohttp==3.9.3
aiosignal==1.3.1
alabaster==0.7.8
ale-py==0.7.4
alembic==1.13.1
angles==1.9.13
annotated-types==0.6.0
anyio==4.1.0
appdirs==1.4.4
apptools==5.2.1
apturl==0.5.2
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
arrow==1.3.0
ase==3.22.1
asttokens==2.2.1
astunparse==1.6.3
async-lru==2.0.4
async-timeout==4.0.3
attrs==23.1.0
autobahn==17.10.1
Automat==0.8.0
AutoROM==0.6.1
AutoROM.accept-rom-license==0.6.1
Babel==2.13.0
backcall==0.2.0
backports.zoneinfo==0.2.1
base_local_planner==1.17.3
bcrypt==3.1.7
beautifulsoup4==4.12.2
black==23.9.1
bleach==6.1.0
blinker==1.4
blosc2==2.0.0
bondpy==1.8.6
Box2D==2.3.10
breezy==3.0.2
Brlapi==0.7.0
build==0.10.0
CacheControl==0.13.1
cachetools==5.3.1
camera-calibration==1.17.0
camera-calibration-parsers==1.12.0
catkin==0.8.10
catkin-pkg==1.0.0
catkin-pkg-modules==1.0.0
cbor==1.0.0
certifi==2023.5.7
cffi==1.16.0
chardet==3.0.4
charset-normalizer==3.2.0
cleo==2.0.1
click==8.1.7
cloudpickle==1.2.2
cmake==3.26.4
colorama==0.4.3
colorlog==6.8.2
comm==0.1.3
command-not-found==0.3
configobj==5.0.6
constantly==15.1.0
contourpy==1.0.7
controller-manager==0.19.6
controller-manager-msgs==0.19.6
coverage==7.3.2
crashtest==0.4.1
cryptography==2.8
cupshelpers==1.0
cv-bridge==1.16.2
cvxpy==1.3.1
cycler==0.10.0
Cython==3.0.0
dataclasses==0.6
DataProperty==1.0.1
datasets==2.18.0
dbus-python==1.2.16
debugpy==1.6.7
decorator==4.4.2
defer==1.0.6
defusedxml==0.6.0
Deprecated==1.2.7
dgl==0.4.3.post2
diagnostic-analysis==1.11.0
diagnostic-common-diagnostics==1.11.0
diagnostic-updater==1.11.0
dill==0.3.8
distlib==0.3.7
distro==1.9.0
distro-info==0.23+ubuntu1.1
docker-pycreds==0.4.0
docopt==0.6.2
docutils==0.18.1
dulwich==0.21.6
duplicity==0.8.12.0
dynamic-reconfigure==1.7.3
ecos==2.0.12
empy==3.3.2
entrypoints==0.3
envisage==7.0.3
et-xmlfile==1.1.0
evdev==1.6.1
exceptiongroup==1.1.1
execnet==1.9.0
executing==1.2.0
Farama-Notifications==0.0.4
fastapi==0.109.2
fasteners==0.14.1
fastimport==0.9.8
fastjsonschema==2.19.0
filelock==3.12.2
filterpy==1.4.5
flameprof==0.4
flatbuffers==23.5.26
fonttools==4.39.3
fqdn==1.5.1
frozenlist==1.4.1
fsspec==2024.2.0
future==0.18.2
gast==0.4.0
gazebo_plugins==2.9.2
gazebo_ros==2.9.2
gem==0.1.12
gems==0.4.0
gencpp==0.7.0
geneus==3.0.0
genlisp==0.4.18
genmsg==0.6.0
gennodejs==2.0.2
genpy==0.6.15
gitdb==4.0.10
GitPython==3.1.37
google-auth==2.21.0
google-auth-oauthlib==1.0.0
google-pasta==0.2.0
googledrivedownloader==0.4
gpg==1.13.1
graphviz==0.20.1
greenlet==3.0.3
grpcio==1.56.0
gym==0.15.7
gym-notices==0.0.8
gymnasium==0.29.1
h11==0.14.0
h5py==3.9.0
hdbscan==0.8.30
hiddenlayer==0.3
httpcore==1.0.2
httplib2==0.14.0
httpx==0.26.0
huggingface-hub==0.21.4
huggingface-sb3==3.0
hyperlink==19.0.0
idna==3.4
image-geometry==1.16.2
imageio==2.33.1
imageio-ffmpeg==0.4.9
imagesize==1.4.1
imitation==1.0.0
importlab==0.8
importlib-metadata==4.13.0
importlib-resources==5.12.0
incremental==16.10.1
iniconfig==2.0.0
install==1.3.5
installer==0.7.0
interactive-markers==1.12.0
ipykernel==6.22.0
ipython==8.12.1
ipywidgets==8.1.1
isodate==0.6.1
isoduration==20.11.0
isort==5.12.0
jaraco.classes==3.3.0
jax-jumpy==1.0.0
jedi==0.18.2
jeepney==0.8.0
Jinja2==3.1.2
joblib==1.2.0
joint-state-publisher==1.15.1
joint-state-publisher-gui==1.15.1
json5==0.9.14
jsonpickle==3.0.3
jsonpointer==2.4
jsonschema==4.20.0
jsonschema-specifications==2023.11.2
jupyter-events==0.9.0
jupyter-lsp==2.2.1
jupyter_client==8.2.0
jupyter_core==5.3.0
jupyter_server==2.12.0
jupyter_server_terminals==0.4.4
jupyterlab==4.0.9
jupyterlab-widgets==3.0.9
jupyterlab_pygments==0.3.0
jupyterlab_server==2.25.2
keras==2.13.1
Keras-Preprocessing==1.1.2
keyboard==0.13.5
keyring==24.2.0
kitchen==1.2.6
kiwisolver==1.3.1
language-selector==0.1
laser_geometry==1.6.7
launchpadlib==1.10.13
lazr.restfulclient==0.14.2
lazr.uri==1.0.3
lazy_loader==0.3
libclang==16.0.0
libcst==1.0.1
lightgbm==3.3.5
lit==16.0.5.post0
littleutils==0.2.2
livereload==2.6.3
llvmlite==0.41.1
lockfile==0.12.2
louis==3.12.0
lxml==4.9.3
lz4==3.0.2+dfsg
lzf==0.1
macaroonbakery==1.3.1
Mako==1.1.0
Markdown==3.4.3
markdown-it-py==3.0.0
MarkupSafe==2.1.3
matplotlib==3.7.4
matplotlib-inline==0.1.6
mayavi==4.8.1
mbstrdecoder==1.1.3
mdurl==0.1.2
meld==3.20.2
message-filters==1.16.0
mistune==3.0.2
monotonic==1.5
more-itertools==10.1.0
moviepy==1.0.3
mpi4py==3.0.3
mpmath==1.3.0
msgpack==1.0.5
multidict==6.0.5
multiprocess==0.70.16
munch==4.0.0
mypy==1.3.0
mypy-extensions==1.0.0
natsort==8.4.0
nbclient==0.9.0
nbconvert==7.12.0
nbformat==5.9.2
nest-asyncio==1.5.6
netifaces==0.10.4
netron==7.1.5
networkx==3.1
ninja==1.11.1
nose==1.3.7
notebook==7.0.6
notebook_shim==0.2.3
numba==0.58.1
numexpr==2.8.5
numpy==1.22.0
nvidia-cublas-cu11==11.10.3.66
nvidia-cublas-cu12==12.1.3.1
nvidia-cuda-cupti-cu11==11.7.101
nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu11==11.7.99
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu11==11.7.99
nvidia-cuda-runtime-cu12==12.1.105
nvidia-cudnn-cu11==8.5.0.96
nvidia-cudnn-cu12==8.9.2.26
nvidia-cufft-cu11==10.9.0.58
nvidia-cufft-cu12==11.0.2.54
nvidia-curand-cu11==10.2.10.91
nvidia-curand-cu12==10.3.2.106
nvidia-cusolver-cu11==11.4.0.1
nvidia-cusolver-cu12==11.4.5.107
nvidia-cusparse-cu11==11.7.4.91
nvidia-cusparse-cu12==12.1.0.106
nvidia-nccl-cu11==2.14.3
nvidia-nccl-cu12==2.19.3
nvidia-nvjitlink-cu12==12.4.99
nvidia-nvtx-cu11==11.7.91
nvidia-nvtx-cu12==12.1.105
oauthlib==3.2.2
ogb==1.3.6
olefile==0.46
openai==1.13.3
opencv-python==4.7.0.72
openpyxl==3.1.2
opt-einsum==3.3.0
optuna==3.5.0
osqp==0.6.2.post9
outdated==0.2.2
overrides==7.4.0
packaging==23.1
pandas==1.5.2
pandocfilters==1.5.0
paramiko==2.6.0
parso==0.8.3
pathspec==0.11.2
pathtools==0.1.2
pathvalidate==3.2.0
pbr==5.11.1
pexpect==4.8.0
pgm-reader==0.0.1
pickleshare==0.7.5
Pillow==10.0.1
pkginfo==1.9.6
pkgutil_resolve_name==1.3.10
platformdirs==3.10.0
pluggy==1.0.0
poetry==1.6.1
poetry-core==1.7.0
poetry-plugin-export==1.5.0
proglog==0.1.10
prometheus-client==0.19.0
prompt-toolkit==3.0.38
protobuf==4.24.4
psutil==5.9.5
ptyprocess==0.7.0
pure-eval==0.2.2
py-cpuinfo==9.0.0
py-ubjson==0.14.0
pyarrow==15.0.1
pyarrow-hotfix==0.6
pyasn1==0.5.0
pyasn1-modules==0.3.0
pycairo==1.16.2
pycnite==2023.9.18
pycparser==2.21
pycryptodomex==3.6.1
pycups==1.9.73
pydantic==2.5.3
pydantic_core==2.14.6
pydot==1.4.2
pyenchant==3.2.2
pyface==8.0.0
pygame==2.5.2
PyGithub==1.43.7
pyglet==1.5.0
Pygments==2.15.1
PyGObject==3.36.0
PyHamcrest==1.9.0
pyhocon==0.3.60
PyJWT==1.7.1
pykalman==0.9.5
pymacaroons==0.13.0
pymunk==6.6.0
PyNaCl==1.3.0
pynput==1.7.6
PyOpenGL==3.1.0
pyOpenSSL==19.0.0
pyparsing==2.4.7
pypng==0.0.20
pyproject_hooks==1.0.0
PyQRCode==1.2.1
PyQt5==5.10.1
PyQt5-Qt5==5.15.2
PyQt5-sip==12.12.2
pyRFC3339==1.1
pyrsistent==0.19.3
pyrvo2==0.0.0
pyserial==3.4
pysqlite3==0.5.0
pysqlite3-binary==0.5.0
pytablewriter==1.2.0
pytest==7.3.1
pytest-cov==4.1.0
pytest-env==1.0.1
pytest-xdist==3.3.0
python-apt==2.0.1+ubuntu0.20.4.1
python-dateutil==2.8.2
python-debian==0.1.36+ubuntu1.1
python-dotenv==1.0.1
python-gitlab==2.0.1
python-gnupg==0.4.5
python-json-logger==2.0.7
python-louvain==0.16
python-pcl==0.3
python-qt-binding==0.4.4
python-snappy==0.5.3
python-xlib==0.33
PyTrie==0.2
pytype==2023.9.27
pytz==2023.3
PyWavelets==1.4.1
pyxdg==0.26
PyYAML==6.0.1
pyzmq==25.0.2
qdldl==0.1.7
qt-dotgraph==0.4.2
qt-gui==0.4.2
qt-gui-cpp==0.4.2
qt-gui-py-common==0.4.2
rapidfuzz==2.15.2
rdflib==7.0.0
referencing==0.31.1
regex==2023.12.25
reportlab==3.5.34
requests==2.31.0
requests-oauthlib==1.3.1
requests-toolbelt==1.0.0
requests-unixsocket==0.2.0
resource_retriever==1.12.7
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rich==13.6.0
rl_zoo3==2.3.0
roboschool==1.0.34
roman==2.0.0
rosapi==0.11.17
rosbag==1.16.0
rosboost-cfg==1.15.8
rosbridge-library==0.11.17
rosbridge-server==0.11.17
rosclean==1.15.8
roscreate==1.15.8
rosdep==0.22.2
rosdep-modules==0.22.2
rosdistro==0.9.0
rosdistro-modules==0.9.0
rosdoc-lite==0.2.11
rosgraph==1.16.0
rosinstall==0.7.8
rosinstall-generator==0.1.23
roslaunch==1.16.0
roslib==1.15.8
roslint==0.12.0
roslz4==1.16.0
rosmake==1.15.8
rosmaster==1.16.0
rosmsg==1.16.0
rosnode==1.16.0
rosparam==1.16.0
rospkg==1.5.0
rospkg-modules==1.5.0
rospy==1.16.0
rospy_message_converter==0.5.9
rosserial_arduino==0.9.2
rosserial_client==0.9.2
rosserial_python==0.9.2
rosservice==1.16.0
rostest==1.16.0
rostopic==1.16.0
rosunit==1.15.8
roswtf==1.16.0
rpds-py==0.13.2
rqt-image-view==0.4.17
rqt-moveit==0.5.10
rqt-reconfigure==0.5.5
rqt-robot-dashboard==0.5.8
rqt-robot-monitor==0.5.14
rqt-rviz==0.7.0
rqt_action==0.4.9
rqt_bag==0.5.1
rqt_bag_plugins==0.5.1
rqt_console==0.4.11
rqt_dep==0.4.12
rqt_graph==0.4.14
rqt_gui==0.5.3
rqt_gui_py==0.5.3
rqt_launch==0.4.9
rqt_logger_level==0.4.11
rqt_msg==0.4.10
rqt_nav_view==0.5.7
rqt_plot==0.4.13
rqt_pose_view==0.5.11
rqt_publisher==0.4.10
rqt_py_common==0.5.3
rqt_py_console==0.4.10
rqt_robot_steering==0.5.12
rqt_runtime_monitor==0.5.9
rqt_service_caller==0.4.10
rqt_shell==0.4.11
rqt_srv==0.4.9
rqt_tf_tree==0.6.3
rqt_top==0.4.10
rqt_topic==0.4.13
rqt_web==0.4.10
rsa==4.9
ruff==0.0.267
rviz==1.14.20
sacred==0.8.5
sb3_contrib==2.3.0
scikit-image==0.21.0
scikit-learn==1.2.2
scipy==1.8.0
screen-resolution-extra==0.0.0
scs==3.2.3
seaborn==0.8.1
seals==0.2.1
SecretStorage==3.3.3
Send2Trash==1.8.2
sensor-msgs==1.13.1
sentry-sdk==1.31.0
service-identity==18.1.0
setproctitle==1.3.2
shapely==2.0.1
shellingham==1.5.3
Shimmy==1.3.0
simplejson==3.16.0
sip==4.19.21
six==1.16.0
smach==2.5.2
smach-ros==2.5.2
smclib==1.8.6
smmap==5.0.1
sniffio==1.3.0
snowballstemmer==2.2.0
socialforce==0.2.3
soupsieve==2.5
Sphinx==7.1.2
sphinx-autobuild==2021.3.14
sphinx-autodoc-typehints==1.24.0
sphinx-copybutton==0.5.2
sphinx-rtd-theme==1.3.0
sphinxcontrib-applehelp==1.0.4
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==2.0.1
sphinxcontrib-jquery==4.1
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
sphinxcontrib-spelling==8.0.0
SQLAlchemy==2.0.28
squaternion==0.3.4
ssh-import-id==5.10
stable_baselines3==2.3.0
stack-data==0.6.2
starlette==0.36.3
sympy==1.12
systemd-python==234
tabledata==1.3.3
tables==3.8.0
tabulate==0.9.0
tcolorpy==0.1.4
tenacity==8.2.3
tensorboard==2.13.0
tensorboard-data-server==0.7.1
tensorboard-plugin-wit==1.8.1
tensorboardX==2.6.2.2
tensorflow-estimator==2.13.0
tensorflow-io-gcs-filesystem==0.32.0
termcolor==2.3.0
terminado==0.18.0
testresources==2.0.1
tf==1.13.2
tf-conversions==1.13.2
tf2-geometry-msgs==0.7.6
tf2-kdl==0.7.6
tf2-py==0.7.6
tf2-ros==0.7.6
tf2-sensor-msgs==0.7.6
thop==0.1.1.post2209072238
threadpoolctl==3.1.0
tifffile==2023.7.10
tiktoken==0.6.0
tinycss2==1.2.1
toml==0.10.2
tomli==2.0.1
tomlkit==0.12.1
topic-tools==1.16.0
torch==1.13.1+cu116
torch-cluster==1.6.3
torch-scatter==2.1.2
torch-sparse==0.6.9
torch-tb-profiler==0.4.3
torch_geometric==2.4.0
torchaudio==0.13.1+cu116
torchsummaryX==1.3.0
torchvision==0.14.1+cu116
torchviz==0.0.2
tornado==6.3.1
tqdm==4.65.0
traitlets==5.9.0
traits==6.4.3
traitsui==8.0.0
triton==2.2.0
trove-classifiers==2023.10.18
Twisted==18.9.0
txaio==2.10.0
typepy==1.3.2
types-python-dateutil==2.8.19.14
typing-inspect==0.9.0
typing_extensions==4.9.0
u-msgpack-python==2.1
ubuntu-advantage-tools==8001
ubuntu-drivers-common==0.0.0
ufw==0.36
unattended-upgrades==0.1
uri-template==1.3.0
urllib3==1.26.16
usb-creator==0.3.7
vcstools==0.1.42
virtualenv==20.24.5
vtk==9.2.6
wadllib==1.3.3
wandb==0.15.11
wasabi==1.1.2
wcwidth==0.2.6
webcolors==1.13
webencodings==0.5.1
websocket-client==1.7.0
Werkzeug==2.3.6
widgetsnbextension==4.0.9
wrapt==1.15.0
wsaccel==0.6.2
wstool==0.1.18
xacro==1.14.16
xkit==0.0.0
xxhash==3.4.1
yarl==1.9.4
zipp==3.16.0
zope.interface==4.7.1

Addition

@zlw21gxy

From #1852 , I downgraded a Stable-baselines3 version from 2.3.0 to 2.0.0, to match a torch version from 1.13.1 to 1.12.1.

Then the warning disappeared with working well

[araffin]

@CAI23sbP it seems you are also using outdated packages, could you please upgrade/remove the one i mentioned in my comment and check each time if it solves the issue? (gym should be removed, torch, cloudpickle upgraded)

if it doesn't work, please start from a fresh virtual env.

[araffin]

@kit1980 I could reproduce the issue, it seems to be linked to using an old version of pytorch (1.13.1).

If you upgrade to PyTorch >= 2.0, problem is solved.
In the meantime, I will remove weight_only=True as it might cause too many issues (until we upgrade our minimum pytorch version).

[kit1980]

Maybe Python version is the problem, not PyTorch?
I don't know really, asked here pytorch/pytorch#111806 (comment)

[kit1980]

I don't know really, asked here

OK, per reply there, 71 is BINFLOAT op, which as added to weights_only in pytorch/pytorch#94910, which was part of PyTorch 2.0 release.