Make check_env assertions in regards to observation_space more actionable

[FieteO]

Description

Provide more actionable assertions when checking the observation_space, as requested in #1399

In particular, add dedicated AssertionError for:

• a different shape of obs and observation_space
• a different dtype of obs and observation_space
• different lower bounds of obs and observation_space
• different upper bounds of obs and observation_space

Misc:

• address make lint warning about checking availability of module
• enhance README section about running tests

Motivation and Context

• I have raised an issue to propose this change (required for new features and bug fixes)

The current mode of failure is not telling much about why it fails.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)

Checklist

• I've read the CONTRIBUTION guide (required)
• I have updated the changelog accordingly (required).
• My change requires a change to the documentation.
• I have updated the tests accordingly (required for a bug fix or a new feature).
• I have updated the documentation accordingly.
• I have opened an associated PR on the SB3-Contrib repository (if necessary)
• I have opened an associated PR on the RL-Zoo3 repository (if necessary)
• I have reformatted the code using make format (required)
• I have checked the codestyle using make check-codestyle and make lint (required)
• I have ensured make pytest and make type both pass. (required)
• I have checked that the documentation builds using make doc (required)

Note: You can run most of the checks using make commit-checks.

Note: we are using a maximum length of 127 characters per line

[FieteO]

All working now, except for the following pytype exception when running make type:

File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 175, in _check_obs: No attribute 'shape' on tuple [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 175, in _check_obs: No attribute 'shape' on dict [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 175, in _check_obs: No attribute 'shape' on int [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 180, in _check_obs: No attribute 'shape' on tuple [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 180, in _check_obs: No attribute 'shape' on dict [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 180, in _check_obs: No attribute 'shape' on int [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 183, in _check_obs: No attribute 'dtype' on tuple [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 183, in _check_obs: No attribute 'dtype' on dict [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 183, in _check_obs: No attribute 'dtype' on int [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 188, in _check_obs: No attribute 'dtype' on tuple [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 188, in _check_obs: No attribute 'dtype' on dict [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 188, in _check_obs: No attribute 'dtype' on int [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]

For more details, see https://google.github.io/pytype/errors.html#attribute-error
ninja: build stopped: subcommand failed.
Leaving directory '.pytype'
make: *** [Makefile:8: pytype] Error 1

Any suggestions for fixing that? I don't have much experience with type checking in python...

[FieteO]

All working now, except for the following pytype exception when running make type:

File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 175, in _check_obs: No attribute 'shape' on tuple [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 175, in _check_obs: No attribute 'shape' on dict [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 175, in _check_obs: No attribute 'shape' on int [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 180, in _check_obs: No attribute 'shape' on tuple [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 180, in _check_obs: No attribute 'shape' on dict [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 180, in _check_obs: No attribute 'shape' on int [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 183, in _check_obs: No attribute 'dtype' on tuple [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 183, in _check_obs: No attribute 'dtype' on dict [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 183, in _check_obs: No attribute 'dtype' on int [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 188, in _check_obs: No attribute 'dtype' on tuple [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 188, in _check_obs: No attribute 'dtype' on dict [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]
File "/home/fiete/Downloads/stable-baselines3/stable_baselines3/common/env_checker.py", line 188, in _check_obs: No attribute 'dtype' on int [attribute-error]
  In Union[dict, int, numpy.ndarray[Any, numpy.dtype], tuple]

For more details, see https://google.github.io/pytype/errors.html#attribute-error
ninja: build stopped: subcommand failed.
Leaving directory '.pytype'
make: *** [Makefile:8: pytype] Error 1

Any suggestions for fixing that? I don't have much experience with type checking in python...

Issue resolved

[araffin]

LGTM, thanks =)
(I refactored the tests and extended the checks to numpy arrays)