Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Make check_env assertions in regard to observation_space more actionable #1399

Closed
1 task done
FieteO opened this issue Mar 22, 2023 · 4 comments
Closed
1 task done
Labels
enhancement New feature or request

Comments

@FieteO
Copy link
Contributor

FieteO commented Mar 22, 2023

🚀 Feature

When checking the implementation of a custom gym with check_env there are a number of ways to fail the validation.
For the observation_space, the dtype or the shape could be wrong (as indicated in #746, #921 and #932) that will all lead to the same AssertionError:

AssertionError: The observation returned by the `reset()` method does not match the given observation space

This is telling the user that something is wrong, but it's not clear what exactly is causing the validation failure.

Out of my head, I could think of two more actionable error messages that would help a user understand what is wrong:
For the wrong shape:

AssertionError: The observation returned by the `reset()` method does not match the given observation space. Expected shape: (x,y), actual: (x,z)

For the wrong dtype:

AssertionError: The observation returned by the `reset()` method does not match the given observation space. Array was of type: float64, expected float32

Arguably there are more failure modes associated with this error.
(Since I ran into this and have assured that the above reasons do not apply)

P.S. If you have suggestions what else can let this assertion fail, I would appreciate that to help me find my issue...

Motivation

No response

Pitch

No response

Alternatives

No response

Additional context

No response

Checklist

  • I have checked that there is no similar issue in the repo
@FieteO FieteO added the enhancement New feature or request label Mar 22, 2023
@araffin
Copy link
Member

araffin commented Mar 22, 2023

Hello,
sounds like a good improvement, feel free to submit a PR ;)

@FieteO
Copy link
Contributor Author

FieteO commented Mar 22, 2023

I justed started taking a look in the code :D
Can you tell me, what observation_space.contains(obs) is checking exactly?

    assert observation_space.contains(
        obs
    ), f"The observation returned by the `{method_name}()` method does not match the given observation space"

I suppose contains is this method, which is defined as

    def contains(self, x) -> bool:
        """Return boolean specifying if x is a valid member of this space."""
        if not isinstance(x, np.ndarray):
            logger.warn("Casting input x to numpy array.")
            try:
                x = np.asarray(x, dtype=self.dtype)
            except (ValueError, TypeError):
                return False

        return bool(
            np.can_cast(x.dtype, self.dtype)
            and x.shape == self.shape
            and np.all(x >= self.low)
            and np.all(x <= self.high)
        )

@FieteO
Copy link
Contributor Author

FieteO commented Mar 22, 2023

@araffin I have now submitted a PR. Since I have irreparably broken my env (something with setuptools or wheels I suppose), would you be so kind as to run the other required checks on your end?

@araffin
Copy link
Member

araffin commented Mar 29, 2023

Closed by #1400

@araffin araffin closed this as completed Mar 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants