Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added option for different arg/return type hints to type_caster #5358

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

timohl
Copy link

@timohl timohl commented Sep 6, 2024

Description

This PR adds the option for defining return_name in a custom type_caster, which allows to have different python type hints for arguments and return value. The check if return_name is available or name should be used as fallback is implemented using the template class as_return_type.

This is entirely optional and should not impact any existing type_caster.
One exception is the type_caster for std::filesystem::path, which was modified to support this new feature.
Here, Union[os.PathLike, str, bytes] is the argument type hint and Path is the return type hint.
This is more accurate than the previous usage of os.PathLike for both argument and return type hint.
I have updated the unit test to check for the new signature.

Suggested changelog entry:

Added option for different arg/return type hints to type_caster.
Updated stl/filesystem to use correct arg/return type hints.

Possible TODOs

  • Update Custom type casters documentation
  • Test if it works in containers like list[Path] -> does not work
  • Fix for all handle_type_name (maybe some recursive template magic adding as_return_type to subtypes)
  • Update type hints for Eigen (currently uses np.ndarray as arg type hint but also takes lists and tuples (should probably be numpy.typing.ArrayLike + maybe typing.Annotated for shape/dtype annotation

I would love to get feedback on this!
Especially on the "Possible TODO" regarding Eigen. Should I implement that here in this PR or should I open a separate PR later?

Currently I am working on adding typing stubs to Open3D (isl-org/Open3D#6917) using pybind11-stubgen.
Applying mypy/pyright on existing example code showed that a lot of type checks failed, which could be fixed by this PR.

This allows to define different python type hints for arguments and
return value. This is implemented using template class `as_return_type`
in order to check if `return_name` is available in the `type_caster` at
compile time. If `return_name` is not available, it falls back to the
previous usage of `name` for both args and return value.
The `type_caster` previously named `os.PathLike` as both argument and
return type. This is inaccurate since it also takes `str` and `byte` as
argument and actually returns `Path`. This commit uses the new
`return_name` feature to define argument type as
`Union[os.PathLike, str, bytes]` and return type as `Path`.
An assert was added to the unit test for this.
@timohl timohl marked this pull request as draft September 6, 2024 16:46
@timohl
Copy link
Author

timohl commented Sep 6, 2024

The new unit test in 7d16bad shows that it does not work for nested types, e.g., list[Path].
This requires to somehow inject as_return_type into all handle_type_name subtypes.
Maybe some recursive template magic could be possible.

Going to pull back as a draft and try to work on that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant