Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Nullable Data Types in Pandas and Snowpark DataFrames During Input Validation #122

Open
kenkoooo opened this issue Oct 27, 2024 · 0 comments

Comments

@kenkoooo
Copy link

It seems that nullable strings, integers, and booleans are not supported as input types. When passing a pandas DataFrame, each column is converted to numpy arrays and validated with np.dtype, which doesn't support certain nullable types. Similarly, when passing a Snowpark DataFrame, it is converted using the signature type, which also relies on np.dtype.

Can we improve validation to use pandas dtypes instead of numpy dtypes, given that some ML models like LightGBM can work with nullable columns?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant