Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a "nearest non-null" interpolation method for polars.Expr.interpolate #20847

Open
MarcBresson opened this issue Jan 22, 2025 · 1 comment
Labels
enhancement New feature or an improvement of an existing feature

Comments

@MarcBresson
Copy link

Description

In case we have a series that ends (or starts) with null values, both linear and nearest interpolation will not be able to fill these values.

There could be a new interpolation named "nearest non-null" that would use the nearest non null value. It would give the following examples:

df = pl.DataFrame(
    {
        "a": [4, None, None, None],
    }
)
df = df.with_columns(pl.col("a").interpolate(method="linear"))
df

# shape: (4, 1)
# a
# --
# f64
# 4.0
# null
# null
# null

but

df = pl.DataFrame(
    {
        "a": [4, None, None, None],
    }
)
df = df.with_columns(pl.col("a").interpolate(method="nearest non-null"))
df

# shape: (4, 1)
# a
# --
# i64
# 4
# 4
# 4
# 4
@MarcBresson MarcBresson added the enhancement New feature or an improvement of an existing feature label Jan 22, 2025
@MarcBresson
Copy link
Author

For now, the workaround is to fill null values both backward and forward i.e.

df = pl.DataFrame(
    {
        "a": [None, 4, None, None, None],
    }
)
df = df.with_columns(pl.col("a").interpolate(method="nearest"))
df = df.with_columns(pl.col("a").fill_null(strategy="forward"))
df
# shape: (4, 1)
# a
# --
# i64
# null
# 4
# 4
# 4
# 4

df = df.with_columns(pl.col("a").fill_null(strategy="backward"))
df
# shape: (4, 1)
# a
# --
# i64
# 4
# 4
# 4
# 4
# 4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or an improvement of an existing feature
Projects
None yet
Development

No branches or pull requests

1 participant