Float64 faster than Float32

[franz101]

First of all very cool library, I was looking for something like this for a while. Curious why Float64 is faster. I assume under the hood type conversion?

import polars as pl
import numpy as np

df = pl.DataFrame(
    {
        "arr": np.random.rand(100_000, 512),
        "arr2": np.random.rand(100_000, 512),
    },
    schema={
        "arr": pl.Array(inner=pl.Float64, width=512),
        "arr2": pl.Array(inner=pl.Float64, width=512),
    },
)

%%timeit
df.select(pld.col('arr').dist_arr.euclidean(pl.col("arr2")).alias('dist'))

109 ms ± 5.68 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

df = pl.DataFrame(
    {
        "arr": np.random.rand(100_000, 512),
        "arr2": np.random.rand(100_000, 512),
    },
    schema={
        "arr": pl.Array(inner=pl.Float32, width=512),
        "arr2": pl.Array(inner=pl.Float32, width=512),
    },
)

631 ms ± 11.9 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

[ion-elgreco]

Its always converting to f64 first, this is obviously not optimal :)

You could add a branch that works on f32

[krds00]

@ion-elgreco
does it make sense to cast the type from float64 to float32 before processing the data? For example, the calculation of moving averages and shifts. Or should it be done only before saving to disk to save space?