Optimised Libraries ⏩ 〰️ 💨

[ns-rse]

There are a number of optimised libraries for many packages, with optimsation at different levels...

Intel Optimisations

• Intel Extensions for Scikit-learn
• Numpy/Scipy with Intel® MKL and Intel® Compilers

Alternative Implementations

• ruff a Python linter written in Rust which is considerably faster than native linters.
• polars dataframes in Rust.

Parallelisation

• Dask scale and parallelise Python data libraries (Numpy, Pandas, Scikit-learn).

[bobturneruk]

Keen for this.

[willfurnass]

List of packages provided by Intel conda channel: https://anaconda.org/intel/repo

[EdwinB12]

https://jax.readthedocs.io/en/latest/jax-101/index.html

[ns-rse]

#12 is kind of related to this topic.

Does anyone have experience of using any of these optimised packages?

• Pros/Cons.
• Gotchas/differences.
• Limitations.

[willfurnass]

Yes, but probably won't be able to join this week.

[willfurnass]

Wrote a bit on numpy accelerated using the MKL some time ago: https://github.com/RSE-Sheffield/hi-perf-ipynb/blob/master/tutorials/01-multithreading.ipynb

[ns-rse]

Thanks for the notebook @willfurnass 👍

Yes, but probably won't be able to join this week.

Good reason to defer till next meeting?

[f-allian]

• rustworkx Optimised Python library alternative to NetworkX
• see also benchmarks