Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why does WrightTools take so long to import? #1083

Open
darienmorrow opened this issue Jul 7, 2022 · 1 comment
Open

Why does WrightTools take so long to import? #1083

darienmorrow opened this issue Jul 7, 2022 · 1 comment

Comments

@darienmorrow
Copy link
Member

I have noticed whenever I am working, when I add a WrightTools import to my script, it takes a ~second longer to run. Running the below code a bunch of times, I see numpy routinely takes 0 ms to import, scipy takes 20-200 ms, and WrightTools takes 1-5 s.

  1. Has anyone else noticed this?
  2. If so, do we understand why this is?
import time
t0 = time.time()
import numpy as np
t1 = time.time()
import scipy as sp
t2 = time.time()
import WrightTools as wt
t3 = time.time()
print('numpy', round(t1-t0, 4))
print('scipy', round(t2-t1, 4))
print('wrighttools', round(t3-t2, 4))
@ksunden
Copy link
Member

ksunden commented Jul 7, 2022

The short answer is that we import a large portion of the scipy stack by default and that takes time.

The long answer is is that a lot of it it will vary based on what you have installed, for instance pandas and xarray are both not hard dependencies of ourselves or any of our dependencies, but nonetheless if they are available will be imported (by e.g. pint.compat)

The even longer answer (and solution) is PEP 690:
https://peps.python.org/pep-0690/

PS, that test is not super good because scipy imports numpy and wt imports both, so you are actually undercounting the time for WT and scipy. (since you have already imported some of what causes imports)

PPS you can do python -X importtime -c "import WrightTools" and you will get a nice trace of all the import time in microseconds

pint is the biggest offender, taking 0.8 of 1.8 seconds alone, due to importing pandas and xarray if they are available (numpy is .123 seconds, matplotlib.projections is 0.473 seconds)

Note that many packages (scipy and mpl come to mind) do not import all of their modules when you import the top level module, so it will take longer to import e.g. scipy.optimize rather than just scipy) PEP 690 aims to alleviate some of that concern so things can be "imported" without being "loaded" yet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

2 participants