module errors: ModuleNotFoundError: No module named 'lantz.errors'

[Bruyant]

Just installed lantz with pyvisa-py,
lantz sims fungen tcp is working but then lantz qtdemo testpanel fail

i can find error.py in lantz/core/ but dunno how to got further.

Thanks

(lantz) bruyant@tartampion:~$ lantz qtdemo testpanel
Please make sure that the simulator is running.
You can start it by running the following command in another terminal:

lantz sims fungen tcp
Traceback (most recent call last):
  File "/home/bruyant/anaconda3/envs/lantz/bin/lantz", line 11, in <module>
    sys.exit(main())
  File "/home/bruyant/anaconda3/envs/lantz/lib/python3.6/site-packages/lantz/__main__.py", line 33, in main
    parser.dispatch(args)
  File "/home/bruyant/anaconda3/envs/lantz/lib/python3.6/site-packages/lantz/__init__.py", line 40, in dispatch
    self.__choices[required_value](pending)
  File "/home/bruyant/anaconda3/envs/lantz/lib/python3.6/site-packages/lantz/qt/__main__.py", line 10, in main
    parser.dispatch(args)
  File "/home/bruyant/anaconda3/envs/lantz/lib/python3.6/site-packages/lantz/__init__.py", line 40, in dispatch
    self.__choices[required_value](pending)
  File "/home/bruyant/anaconda3/envs/lantz/lib/python3.6/site-packages/lantz/qt/__main__.py", line 29, in testpanel_demo
    from lantz.drivers.examples import LantzSignalGenerator
  File "/home/bruyant/anaconda3/envs/lantz/lib/python3.6/site-packages/lantz/drivers/examples/__init__.py", line 16, in <module>
    from .fungen import LantzSignalGenerator
  File "/home/bruyant/anaconda3/envs/lantz/lib/python3.6/site-packages/lantz/drivers/examples/fungen.py", line 13, in <module>
    from lantz.errors import InstrumentError
ModuleNotFoundError: No module named 'lantz.errors'

[hgrecco]

This is an (unintended) consequence of the namespace packages reorganization. lantz.errors is no longer accesible from the outside directly. You can change it to

from lantz import errors
InstrumentError = errors.InstrumentError

(we should do it for all the lantz codebase)

[hgrecco]

I wonder if there is any way to expose it while keeping namespace packages.

[bilderbuchi]

maybe you can tweak the __all__ object in lantz's __init__.py? I don't know for sure, tbh.

[hgrecco]

Namespace packages are different beast, I think it will not work. But I will try.

[bilderbuchi]

Ok. I am not very experienced with them.

[hgrecco]

Fixed!

[hgrecco]

And thanks for the feedback. Changing the structure of a project often break some internal imports specially in decoupled projects.

[bilderbuchi]

Would be interesting to know if there is a better solution compared to this workaround of changing all the imports, which kinda makes the code clunkier.

[hgrecco]

I think a good way would be to promote InstrumentError to the root of lantz. In that way, you would just do:

from lantz import InstrumentError

[bilderbuchi]

Hm, could it be that you are not allowed to have an __init__.py in the lantz "root package", either? The file structure in this packaging guide for namespace packages seems to tell me that. Later on it says

It is extremely important that every distribution [i.e. namespace package/subpackage] that uses the namespace package omits the init.py or uses a pkgutil-style init.py. If any distribution does not, it will cause the namespace logic to fail and the other sub-packages will not be importable.

which could be what is observed here. It also says

Because mynamespace doesn’t contain an init.py, setuptools.find_packages() won’t find the sub-package. You must explicitly list all packages in your setup.py.

which is not present in the lantz setup.py (only lantz itself is mentioned)- maybe adding all the subpackages would also fix this issue.

[hgrecco]

I do not think this is the problem. The lantz root package is properly integrated into the namespace thanks to:

__path__ = __import__('pkgutil').extend_path(__path__, __name__)

The problem is that errors is not re-exported. You se, errors is acutally a subpackage of lantz core and even though I import it in lantz/init.py it is not re-exported.

[bilderbuchi]

I tried to troubleshoot this strange import behaviour, but was not too successful, sorry. I did not find out why the errors module is not imported into the lantz namespace correctly. To be honest, I don't get why importing objects (like Action) from a sub-namespace-package works as expected, but importing modules does not. I suspect

What is confusing is that I can do this:

In [3]: lantz.errors                                                                                     
Out[3]: <module 'lantz.core.errors' from '/home/bilderbuchi/lantz-temp/lantz-core/lantz/core/errors.py'>

so this looks ~ as expected, in that a module lantz.errors is identified, but then

In [4]: import lantz.errors                                                                              
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-4-2ab2fb1e8a2b> in <module>
----> 1 import lantz.errors

ModuleNotFoundError: No module named 'lantz.errors'

fails as before.

However, a potentially better workaround than lantzproject/lantz-drivers@52edaaf could maybe be, as you suggested, to promote InstrumentError to the lantz root, this would make usage less verbose/more straightforward, no?

[bilderbuchi]

My gut feeling says that (import) usage should be fully exchangeable between namespace and non-namespace packages, so I feel like what we see here indicates some problem with the namespace setup, but I can't find out what's wrong. Seeing as you only support high Python 3 versions, have you ever considered going with native namespace packages instead of the pkgutil option?

[hgrecco]

Happy new year! I actually use a combination fo both native namepsace packages and pkgutil. Because, AFAIK, it is not possible to have root package and subpackages. In other words, it is not possible to have lantz AND lantz.core, lantz.sims. I needed to add the pkgutil option for this reason.

The following code in lantz/__init__.py and does not seem to have any secondary effect

import sys
from lantz.core import errors, log
sys.modules['lantz.log'] = log
sys.modules['lantz.errors'] = errors

In any case, I do think that InstrumentError is important enough to promote it.

By the way, I have published the package on pypi as lantzdev. See https://github.com/lantzproject/lantz for more info. Also I have added a examples repo: https://github.com/lantzproject/examples

[bilderbuchi]

To you, too!
Ha, interesting approach! Although (again) feels like we are workingh/acking around a problem somehow. Would be interesting to post this problem to Stackoverflow or similar to get input from the wider Python community, if there is something we are missing. I won't find the time to formulate a minimal reproduction example, though. :-(

Yeah, I've seen that. I hope that this will be re-integrated into vanilla lantz (on pypi too) as soon as you have consolidated the way to go forward with lantz. I would not worry too much about getting consensus out of the very scattered community, seeing as vanilla lantz is basically dead, I think continuing/restarting development there would be the most logical approach.
Somewhat related, will it be possible to integrate the work by others in the meantime into your current package structure (as mentioned in LabPy/labpy-discussion#21 (comment))?

[hgrecco]

I will try to assemble a minimal example and post it in stackoverflow.

Regarding moving lantz forward. I think that having all individual packages (lantz-X) in PyPI and also the umbrella package (lantzdev) published in pypi allow us to experiment with the concept that I have presented regarding installing, upgrading and distributing.

Until now, the installation is really simple and straight forward. I have no seen any drawbacks (apart from these import glitches). In any case, these glitches are allowing me to think that needs to be exposed and what does not. Compartamentalization is a good thing and many things that I have recently added (logging capabilities backend/frontend, connect_feat in frontend, etc) arise from this. Also development in subpackages is working, at least for me.

My proposal is: let's move this repo forward. I would be happy to incorporate the new drivers that have been developed and receive feedback and PR here.