Install with Poetry fails

[volkerjaenisch]

I like this project!
So please do not take this as a rant.

I am new to hbmqtt and like to contribute.

volker@runner:/tmp$ poetry new mqtt_plugin
Created package mqtt_plugin in mqtt_plugin
volker@runner:/tmp$ cd mqtt_plugin
volker@runner:/tmp/mqtt_plugin$ poetry add hbmqtt
Creating virtualenv mqtt-plugin-236rgbPe-py3.7 in /home/volker/workspace/venvs
Using version ^0.9.6 for hbmqtt

Updating dependencies
Resolving dependencies... (0.2s)

Writing lock file

Package operations: 18 installs, 0 updates, 0 removals

• Installing typing-extensions (3.7.4.3)
• Installing zipp (3.4.1)
• Installing importlib-metadata (3.7.2)
• Installing pyparsing (2.4.7)
• Installing six (1.15.0)
• Installing attrs (20.3.0)
• Installing docopt (0.6.2)
• Installing more-itertools (8.7.0)
• Installing packaging (20.9)
• Installing passlib (1.7.4)
• Installing pluggy (0.13.1)
• Installing py (1.10.0)
• Installing pyyaml (5.4.1)
• Installing transitions (0.8.7)
• Installing wcwidth (0.2.5)
• Installing websockets (8.1)
• Installing hbmqtt (0.9.6): Failed

EnvCommandError

Command ['/home/volker/workspace/venvs/mqtt-plugin-236rgbPe-py3.7/bin/pip', 'install', '--no-deps', 'file:///home/volker/.cache/pypoetry/artifacts/91/65/bb/56a3db3b1dfc4d198422421f13ab80f919e0212fd8e405267bee9b7f42/hbmqtt-0.9.6.linux-x86_64.tar.gz'] errored with the following return code 1, and output:
Processing /home/volker/.cache/pypoetry/artifacts/91/65/bb/56a3db3b1dfc4d198422421f13ab80f919e0212fd8e405267bee9b7f42/hbmqtt-0.9.6.linux-x86_64.tar.gz
ERROR: Command errored out with exit status 1:
command: /home/volker/workspace/venvs/mqtt-plugin-236rgbPe-py3.7/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-wudkqj08/setup.py'"'"'; file='"'"'/tmp/pip-req-build-wudkqj08/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-wc6mq8gm
cwd: /tmp/pip-req-build-wudkqj08/
Complete output (5 lines):
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python3.7/tokenize.py", line 447, in open
buffer = _builtin_open(filename, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-req-build-wudkqj08/setup.py'
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
WARNING: You are using pip version 20.3.3; however, version 21.0.1 is available.
You should consider upgrading via the '/home/volker/workspace/venvs/mqtt-plugin-236rgbPe-py3.7/bin/python -m pip install --upgrade pip' command.

at ~/.local/pipx/venvs/poetry/lib/python3.7/site-packages/poetry/utils/env.py:1074 in run
1070│ output = subprocess.check_output(
1071│ cmd, stderr=subprocess.STDOUT, **kwargs
1072│ )
1073│ except CalledProcessError as e:
→ 1074│ raise EnvCommandError(e, input=input)
1075│
1076│ return decode(output)
1077│
1078│ def execute(self, bin, *args, **kwargs):

• Installing pytest (5.4.3)

Failed to add packages, reverting the pyproject.toml file to its original content.

[volkerjaenisch]

Installing hbmqtt with pip in the poetry environment was successful.

Also adding a plugin via the poetry pyproject.toml file then was successful.

I added to the pyproject.toml file:

[tool.poetry.dependencies]
python = "^3.7"
hbmqtt = "^0.9.6"

by hand. Then I included the SQLitePlugin in the pyproject.toml file as entry point:

[tool.poetry]
name = "mqtt_plugin"
version = "0.1.0"
description = ""
authors = ["volker <[email protected]>"]

[tool.poetry.dependencies]
python = "^3.7"
hbmqtt = "^0.9.6"

[tool.poetry.dev-dependencies]
pytest = "^5.2"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.poetry.plugins."hbmqtt.broker.plugins"]
persistence = "hbmqtt.plugins.persistence:SQLitePlugin"

My hbmqtt config file "broker.yaml":

listeners:
  default:
    type: tcp
    bind: 0.0.0.0:1883
sys_interval: 20
auth:
  allow-anonymous: true
persistence:
  file : 'db.sqlite'
topic-check:
  enabled: False

hbmqtt -c broker.yaml -d
The debug log:

/home/volker/workspace/venvs/mqtt-plugin-EPlEs257-py3.7/bin/python /home/volker/workspace/PROGRAMS/pycharm-2020.3.3/plugins/python/helpers/pydev/pydevd.py --multiproc --qt-support=auto --client 127.0.0.1 --port 37219 --file /home/volker/workspace/venvs/mqtt-plugin-EPlEs257-py3.7/bin/hbmqtt -c broker.yaml -d
Connected to pydev debugger (build 203.7148.72)
[2021-03-13 00:48:53,883] :: DEBUG - Using selector: EpollSelector
[2021-03-13 00:48:53,885] :: DEBUG - Loading plugins for namespace hbmqtt.broker.plugins
[2021-03-13 00:48:53,889] :: DEBUG -  Loading plugin persistence = hbmqtt.plugins.persistence:SQLitePlugin
[2021-03-13 00:48:53,963] :: DEBUG -  Initializing plugin persistence = hbmqtt.plugins.persistence:SQLitePlugin
[2021-03-13 00:48:53,964] :: INFO - Database file 'db.sqlite' opened

What I noticed is a mismatch of namespaces. In most of the hbmqtt config.yaml files there is a section
plugins shown. This section is never used. Each plugin has to be defined at the top namespace.

[FlorianLudwig]

Hi @volkerjaenisch

Please also see:

• poetry unable to install hbmqtt python-poetry/poetry#3649 - a fix in the work from poetry's side
• https://github.com/Yakifo/amqtt - our fork of HBMQTT, aMQTT. HBMQTT is now longer actively maintained.
• I am reworking the build system of aMQTT. We will move aMQTT itself to poetry too. First pre-release is on pypi - and installable with poetry.

[volkerjaenisch]

Thank you very much for this useful information.

[FlorianLudwig]

@volkerjaenisch if you try aMQTT, let us know how it works out for you :)