locking with python3-etcd3 0.12.0 on etcd 3.4.23 fails?

[jimrobinson]

I'm using a test cluster of three etcd nodes running etcd version 3.4.23.

When using python etcd3==0.12.0 my attempts to test etcd3 locks (see test_lock.txt) across multiple processes blows up, with different errors depending on which version of tenacity I'm using. With the following:

etcd3==0.12.0
grpcio==1.53.0
protobuf==3.20.3
six==1.16.0
tenacity==8.2.2

I get the error:

Traceback (most recent call last):
  File "/app/./test_lock.py", line 24, in <module>
    p.map(test_lock, [i+nid for i in range(nproc)])
  File "/usr/local/lib/python3.10/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/local/lib/python3.10/multiprocessing/pool.py", line 771, in get
    raise self._value
TypeError: Lock.acquire.<locals>.wait() missing 1 required positional argument: 'delay_since_first_attempt'

and with a lower version of tenacity:

etcd3==0.12.0
grpcio==1.53.0
protobuf==3.20.3
six==1.16.0
tenacity==8.1.0

I get the error

Traceback (most recent call last):
  File "/app/./test_lock.py", line 24, in <module>
    p.map(test_lock, [i+nid for i in range(nproc)])
  File "/usr/local/lib/python3.10/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/local/lib/python3.10/multiprocessing/pool.py", line 771, in get
    raise self._value
TypeError: Lock.acquire.<locals>.wait() got an unexpected keyword argument 'retry_state'

Installing python-etcd3 from the master branch (using pip install git+https://github.com/kragniz/python-etcd3) appears to resolve the issue, so:

etcd3 @ git+https://github.com/kragniz/python-etcd3@e58a899579ba416449c4e225b61f039457c8072a
grpcio==1.53.0
protobuf==3.20.3
six==1.16.0

appears to work fine:

python3 ./test_lock.py
does 1 have the lock? True
does 2 have the lock? True
does 3 have the lock? True
does 4 have the lock? True

[zignd]

I'm also getting the same error when I try to use Lock.acquire() on a lock that is already locked. So I decided to investigate and this is what I found:

• It seems that the library code published as 0.12.0 is from Mar 31, 2020 and it uses this tenacity library to handle the retries inside the acquire method.
• The code in the master branch has a change from Apr 9, 2020 (one month later) which can be seen in this pull request in which the author removes tenacity mentioning some sort of issue that he was not able to fix.

[zignd]

@kragniz would you mind publishing the library again please considering that the changes in the master branch seem to fix this issue?

[wws0222]

Can a new version of the newly added MR be released?

[jameszrx]

Is this issue resolved yet? I am still hitting the same issue when using 0.12.0, and this issue exists in 0.11.0, 0.10.0, and directly taking dependency on master branch in production use cases is something I'd only take as the last resort. @kragniz, can you or other maintainers release a new package version with the fix? Thank you!