puresnmp 1.10.2 depends on typing module

[davnat]

puresnmp 1.10.2 depends on typing module

[exhuma]

Indeed. Thanks for the detailed info.

This is an oversight on my part from the switch to poetry. I'll get this done as soon as I'm back on my dev-box.

[exhuma]

fwiw, I'm also actively working on a 2.0 release with a new architecture supporting SNMPv3 as well. I will try to keep it as backwards compatible as possible, but in order to keep the "external API" clean, it won't be 100% backwards compatible.

If you only recently started using the library, consider limiting it to "<2.0", just in case.

A pre-release might be coming out by the end of 2020, if not then, then early 2021. The core for SNMPv3 is already working, I'm only refactoring an cleaning up the external API so it shouldn't take too long.

[exhuma]

I just tried to repro the issue but everything worked for me.

I ran the following commands:

mkdir puresnmptest
cd puresnmptest
poetry init
[snip]
poetry add puresnmp
poetry add -D black

So I have both puresnmp and black in the same env without the error you mention. Can you let me know how to reproduce this issue?

I don't mind publishing the package based on poetry, but because the 1.x branch still supports Python 2 I'm a bit cautious about it.

The 2.x branch which should - as mentioned - be released in the coming weeks (albeit first, and for a while as a pre-release) will anyway be based on poetry.

[davnat]

I had an issue very similar to the one I linked (that I found searching for clues) when calling poetry install in a project with both puresnmp and black (and many other packages), but the error is somewhat random, I cannot always reproduce it.

As an example, the first time I added puresnmp I had no problems, and I've been able to have my package working by removing puresnmp from the dependencies, call poetry install and then poetry add puresnmp: I suppose the problem can show itself when packages using the typing module in a way that is somehow specific to the Python version in use are installed after puresnmp has installed the wrong version of typing.

In any case, I think you are reproducing the error if you find the typing module installed in the virtualenv, i. e. poetry show --tree puresnmp shows typing as a dependency.

[exhuma]

That's really strange as I don't see it in the dependencies in the tree for me. I will investigate a bit further if this is blocking for you.

If it is not blocking however, I would prefer to close this issue if I don't find anything conclusive and focus on the release of the upcoming version.

What do you think?

[davnat]

Really strange indeed: I tried the commands you mentioned (or very similar):

mkdir test
cd test
poetry init -n
poetry add puresnmp
poetry add -D black

on both Ubuntu 20.04 (Python 3.8.5) and Debian Buster (Python 3.7.3), and on both machines:

• poetry add puresnmp installed typing 3.7.4.3
• poetry add -D black produced an error

Here's the one produced on Ubuntu (the other is nearly identical):

$ poetry add -D black
Using version ^20.8b1 for black

Updating dependencies
Resolving dependencies... (1.9s)

Writing lock file

Package operations: 9 installs, 0 updates, 0 removals

  • Installing appdirs (1.4.4)
  • Installing click (7.1.2)
  • Installing mypy-extensions (0.4.3)
  • Installing pathspec (0.8.1)
  • Installing regex (2020.11.13)
  • Installing toml (0.10.2)
  • Installing typed-ast (1.4.1)
  • Installing typing-extensions (3.7.4.3)
  • Installing black (20.8b1): Failed

  EnvCommandError

  Command ['/home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/bin/pip', 'install', '--no-deps', 'file:///home/davide/.cache/pypoetry/artifacts/38/f4/61/550fd050005dc1c64850374a7f98dd941a4633d72e87f147bebf26bdc3/black-20.8b1.tar.gz'] errored with the following return code 1, and output: 
  Processing /home/davide/.cache/pypoetry/artifacts/38/f4/61/550fd050005dc1c64850374a7f98dd941a4633d72e87f147bebf26bdc3/black-20.8b1.tar.gz
    Installing build dependencies: started
    Installing build dependencies: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/bin/python /home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-d69pdeyn/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=41.0' setuptools-scm wheel
         cwd: None
    Complete output (42 lines):
    Traceback (most recent call last):
      File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "/home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/lib/python3.8/site-packages/pip/__main__.py", line 26, in <module>
        sys.exit(_main())
      File "/home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/lib/python3.8/site-packages/pip/_internal/cli/main.py", line 73, in main
        command = create_command(cmd_name, isolated=("--isolated" in cmd_args))
      File "/home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/lib/python3.8/site-packages/pip/_internal/commands/__init__.py", line 105, in create_command
        module = importlib.import_module(module_path)
      File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
      File "<frozen importlib._bootstrap>", line 991, in _find_and_load
      File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 783, in exec_module
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "/home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 17, in <module>
        from pip._internal.cli.req_command import RequirementCommand, with_cleanup
      File "/home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 18, in <module>
        from pip._internal.index.collector import LinkCollector
      File "/home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/lib/python3.8/site-packages/pip/_internal/index/collector.py", line 14, in <module>
        from pip._vendor import html5lib, requests
      File "/home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/lib/python3.8/site-packages/pip/_vendor/requests/__init__.py", line 125, in <module>
        from . import utils
      File "/home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/lib/python3.8/site-packages/pip/_vendor/requests/utils.py", line 25, in <module>
        from . import certs
      File "/home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/lib/python3.8/site-packages/pip/_vendor/requests/certs.py", line 15, in <module>
        from pip._vendor.certifi import where
      File "/home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/lib/python3.8/site-packages/pip/_vendor/certifi/__init__.py", line 1, in <module>
        from .core import contents, where
      File "/home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/lib/python3.8/site-packages/pip/_vendor/certifi/core.py", line 12, in <module>
        from importlib.resources import path as get_path, read_text
      File "/usr/lib/python3.8/importlib/resources.py", line 11, in <module>
        from typing import Iterable, Iterator, Optional, Set, Union   # noqa: F401
      File "/home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/lib/python3.8/site-packages/typing.py", line 1359, in <module>
        class Callable(extra=collections_abc.Callable, metaclass=CallableMeta):
      File "/home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/lib/python3.8/site-packages/typing.py", line 1007, in __new__
        self._abc_registry = extra._abc_registry
    AttributeError: type object 'Callable' has no attribute '_abc_registry'
    ----------------------------------------
  ERROR: Command errored out with exit status 1: /home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/bin/python /home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-d69pdeyn/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=41.0' setuptools-scm wheel Check the logs for full command output.
  WARNING: You are using pip version 20.3.1; however, version 20.3.3 is available.
  You should consider upgrading via the '/home/davide/.cache/pypoetry/virtualenvs/test-rVkrY_vP-py3.8/bin/python -m pip install --upgrade pip' command.
  

  at ~/.local/lib/python3.8/site-packages/poetry/utils/env.py:1074 in _run
      1070│                 output = subprocess.check_output(
      1071│                     cmd, stderr=subprocess.STDOUT, **kwargs
      1072│                 )
      1073│         except CalledProcessError as e:
    → 1074│             raise EnvCommandError(e, input=input_)
      1075│ 
      1076│         return decode(output)
      1077│ 
      1078│     def execute(self, bin, *args, **kwargs):


Failed to add packages, reverting the pyproject.toml file to its original content.

As already mentioned, my workaround is to remove puresnmp from pyproject.toml, install the package and re-add puresnmp, so I'm not completely blocked by this. Nevertheless, this workaround won't work in our CI, so it feels somewhat uncomfortable to me.

I only started using this package recently, and I only have ~200 lines of code depending on it: it won't be a problem for me to upgrade to the 2.x version, but I really need a working package, because the project needs to be delivered very soon.

Besides this, I'm starting considering that this could be a Poetry issue, because AFAIK poetry add puresnmp should produce the very same poetry.lock file everywhere...

I'm using Poetry 1.1.4 installed via pip3 install --user poetry on both my machines.

[exhuma]

Might also be related to the Python version. I've tested it in 3.6 when I tried to repro it.

In any case, as you mention that this is an annoyance in your CI I will get to it and re-release. I hate it myself when the CI is borked because of a messy package 😉

[exhuma]

I have released v1.10.2.post1 just now which switched over to poetry for packaging.

Can you let me know if this fixed the issue?

[davnat]

I confirm that version 1.10.2.post1 fixes the issue: I just tried on both my machines and everything is now working as expected (the typing module is not installed, and black installation works).

Thank you very much, this is very appreciated.

[exhuma]

No worries. And happy coding 😄