Single import-dependence on pynng causes M1 Mac install error

[materialsproject]

Single import-dependence on pynng causes M1 Mac install error

[munrojm]

We appreciate the offer. Ideally, I think it would be good if there was another well-maintained library we could use. My plan is to dedicate some time later this week to working through some maggma issues. I will look more into the details then. Do you have a sense of how much would need to be added? There looks to be a lot of stuff in the Socket class.

[shyamd]

Another option is ZeroMQ

[janosh]

@munrojm Fix confirmed! Installation is error-free on the zero_mq branch. Thanks a lot.

pip install logs

$ git checkout zero_mq
$ pip install .
Processing /Users/janosh/Repos/maggma
  Preparing metadata (setup.py) ... done
Requirement already satisfied: setuptools in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from maggma==0.38.2.dev24+gdd58756) (59.5.0)
Requirement already satisfied: pymongo>=4.0 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from maggma==0.38.2.dev24+gdd58756) (4.0.1)
Requirement already satisfied: monty>=1.0.2 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from maggma==0.38.2.dev24+gdd58756) (2022.1.12.1)
Requirement already satisfied: mongomock>=3.10.0 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from maggma==0.38.2.dev24+gdd58756) (3.23.0)
Requirement already satisfied: pydash>=4.1.0 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from maggma==0.38.2.dev24+gdd58756) (5.1.0)
Requirement already satisfied: jsonschema>=3.1.1 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from maggma==0.38.2.dev24+gdd58756) (4.4.0)
Requirement already satisfied: tqdm>=4.19.6 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from maggma==0.38.2.dev24+gdd58756) (4.62.3)
Requirement already satisfied: mongogrant>=0.3.1 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from maggma==0.38.2.dev24+gdd58756) (0.3.3)
Requirement already satisfied: aioitertools>=0.5.1 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from maggma==0.38.2.dev24+gdd58756) (0.8.0)
Requirement already satisfied: numpy>=1.17.3 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from maggma==0.38.2.dev24+gdd58756) (1.22.1)
Requirement already satisfied: pydantic>=0.32.2 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from maggma==0.38.2.dev24+gdd58756) (1.8.2)
Requirement already satisfied: fastapi>=0.42.0 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from maggma==0.38.2.dev24+gdd58756) (0.72.0)
Requirement already satisfied: pyzmq==22.3.0 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from maggma==0.38.2.dev24+gdd58756) (22.3.0)
Requirement already satisfied: dnspython>=1.16.0 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from maggma==0.38.2.dev24+gdd58756) (2.1.0)
Requirement already satisfied: sshtunnel>=0.1.5 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from maggma==0.38.2.dev24+gdd58756) (0.4.0)
Requirement already satisfied: msgpack>=0.5.6 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from maggma==0.38.2.dev24+gdd58756) (1.0.3)
Requirement already satisfied: orjson>=3.6.0 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from maggma==0.38.2.dev24+gdd58756) (3.6.5)
Requirement already satisfied: starlette==0.17.1 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from fastapi>=0.42.0->maggma==0.38.2.dev24+gdd58756) (0.17.1)
Requirement already satisfied: anyio<4,>=3.0.0 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from starlette==0.17.1->fastapi>=0.42.0->maggma==0.38.2.dev24+gdd58756) (3.4.0)
Requirement already satisfied: attrs>=17.4.0 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from jsonschema>=3.1.1->maggma==0.38.2.dev24+gdd58756) (21.3.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from jsonschema>=3.1.1->maggma==0.38.2.dev24+gdd58756) (0.18.0)
Requirement already satisfied: Click in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from mongogrant>=0.3.1->maggma==0.38.2.dev24+gdd58756) (8.0.3)
Requirement already satisfied: Flask>=1.0 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from mongogrant>=0.3.1->maggma==0.38.2.dev24+gdd58756) (2.0.2)
Requirement already satisfied: requests in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from mongogrant>=0.3.1->maggma==0.38.2.dev24+gdd58756) (2.27.1)
Requirement already satisfied: sentinels in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from mongomock>=3.10.0->maggma==0.38.2.dev24+gdd58756) (1.0.0)
Requirement already satisfied: six in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from mongomock>=3.10.0->maggma==0.38.2.dev24+gdd58756) (1.16.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from pydantic>=0.32.2->maggma==0.38.2.dev24+gdd58756) (4.0.1)
Requirement already satisfied: paramiko>=2.7.2 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from sshtunnel>=0.1.5->maggma==0.38.2.dev24+gdd58756) (2.9.1)
Requirement already satisfied: Jinja2>=3.0 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from Flask>=1.0->mongogrant>=0.3.1->maggma==0.38.2.dev24+gdd58756) (3.0.3)
Requirement already satisfied: itsdangerous>=2.0 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from Flask>=1.0->mongogrant>=0.3.1->maggma==0.38.2.dev24+gdd58756) (2.0.1)
Requirement already satisfied: Werkzeug>=2.0 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from Flask>=1.0->mongogrant>=0.3.1->maggma==0.38.2.dev24+gdd58756) (2.0.2)
Requirement already satisfied: cryptography>=2.5 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from paramiko>=2.7.2->sshtunnel>=0.1.5->maggma==0.38.2.dev24+gdd58756) (3.4.8)
Requirement already satisfied: pynacl>=1.0.1 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from paramiko>=2.7.2->sshtunnel>=0.1.5->maggma==0.38.2.dev24+gdd58756) (1.4.0)
Requirement already satisfied: bcrypt>=3.1.3 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from paramiko>=2.7.2->sshtunnel>=0.1.5->maggma==0.38.2.dev24+gdd58756) (3.2.0)
Requirement already satisfied: idna<4,>=2.5 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from requests->mongogrant>=0.3.1->maggma==0.38.2.dev24+gdd58756) (3.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from requests->mongogrant>=0.3.1->maggma==0.38.2.dev24+gdd58756) (2.0.9)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from requests->mongogrant>=0.3.1->maggma==0.38.2.dev24+gdd58756) (1.26.7)
Requirement already satisfied: certifi>=2017.4.17 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from requests->mongogrant>=0.3.1->maggma==0.38.2.dev24+gdd58756) (2021.10.8)
Requirement already satisfied: sniffio>=1.1 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from anyio<4,>=3.0.0->starlette==0.17.1->fastapi>=0.42.0->maggma==0.38.2.dev24+gdd58756) (1.2.0)
Requirement already satisfied: cffi>=1.1 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from bcrypt>=3.1.3->paramiko>=2.7.2->sshtunnel>=0.1.5->maggma==0.38.2.dev24+gdd58756) (1.15.0)
Requirement already satisfied: MarkupSafe>=2.0 in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from Jinja2>=3.0->Flask>=1.0->mongogrant>=0.3.1->maggma==0.38.2.dev24+gdd58756) (2.0.1)
Requirement already satisfied: pycparser in /Users/janosh/.venv/py310/lib/python3.10/site-packages (from cffi>=1.1->bcrypt>=3.1.3->paramiko>=2.7.2->sshtunnel>=0.1.5->maggma==0.38.2.dev24+gdd58756) (2.21)
Building wheels for collected packages: maggma
  Building wheel for maggma (setup.py) ... done
  Created wheel for maggma: filename=maggma-0.38.2.dev24+gdd58756-py3-none-any.whl size=82120 sha256=c9c9f9432e573b1540c5a266cb6c68a1244ede3485d5d477f0745e97e43134cf
  Stored in directory: /private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-ephem-wheel-cache-cjwcqz5v/wheels/88/14/03/17c3c16e3f5dc370a0e5e126414cdd74713b7d767f0831870b
Successfully built maggma
Installing collected packages: maggma
  Attempting uninstall: maggma
    Found existing installation: maggma 0.38.2.dev25+gd7d93f0.d20220125
    Uninstalling maggma-0.38.2.dev25+gd7d93f0.d20220125:
      Successfully uninstalled maggma-0.38.2.dev25+gd7d93f0.d20220125
Successfully installed maggma-0.38.2.dev24+gdd58756

[munrojm]

Great! Thank you again for bringing this up. Will close the issue when I finalize and merge.

[janosh]

pynng appears to be unmaintained which would be a reason to incorporate Pair1 into maggma.

[munrojm]

Hi @janosh, we appreciate you bringing this up. Maintaining Pair1 functionality within the code could be one option. Before deciding, I would like to take some time to see if there are any other workable solutions. I will keep this thread open until we have a resolution.

[janosh]

Happy to submit the PR that incorporates Pair1 into maggma if no other solutions arise.

[munrojm]

Looks like a good option. I'll give it a shot.

[munrojm]

@janosh, I have replaced the use of Pair1 with ZeroMQ's REQ/REP client-server setup through pyzmq. Would you have any time to test out installation? I unfortunately don't have access to an M1 machine. It is in the zero_mq branch (PR #543).