ENOMEM on ussl.wrap_socket() running upip on ESP32

[pfalcon]

ENOMEM on ussl.wrap_socket() running upip on ESP32

[pfalcon]

Yes, ENOMEM is "not enough memory", but not on Python side, but on system side. If you look at:

File "upip.py", line 151, in url_open

It is:

s = ussl.wrap_socket(s, server_hostname=host)

I.e., there it creates an SSL socket. That may require allocating quite large buffers, up to 16KB, and that of contiguous memory. I don't use ESP32, so not sure how large buffer(s) are allocated for it. E.g. on ESP8266 buffer specifically set to a low value (4-5K) to increase the chance of successful SSL socket creation (but then amount of data transferred is usually limited to that amount).

>>> gc.mem_free()
63680

That shows amount of total free memory, here the size of the largest contiguous block would be important. A general suggestion for cases like this is "try to run upip immediately after reset, when memory isn't yet fragmented by other operations".

[MathijsNL]

Thanks for your reply.

My main struggle is that after uploading the regular MicroPython firmware (daily build) Upip works and this error doesn't show up at all.

After uploading the Pycopy firmware immediately after booting it can't install a single package because this error is always there.

Do you have any suggestions on how to figure this out? Could I have made an error compiling the firmware?

[pfalcon]

That's indeed a strange and sad situation. Pycopy is somewhat behind the upstream in ESP32-related changes, I'm pulling them in these days, hopefully that might resolve the issue. Otherwise, the main focus of Pycopy is development of language-level features, nor particular hardware platforms, I treat Unix port as the reference platform.

So, trying to reproduce the issue with Unix port would be the way to tell whether it's regression in Pycopy/upip, or "hardware-specific" issue with ESP32. Note that it's easy to mimic a small heap condition on Unix port, see pycopy --help.

A quick test shows that following works:

pycopy -X heapsize=8wK -m upip install -p . picoweb

(For cleanness of experiment, would of course need to build 32-bit unix version, the above is with default-nowadays 64-bit.)

[pfalcon]

Looks like that's it: pfalcon/pycopy@cbf14fd

[MathijsNL]

Confirmed, it is working perfectly now. Can this be closed?

MicroPython v2.11.0.16-179-g979ec245b on 2019-12-06; ESP32 module with ESP32
Type "help()" for more information.
>>>
>>> import upip
>>> upip.install('picoweb')
Installing to: /lib/
Warning: pypi.org SSL certificate is not validated
Installing picoweb from https://files.pythonhosted.org/packages/27/47/5d70d23ecc778bff5fdae3ef2eb57d59aff4a260da1b90