Publish as a wheel – and hash issues with newly-published distributions on existing releases v2.1.7, v2.1.6, v2.1.5

[springload]

Publish as a wheel – and hash issues with newly-published distributions on existing releases v2.1.7, v2.1.6, v2.1.5

[thibaudcolas]

Hey @njt1982, thank you for reporting this and… sorry for the time it must’ve taken you to troubleshoot this 😳

Since the source distribution hasn’t changed, I was expecting people who were using pipenv / Poetry / requirements.txt hashes to still get the same distribution as before, not that those tools would ask you to change the hashes.

It looks like what has happened instead is that pipenv resolved the install to the new wheel, even though it resolved to the source distribution before, and then complains that the distribution’s hash is different. It might be because pipenv doesn’t keep the "name of the distribution it resolved to" in its lockfile?

---

I’m not sure this is the Python packaging ecosystem working as intended, or a problem, but it’s good to have this info out there for others as you mention.

Looking at freedomofpress/securethenews@98002b5 linked to above your comment, it looks like pip also fails, but asks for the additional hash to be added for the new distribution. I’d have expected it also to keep on installing the existing source distrib.

Edit: actually, a colleague did warn me that this would be happening for older versions of pip. But I hadn’t researched which versions exactly. Would be interested to hear more about this if anyone does the research / stumbles upon this and has the info.

I don’t think there is much I can do to fix this now, but I’ll re-name, re-open, and pin this issue so people running into this can at least find the info more easily.

[njt1982]

I don’t think there is much I can do to fix this now

Yeah - I think the best thing to do is leave it as it (otherwise the risk is breaking it again for those who have just fixed it :) ).

It's fine - I think it was a reasonable assumption that existing lock files would continue to use the same source.

Good idea to pin it for visibility.

---

TBH my initial concern was this line:

Otherwise, examine the package contents carefully; someone may have tampered with them.

I wondered if someone had hijacked the package at source. 😉

[thibaudcolas]

It’s been a month without any further activity so I’ll now close this again.

[thibaudcolas]

After discussion on #133 and further research / consulting with others, I have:

• Published wheels for existing releases for v2.1.5, v2.1.6, v2.1.7. It’s intentional for PyPI to support this (see pypa/packaging-problems#75), and people who really want to lock down their reproducible builds can use hashes. For everyone else this should not cause any particular problems.
• Published a wheel for v4.1.0, as a new release, in addition to the usual source distribution
• All future releases will have wheel and source distributions from day one.

I was a weary of breaking anyone’s builds by publishing extra artifacts on existing releases, but it has the big advantage of not needing any extra work on the git/GitHub side, which is a big plus for a library like this that currently only has one active branch for the latest release, and generally doesn’t release bug fixes for non-latest releases.

[njt1982]

Hi,

FYI for anyone else who come across this... 😉

This caused us to need to update the hash in our lock file:

[pipenv.exceptions.InstallError]: Collecting draftjs-exporter==2.1.7 (from -r /tmp/pipenv-3xi9mnch-requirements/pipenv-wko15_do-requirement.txt (line 1))
 
[pipenv.exceptions.InstallError]:   Using cached https://files.pythonhosted.org/packages/43/21/5ecec14572c86eed3173b8cd8ece268ec20be2faa4ead09669531739d31e/draftjs_exporter-2.1.7-py3-none-any.whl
 
[pipenv.exceptions.InstallError]: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
 
[pipenv.exceptions.InstallError]:     draftjs-exporter==2.1.7 from https://files.pythonhosted.org/packages/43/21/5ecec14572c86eed3173b8cd8ece268ec20be2faa4ead09669531739d31e/draftjs_exporter-2.1.7-py3-none-any.whl#sha256=d415a9964690a2cddb66a31ef32dd46c277e9b80434b94e39e3043188ed83e33 (from -r /tmp/pipenv-3xi9mnch-requirements/pipenv-wko15_do-requirement.txt (line 1)):
 
[pipenv.exceptions.InstallError]:         Expected sha256 5839cbc29d7bce2fb99837a404ca40c3a07313f2a20e2700de7ad6aa9a9a18fb
 
[pipenv.exceptions.InstallError]:              Got        d415a9964690a2cddb66a31ef32dd46c277e9b80434b94e39e3043188ed83e33
 
ERROR: Couldn't install package: draftjs-exporter
 
 Package installation failed...

That old hash has been working fine for the last 10 months but started failing today.

Needed to change it from 5839cbc29d7bce2fb99837a404ca40c3a07313f2a20e2700de7ad6aa9a9a18fb to d415a9964690a2cddb66a31ef32dd46c277e9b80434b94e39e3043188ed83e33