Python fails on TLS connections due to missing certificates

[rolisoft]

Python fails on TLS connections due to missing certificates

[RoliSoft]

Hi,

Alpine uses busybox, which doesn't seem to work under WSL as a replacement for bash. Not sure about void.

This isn't an issue with the distribution switcher, rather with WSL crashing on distributions that deviate too much from the standard.

I implemented hooks for fixing most of the distributions' issues, but some distributions plain and simple don't start. You could try some more popular at first, such as Debian or Fedora, which will work.

[biax]

im actually an archlinux user, since the two mentioned dont work, i wanna try archlinux, what commands do i need to get archlinux running?

i've tried:

get-source.py archlinux
get-source.py base/archlinux
get-prebuilt.py archlinux
get-prebuilt.py base/archlinux

it seems that all get-prebuilt.py commands give me:

Traceback (most recent call last): File "get-prebuilt.py", line 25, in <module> with urllib.request.urlopen('https://auth.docker.io/token?service=registry.docker.io&scope=repository:%s:pull' % fimage) as f: File "C:\Users\admin\AppData\Local\Programs\Python\Python35\lib\urllib\request.py", line 163, in urlopen return opener.open(url, data, timeout) File "C:\Users\admin\AppData\Local\Programs\Python\Python35\lib\urllib\request.py", line 466, in open response = self._open(req, data) File "C:\Users\admin\AppData\Local\Programs\Python\Python35\lib\urllib\request.py", line 484, in _open '_open', req) File "C:\Users\admin\AppData\Local\Programs\Python\Python35\lib\urllib\request.py", line 444, in _call_chain result = func(*args) File "C:\Users\admin\AppData\Local\Programs\Python\Python35\lib\urllib\request.py", line 1297, in https_open context=self._context, check_hostname=self._check_hostname) File "C:\Users\admin\AppData\Local\Programs\Python\Python35\lib\urllib\request.py", line 1256, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)>

[RoliSoft]

VoidLinux works for me:

C:\Users\RoliSoft\Desktop\wsl-distrib > python get-prebuilt.py voidlinux/voidlinux
[*] Requesting authorization token...
[*] Fetching manifest info for voidlinux/voidlinux:latest...
[*] Downloading layer sha256:cc40ca6dff2d219ffebac10ec6c623dd00a717c157d2abd3e0c9a39ea877e692...
[*] Rootfs archive for voidlinux/voidlinux:latest saved to rootfs_voidlinux_voidlinux_latest.tar.gz.

C:\Users\RoliSoft\Desktop\wsl-distrib > python install.py voidlinux/voidlinux
[*] Probing the Linux subsystem...
[*] Default user is RoliSoft at /home/RoliSoft.
[*] Reading /etc/{passwd,shadow,group,gshadow} entries for root and RoliSoft...
[*] Removing leftover rootfs-temp...
[*] Beginning extraction...
[*] Backing up current rootfs to rootfs_debian_sid...
[*] Switching to new rootfs...
[*] Writing entries of root and RoliSoft to /etc/{passwd,shadow,group,gshadow}...
[*] Switching default user to root...
[*] Running post-install hook all...
[*] Detecting operating system...
Unsupported operating system.
[!] Failed to run hook in WSL: Command '['cmd', '/C', 'C:\\WINDOWS\\System32\\bash.exe', '-c', 'REGULARUSER="RoliSoft" WINVER="14393" /root/hook_postinstall_all.sh']' returned non-zero exit status 1
[*] Switching default user back to RoliSoft...

C:\Users\RoliSoft\Desktop\wsl-distrib > bash
RoliSoft@ROLISOFT-PC ˜/wsl-distrib $ xbps-install -S
[*] Updating `http://repo.voidlinux.eu/current/x86_64-repodata' ...

[RoliSoft]

To install ArchLinux run:

python get-prebuilt.py pritunl/archlinux
python install.py pritunl/archlinux

If you get Python errors, you can try the compiled version of the scripts from here: https://github.com/RoliSoft/WSL-Distribution-Switcher/releases

In that case, instead of python whatever.py run whatever.exe and the parameters.

[RoliSoft]

Also, the error you're getting from get-prebuilt.py is that it can't verify the SSL certificate of auth.docker.io.

You either have a weird setup where a firewall is MITM-ing you, or your Python installation is broken. Try the compiled exes.

[biax]

Okay you might wanna write a note in your readme. musl-based distros don't seem to work with WSL.

https://repo.voidlinux.eu/live/current/

I downloaded rootfs-musl version which was why i had the error. I now downloaded the glibc version, and install.py, it just worked.

[biax]

I'm using python-3.5.2-amd64.exe

Definitely not firewall issue. I'll try your compiled version.

Now that voidlinux is running, are you able to see if anything obvious needs patching? You mentioned something about hooks on your readme.

[RoliSoft]

python-3.5.2-amd64.exe

I'm using the same, so it should work fine. You could try running:

pip install certifi

While this installs a CA database, not sure if this actually fixes your problem, since this database won't be used by default, as far as I can tell.

[biax]

anyway, i can't get the package manager to work in voidlinux.

lxrun /setdefaultuser root
xbps-install -S # syncs

it immediately tells me "Killed"

Do you know a quick fix for it? If you don't I'm gonna give up and just move to archlinux now.

[RoliSoft]

Alpine also uses musl it seems. The issue might actually be this. I'll do some more experiments later and update the README accordingly.

As for your issue, VoidLinux is not a distribution I am familiar with (other than hearing about it) and there is no support for it in the hooks.

So, if you want a smooth experience, I'd recommend using pritunl/archlinux, since that is what I optimized it for. Make sure to rename the hook_postinstall_all.sample.sh file to hook_postinstall_all.sh, so the hook script actually runs after installation, and fixes the quirks.

The post-install script is non-interactive, so just let it do its thing. If you interrupt it with Ctrl+C, you might end up with a broken installation. (Of course, you can always just re-run the installation for a fresh start.)

[biax]

Running your latest release get-prebuilt.exe also gives me that ssl cert error

[RoliSoft]

Interesting. Also, the sync command works for me, it's not killed:

RoliSoft@ROLISOFT-PC ˜ $ su -l
Password:
# xbps-install -S
[*] Updating `http://repo.voidlinux.eu/current/x86_64-repodata' ...
x86_64-repodata: 1161KB [avg rate: 12GB/s]
#

Not sure how to proceed with the troubleshooting.

[biax]

I'm on 14393, you?

[RoliSoft]

Same, the anniversary update. Only using Insider in VMs.

[biax]

Any suggestions? Start fresh? Honestly its my first time using WSL.

So how do I do this? lxrun /uninstall /full?

[RoliSoft]

Yes, and then reinstall it, make sure you have a working Ubuntu installation first.

Then try installing something like pritunl/archlinux, which should work flawlessly.

As for your issue, you could try seeing if you can fix your Python installation: https://www.google.com/search?q=python+windows+ssl+certificate+verify+failed

Alternatively, you can edit the get-prebuilt.py script, add to header:

import ssl

ssl._create_default_https_context = ssl._create_unverified_context

This monkeypatches SSL verification to ignore verification errors.

[biax]

the monkeypatch fixed it for now.

voidlinux is working now without the "killed". what are the sort of things that i should test to see if everything is good?

i saw comments on "dnf" or something.

[biax]

note: voidlinux original rootfs tarball from https://repo.voidlinux.eu/live/current/ spews that killed error. using the docker version is fine (strangely?)

[biax]

i dont know why i have this python issue (But not you) but anyway.. i asked folks on irc and they seem to think that python isnt communicating with windows cert store.

unfortunately, i do not know anything about python. so ill just leave you some information that you might find useful:

biax: https://docs.python.org/3.2/library/urllib.request.html#urllib.request.urlopen you can patch the global urlopener to use the ca path in certifi

https://pypi.python.org/pypi/wincertstore

[biax]

also, are you familiar with this? https://github.com/turbo/alwsl

Archlinux appears to be working with your setup, any idea what they are trying to accomplish that your setup cant?

Pardon my many comments & questions.

Thanks for this wonderful piece of software by the way!

[RoliSoft]

i saw comments on "dnf" or something.

dnf is Fedora's package manager. One of the issues with Arch was chrooting, but the postinstaller fixes that completely.

note: voidlinux original rootfs tarball from ... spews that killed error.

Maybe it needs some initialization? Not sure. You could check the Dockerfile of one of the working VoidLinux docker archives to check what they do in extra after including the rootfs.

you can patch the global urlopener to use the ca path in certifi

I know I can, but since no one met this issue before, I did not know I had to. I'm still not sure how to proceed, since for other users it works properly, and I don't want to just shovel on dependencies if not absolutely required.

Archlinux appears to be working with your setup, any idea what they are trying to accomplish that your setup cant?

There isn't really much competition here, since we're trying to do different things. My project tries to be as generic as possible, support as many distributions as possible by supporting a wide range of sources (docker hub [prebuilt], docker library [sources], rootfs archives, squashfs images) and fix any known issues if the distribution is recognized.

alwsl is an ArchLinux distribution tweaked to work under WSL. The batch file downloads a prebuilt rootfs image from the owner's servers and installs that as rootfs. If you only want to use ArchLinux, and my script is giving you a hard time, you could use that project as well.

[biax]

alright then. in any case, things are up and running so till again then.

truly appreciate all your help.

[RoliSoft]

So I ran the script under a debugger, and it seems that they were right on IRC. \Python35\Lib\ssl.py has load_default_certs which calls _load_windows_store_certs and automatically reads the Windows Certificate Store.

In your case, it fails to communicate with it. Try running it as admin? Although you shouldn't need to, but at least we can check if this changes anything.

[biax]

I did run it using a created admin account with UAC disabled even. I've never had any privileges issue. I'll try opening cmd as administrator...

Update: same story running cmd as admin.

[biax]

Maybe I shouldn't edit/update comments so I'll repeat again:

same issue running cmd as administrator

On 2016-10-24 04:28, Roland Bogosi wrote:

So I ran the script under a debugger, and it seems that they were
right on IRC. |\Python35\Lib\ssl.py| has |load_default_certs| which
calls |_load_windows_store_certs| and automatically reads the Windows
Certificate Store.

In your case, it fails to communicate with it. Try running it as
admin? Although you shouldn't need to, but at least we can check if
this changes anything.

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
#20 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AOdsCsH1yN-43e3i9Z0bmIsaWj1y89Jqks5q28NWgaJpZM4KeMoK.

[RoliSoft]

Please check whether commit 1adbfbc fixes your issue.

If the certificate checking works properly, it should fall back to using certifi, or fail with a message asking you to install certifi.