Windows: v1.3.1 stopped working with VPN on

[eugenesvk]

Windows: v1.3.1 stopped working with VPN on

[Sculas]

Which VPN are you using? If it's one where it uses a custom network adapter instead of intercepting the default one, it might explain why it's choosing Ethernet/WiFi over the VPN network adapter.

Well... it looks like I misread the issue initially, sorry about that. I mistakenly thought the VPN was blocking the primary Ethernet/LAN adapter (aka killswitch in most VPNs), but looking better at the verbose logs, it says Default Interface not found (which changes the story).

Anyway, I've implemented verbose logging in my dns-adapter-fix branch, which you can install using this command:

cargo install cargo-binstall --locked --force --git https://github.com/Sculas/cargo-binstall.git --branch dns-adapter-fix

Then, run this command (using cargo-nextest as an example here), again while you're connected to your VPN:

cargo binstall cargo-nextest -y --log-level trace --json-output > out.txt

Open the out.txt file and look for Using custom DNS resolver configuration, it should be around line 10. Copy the text until it says Resolver configuration complete, censor/remove anything you'd rather not share (e.g. DNS servers), and send it here (using a <details> tag if very long).

Lastly, run ipconfig and check if the IPv4/IPv6 addresses are the same as the local IP address mentioned in out.txt above. Are they different or not? Does the aforementioned local IP address correspond to any of the adapters shown in ipconfig?

Unfortunately, I don't have a VPN client at hand that uses OpenVPN, so I have to ask a lot of questions, but I'm sure we'll figure it out :)

[Sculas]

@NobodyXu Done! #1436

[Sculas]

Are you sure you're on the right repository? You should be on https://github.com/cargo-bins/cargo-binstall, not https://github.com/Sculas/cargo-binstall (this is my fork, which is outdated as of writing).

EDIT: Looking at the logs, you're also missing the logs that were added in #1436, which confirms that you're on the wrong repository :)

[Sculas]

Ah, I only just now noticed you're on Linux 😅. That explains why I couldn't find the log again (I was searching for the Windows log line). Given that this was initially only a problem on Windows, we only implemented the fix for that and kept using the auto system config on Linux. It really does look like trust-dns's system config is unreliable.

Normally I would now say "Why is trust-dns used anyway given the issues it's causing? Can't it be disabled by default?", but I'm assuming it was done for a reason, so we should fix it instead. Seems like we'll just have to port over the changes from the Windows-side to the Linux-side (find a replacement for the ipconfig crate, default_net can be removed since we're only using 1 small function which we can extract).

[Sculas]

Which VPN are you using? If it's one where it uses a custom network adapter instead of intercepting the default one, it might explain why it's choosing Ethernet/WiFi over the VPN network adapter. Seems like we're missing debug logs in the resolver, that's definitely something that needs to be added, because it'll tell us what's happening. I'm currently unable to do this myself, so if someone wants to do this: log the available adapter names, possible candidates and the adapter which is selected. If not, I'll look into PR'ing that myself soon so we can start figuring out the issue.

[Sculas]

That could very well be correct. Since I wanted to log all candidates (and not just the one that was found first), I rewrote the default_net::get_default_interface() function but found I could skip the for-loop done by default_net because I didn't need the default_net interface, but the one from ipconfig. I guess that accidentally fixed it? (could be a bug in default_net::interfaces()?)

@NobodyXu I didn't spend too much effort in that branch since I just wanted to know if it works or not. Is it okay if I PR the branch and resolve any of the requested changes you may have?

[NobodyXu]

I would probably release a new version this weekend

[Sculas]

Hmm... trust-dns's system config reads the nameservers from /etc/resolv.conf, which could mean something might be wrong there? There might be more to this, but I'll have to look at the logs properly when I'm back.

Anyway, I think we should move this to a new issue rather than pinging the issue author every time for an unrelated problem :)

[Sculas]

That makes sense. Hence I assumed you had a valid reason for it :)

[NobodyXu]

cc @Sculas @GNQG

[passcod]

Can you upgrade to 1.4.1 to see if that helps any by any random chance?

[eugenesvk]

Can you upgrade to 1.4.1 to see if that helps any by any random chance?

that's the one I've started with to check the new great package signatures feature, it's also broken

[NobodyXu]

I think we will have to add more logging to discover the problem.
Would like to do this myself but is busy right now.

[eugenesvk]

Which VPN are you using?

I'm using OpenVPN, don't know whether that uses a custom network adapter (is it this TUN thing?)

(the vpn provider is PIA, tested their native app as well, same result, also not sure whether/what kind of adapters they are using)

[Sculas]

@eugenesvk You can check which network adapters exist in the Control Panel. You can find it manually, or press Win + R and run this command to open the Network Connections window immediately: explorer ::{7007ACC7-3202-11D1-AAD2-00805FC1270E}.
That should open a tab listing all of the available network adapters. If possible, can you send a screenshot of your network adapters while you're connected to your VPN? (if needed, you can censor anything you don't want to share)

[eugenesvk]

this is the active list while connecting to the OpenVPN client, so it does indeed use a network adapter (though it's TAP, not TUN :))

and the inactive ones (don't think these are relevant, but not sure

[eugenesvk]

Thanks, will build&check your branch later

Unfortunately, I don't have a VPN client at hand that uses OpenVPN,

Maybe try with a native client as that one also bugs for me, I've just tested with OpenVPN as a double-check

[Sculas]

Maybe try with a native client as that one also bugs for me, I've just tested with OpenVPN as a double-check

I only have AdGuard VPN which intercepts the primary Ethernet adapter instead of having its own network adapter, hence I'm unable to reproduce this myself.

[eugenesvk]

Hm, your version works fine (checked again the downloaded 1.4.2, and that one still fails).

Can the fact that it's buit locally vs. downloaded from the CI explain it (the locally built version is also much smaller, 8.5m vs 13.5m)?

Not sure whether the logs are relevant without an error, but FYI https://gist.github.com/eugenesvk/4a315b6ab8855bac2749a14deef07f9e

The IP address matches the one from OpenVPN TAP-Windows6 in ipconfig

[NobodyXu]

dns-adapter-fix contains latest update on main: We upgrade to trust-dns-resolver v0.23 and update transitive dependencies, which could be the reason why you cannot reproduce it.

@eugenesvk Can you try installing the latest version from main via cargo install cargo-binstall --git https://github.com/cargo-bins/cargo-binstall?

[eugenesvk]

the main repo still fails, so I guess the refactoring done to add logging also fixed the issue?

[NobodyXu]

Yes @Sculas

[ilyagr]

Update: I used the wrong repo to install from. You can ignore most of this comment. See my comment below instead.

I have a very similar error that I get when using binstall from a work computer.

I just compiled cargo binstall with cargo install cargo-binstall --locked --force --git https://github.com/Sculas/cargo-binstall.git --branch main, which should include the fix, and it's still failing for me.

I ran cargo binstall cargo-nextest -y --log-level trace --json-output | tee out.txt. Here's the result.

cargo-binstall-logs.txt

The final error from the file is:

Fatal error:
  × For crate cargo-nextest: could not GET https://index.crates.io/
  │ config.json: error sending request for url (https://index.crates.io/
  │ config.json): error trying to connect: tcp connect error: Network is
  │ unreachable (os error 101)
  ├─▶ could not GET https://index.crates.io/config.json: error sending request
  │   for url (https://index.crates.io/config.json): error trying to connect:
  │   tcp connect error: Network is unreachable (os error 101)
  ├─▶ error sending request for url (https://index.crates.io/config.json):
  │   error trying to connect: tcp connect error: Network is unreachable (os
  │   error 101)
  ├─▶ error trying to connect: tcp connect error: Network is unreachable (os
  │   error 101)
  ├─▶ tcp connect error: Network is unreachable (os error 101)
  ╰─▶ Network is unreachable (os error 101)

[ilyagr]

Thanks @Sculas, you are right! Glad I shared the command I used.

However, I still have the problem after compiling from the right repo (commit af04e45 according to cargo install), logs below.

Observation: I noticed that it only have when my computer is on a wired connection (Ethernet over USB). On wireless, it works. I have a suspicion: my computer disables the WiFi conncetion when I plug it into ethernet (otherwise I get other problems). Perhaps binstall is trying to use the WiFi connection for some reason, even when it's disabled?

Let's see if these logs are any better. The final error is ~ the same as in my last comment. Again, I used cargo binstall cargo-nextest -y --log-level trace --json-output | tee out.txt.

cargo-binstall-logs.txt

[NobodyXu]

Normally I would now say "Why is trust-dns used anyway given the issues it's causing? Can't it be disabled by default?", but I'm assuming it was done for a reason, so we should fix it instead.

Well the reasons are that:

• reqwest defaults to getaddrinfo from libc, which is half broken and differs on different libc (glibc vs musl libc vs MacOS libc), https://jvns.ca/blog/2022/02/23/getaddrinfo-is-kind-of-weird/
• getaddrinfo is not async
• getaddrinfo does not support dns over quic/https/tls.
• musl's getaddrinfo didn't even support dns over tcp

[NobodyXu]

That's why we also want to upstream this change to reqwest or trust-dns-resolver: gix also uses reqwest::blocking.

Currently we just let it use getaddrinfo since we no longer enable reqwest/trust-dns feature, though we might want to upstream and re-enable that.