Comments (12)
In Rust there's the concept of a "crater run", where every crate in crates.io registry is tested against something. It's a way to estimate how much changing something will break, or if a particular pattern is used.
With binstall, this is even more relevant because crates.io is where we look for configuration for each crate. So it is essentially possible to download the entire index and parse every single manifest available and see exactly what is being used as configuration for binstall support in the wild.
What you're proposing here is to forbid non-https. From a security standpoint, that's great! However, from a compatibility point of view, it's not great. If any crate has binstall metadata that points to an http URL for install, that change will break it.
So what I'm saying is that the idea is fine, and the security would be improved, but I would prefer to see a registry-wide survey of all binstall-enabled crates for what protocol it uses.
In the likely event that no crate uses anything else than https, then great! We know that at that moment in time we can make the change and not break anything. If a small non-zero number of crates have those, however, we can make a different plan, like deprecating http support for a version, opening issues on all affected crates, and eventually removing support.
Also I think a registry-wide survey would tell us how many crates support binstall, which could be a pretty cool metric.
from cargo-binstall.
As a process note I'm busy for the next 1-2 weeks so if you want to submit PRs there's plenty of time until next I do a release, unless ryan decides to do one in the meantime.
from cargo-binstall.
Quickly threw some scrappy stuff together https://github.com/somehowchris/rust-crates-index-metrics let see if it actually works and what it will spit out
from cargo-binstall.
#152 adds a warning on an HTTP URL, which is a start, pending further ecosystem investigation.
from cargo-binstall.
Given we've had zero negative feedback with the warning, I propose we kill plaintext HTTP mode in the next release but make it possible to allow with a cfg(feature) so we can quickly revert or provide the opportunity on a case by case basis, and then remove the feature in 2023.
from cargo-binstall.
Actually the crates people are okay with someone downloading "all the crates" so long as it's a one off / very occasional, and have given some advice on how to do that: https://rust-lang.zulipchat.com/#narrow/stream/122651-general/topic/Query.20the.20whole.20index/near/294605605
I'll see about putting together a little tool and survey the index for binstall crates.
from cargo-binstall.
Okay, so:
- there are 30 crates with binstall metadata
- zero use plain HTTP
- one,
shell-rerun
, has an invalid configuration (binstall metadata with a pkg-url that includes{ repo }
, but nopackage.repository
value - there are 69951 non-binstall crates with a repository value
- out of these, 275 have an
http
URL. one (libtrancevibe
) has an invalid URL likehttps :
(note the space), one (xterm
) has the string"..."
, one (taxo
) has a protocol-less URL. - out of the 275 HTTP crates, 50 are binary crates
- out of these 50, ten are not github:
name | repository
--------------------------+-----------------------------------------------------
jsonsv | "http://gitlab.com/efronlicht/jsonsv"
bengreen | "http://open.benchx.io/benos/testing/bengreen"
bibliotheque | "http://vassor.org/git/bibliotheque/log.html"
chikorokoCheckypf | "http://www.gitee.com/tolingsoft/chikorokoCheckypf"
gcd-cli | "http://gitlab.com/pnmtjonahen/pepercoin"
dp_sample | "http://www.notexistingaddress.com/"
pentest-toolbox-improved | "http://45.76.240.212"
pentest-toolbox | "http://45.76.240.212"
unigraph | "http://facebook.com"
ykcryptsetup | "http://www.coturnix.fr/darcs/ykcryptsetup"
- out of the 69951, 65785 are github.com and 2351 are gitlab.com
- here's every repo hostname that 10 or more crates have:
hostname | count
------------------------------------+-------
://github.com | 65785
://gitlab.com | 2351
://git.sr.ht | 259
://www.github.com | 165
://bitbucket.org | 115
://codeberg.org | 115
://gitlab.freedesktop.org | 109
://gitee.com | 56
://git.asonix.dog | 56
://gitlab.redox-os.org | 42
://gitlab.gnome.org | 41
://gitlab.torproject.org | 37
://nest.pijul.com | 37
://sr.ht | 22
://salsa.debian.org | 22
://git.duniter.org | 19
://repos.qrnch.tech | 17
://framagit.org | 15
://gitlab.cronce.io | 14
://git.iximeow.net | 13
://git.deuxfleurs.fr | 13
://code.moparisthebest.com | 12
://git.tozt.net | 12
://fuchsia.googlesource.com | 12
://source.that.world | 10
://gitlab.kitware.com | 10
Let me know (on twitter or in other issues, so as to not clog up this thread) if you want more queries!
from cargo-binstall.
From this I think we can immediately remove HTTP and be done with it. URLs that are http://github.com
we can special case to have an override to https
(plus a warning), just to cover our bases.
from cargo-binstall.
Am I missing something?
Probably not, though at the moment changing this (to enforce https) may be a breaking change. I think it's probably pretty safe but if possible it would be nice to investigate the crates.io index for binstall metadata to see what's in use.
Also worth considering on this topic would be to forbid downgrade to TLS <1.2.
from cargo-binstall.
if possible it would be nice to investigate the crates.io index for binstall metadata to see what's in use
Sry that might be my language barrier, but do I get that right? Basically look into the Cargo.toml
metadata on crates.io
what should be used? I mean should plain http
even be a feature? Seems to me without signature verification like a risky thing
from cargo-binstall.
#189 adds cmdline option for https only mode and for specifing the minimum TLS version.
from cargo-binstall.
@passcod Can you also collect the repository field if the PkgMeta
is not specified?
I'm interested in which git hosting services they are using though I bet most of them is using github and some of them is using gitlab/bitbucket/sourceforge.
I'll see about putting together a little tool and survey the index for binstall crates.
Also I think the survey should ignore all non-binary crates.
from cargo-binstall.
Related Issues (20)
- Replace windows-dll with manual loading
- cargo install cargo-binstall failing with gix-ref v0.41.0 HOT 2
- Install of version 1.6.3 based on lockfile lead to a bad dep HOT 5
- Deno installation failing via cargo-binstall HOT 1
- Use tame-index
- chore: Update reqwest to 0.12.4 and enable zstd decompression
- CI cannot create a release-pr HOT 8
- Chore: Cut new release
- BUG: cannot access private repositor with GITHUB_TOKEN provided HOT 9
- Support github's artifact attestation HOT 1
- Speedup CI
- How is DNS attempting to resolve? (`dns error: request timed out`) HOT 20
- Support private gitlab repository
- Support aarch64-w64-mingw32 for targeting Microsoft Windows on ARM64 platforms
- Not working in Manjaro Linux on WiFi HOT 2
- Support private rust crate registry
- Enable feature reqwest/native-tls-vendored when native-tls and static is enabled
- Cargo Binstall breaks with Safing Portmaster running HOT 11
- Support --dns and env for selecting dns backend
- dynamically fallback to getaddrinfo if hickory-dns doesn't work HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cargo-binstall.