tarides / ocaml-platform-installer Goto Github PK
View Code? Open in Web Editor NEWThe best way for developers to write software in OCaml
License: ISC License
The best way for developers to write software in OCaml
License: ISC License
There are a couple of things that would be good to add to our release workflow to make it more hygienic. For example:
dune subst
(we have a %%VERSION%%
watermark e.g. in the --version
query)As pointed out in https://github.com/tarides/ocaml-platform/issues/19, I think it might make sense to remove mdx
from the OCaml Platform, but I might be missing context.
What were the reasons why mdx
ended up in the Platform in the first place? Does anyone know? Was that for running code blocks in markdown or for its cram test feature (which I think is slowly getting deprecated in favor of the dune cram tests, right?)?
Reported by @voodoos
ocaml-platform: [DEBUG] Running: 'opam' 'install' 'ocaml-system' '--yes' '-q' '--color=never' '--switch' '/var/folders/w6/k6g9gfrj59lf_3ymz3312tcc0000gn/T/ocaml-platform-sandbox-ce3848' '--root' '/Users/ulysse/.opam'
ocaml-platform: [INFO] Error in 'opam' 'install' 'ocaml-system' '--yes' '-q' '--color=never' '--switch' '/var/folders/w6/k6g9gfrj59lf_3ymz3312tcc0000gn/T/ocaml-platform-sandbox-ce3848' '--root' '/Users/ulysse/.opam':
[ERROR] The compilation of ocaml-system.4.14.1 failed at "ocaml gen_ocaml_config.ml".
#=== ERROR while compiling ocaml-system.4.14.1 ================================#
# context 2.1.3 | macos/x86_64 | | file:///Users/ulysse/.opam/plugins/ocaml-platform/platform_sandbox_compiler_packages
# path /private/var/folders/w6/k6g9gfrj59lf_3ymz3312tcc0000gn/T/ocaml-platform-sandbox-ce3848/_opam/.opam-switch/build/ocaml-system.4.14.1
# command ~/.opam/opam-init/hooks/sandbox.sh build ocaml gen_ocaml_config.ml
# exit-code 1
# env-file ~/.opam/log/ocaml-system-15561-5d2dad.env
# output-file ~/.opam/log/ocaml-system-15561-5d2dad.out
### output ###
# ERROR: The compiler found at /var/folders/w6/k6g9gfrj59lf_3ymz3312tcc0000gn/T/ocaml-platform-system-compiler-9951d0/ocamlc has version 4.14.1+dev0-2022-03-28,
# and this package requires 4.14.1.
# You should use e.g. 'opam switch create ocaml-system.4.14.1+dev0-2022-03-28' instead.
The former state can be restored with:
/usr/local/bin/opam switch import "/private/var/folders/w6/k6g9gfrj59lf_3ymz3312tcc0000gn/T/ocaml-platform-sandbox-ce3848/_opam/.opam-switch/backup/state-20220907140002.export"
ocaml-platform: [ERROR] Command 'opam install ocaml-system --yes -q --color=never --switch
/var/folders/w6/k6g9gfrj59lf_3ymz3312tcc0000gn/T/ocaml-platform-sandbox-ce3848
--root /Users/ulysse/.opam' failed: exited with 31
Whether or not to statically link is done depending on the target platform.
Instead, this should be externally communicated to the compiler, for instance via an environment variable.
By reading a potential .ocamlformat
file.
After installing ocaml-platform on macOS Monterey (on Arm64), opam has been installed in /usr/local/bin/opam
even if it was already in /opt/homebrew/bin/opam
A summary of tool/version information that ocaml-platform
manages would give at least two benefits to users of the installer:
ocaml-platform
is meant to be the installer for the OCaml Platform. We need to make sure that the set of tools in the OCaml Platform is perceived as coherent on two levels: first, it needs to be intrinsically coherent; and second, it needs to be coherent with the set of tools installed by ocaml-platform
.
With "intrinsically coherent" I mean that the choice of tools inside the OCaml Platform and the way they're placed into the four different categories of the OCaml Platform is coherent. For example, with the current choice, I myself would ask: why are bun
and mdx
in the OCaml Platform, while e.g. the js_of_ocaml
compiler and bisect_ppx
aren't?
With "coherent with the set of tools installed by ocaml-platform
", I simply mean that we need to choose which categories of the OCaml Platform we want to be covered by ocaml-platform
. Let's say that's only the active tools. Then, we need to make sure that we can explain very well any possible exception to the 1-1 correspondence between the active OCaml Platform tools and the tools installed by ocaml-platform
.
What's the reason why the current state of the OCaml Platform is the way it is?
In the video on the OCaml Platform site, avsm points out clear necessary rules for projects to form part of the OCaml Platform:
only tools should be in the Platform (no libraries), at least for now
characteristics a tool needs to fulfil to be a fit for the OCaml Platform are:
- it improves the OCaml development process
- it's actively maintained: there's a strategy for incremental changes and a guarantee of backwards compatibility over the upcoming years
- it's liberally licensed
- it assists with the reuse and publication of OCaml code (btw, how is that different from liberally licensed?)
- it has a minimum number of users (no concrete number though)
- it has maintainers in the community
- it's open to any contributor
tools inside the Platform shouldn't be duplicated; i.e. there shouldn't be two tools inside the active
category of the Platform serving the same purpose
I think that's a good basis for how to shape the Platform in the future. The question in this issue is where the "initial"/current state of the Platform comes from though.(*)
Back to the question above: Is there a rule/description/definition/explanation that corresponds to the current state of the OCaml Platform? I think we'll need to come up with something very simple and then adapt the OCaml Platform to it (not the other way around).
What about keeping the initial state as minimal as possible? Concretely, what about the following?
The initial state of the Platform contains all tools that fulfil the rules above and that serve a purpose that needs to be served to keep OCaml a real-world language.
An ok practical rule of thumb to find out if a tool "serves a purpose that needs to be served to keep OCaml a real-world language", could be: is the tool important enough to be considered by the ocaml-release-readiness(**)? Of course, that rule of thumb doesn't directly capture tools like opam
or dune-release
/opam-publish
since they're automatically compiler-forward-compatible, but I do think it indirectly captures those tools: if those tools weren't automatically compiler-forward-compatible, they'd definitely be included into the ocaml-release-readiness efforts.
If we followed that rule, I think we'd have to remove exactly mdx
and bun
from the Platform and wouldn't need to add anything to the Platform.
If we follow the description above (only having really important tools in the Platform), I think for the initial state, it would make sense not to have any tools in the category of sustained
tools at all at the beginning. So I'd say, we could simply promote all tools from incubate
to active
.
ocaml-platform
I think we can leave this part for later tbh. We should have an intrinsically coherent OCaml Platform and then ocaml-platform
should install all active
(or whatever categories we agree on) tools in the Platform, unless there's a good reason for a tool not to be installed by ocaml-platform
.
I've just stated a couple of things that are probably controversial. So some questions:
The initial state of the Platform contains all tools that fulfil the rules above and that serve a purpose that needs to be served to keep OCaml a real-world language.
mdx
and bun
from the Platform and wouldn't add anything to it?incubate
category for the initial state?The proposed changes in our current docs section are (I'm just copying and pasting now):
I first tried to come up with a justification for those proposed changes, but then figured it would be better to do it the other way around: first define a good rule/description/definition/explanation for the initial state of the Platform and then propose changes to match that definition. That has led to my proposed changes above: remove exactly mdx
and bun
from the Platform. By the way, my thoughts about dune-release
<-> opam-publish
aren't covered in this issue since that's another topic; I'll open another issue for that. So the differences between those proposed changes and my proposed changes are:
ppxlib
from the Platform (in which case I'd promote it to Active), but I'd discuss that in a separate issuemdx
to Active, I'd remove it from the platform. Again: I'd discuss that in a separate issuedune-release
to Active and move opam-publish
to Sustain; instead, I'd discuss their co-existence (in a separate issue)(*) By "initial state", I mean the following: The idea is to make the OCaml Platform a community project, where any community member can suggest to add a new project to the platform. Once we're there (🤞), the set of rules above will serve as a good basis for discussions on whether a project should be added to the Platform or change category in there.
(**) With the "ocaml-release-readiness" I'm referring to all the work that's being done to orchestrate new compiler releases, making sure that all important tools, that won't automatically support the new compiler, will do so on time for the release.
As described in https://github.com/tarides/ocaml-platform/issues/13, our first approach to populating our cache is via a sandboxing mechanism. While that sandboxing approach has many advantages (also described on that issue), it also has at least one disadvantage: it's very slow. So, once our cache mechanism is fully implemented (i.e. at least issues https://github.com/tarides/ocaml-platform/issues/13 and https://github.com/tarides/ocaml-platform/issues/11), we'll need to implement an optimization for that.
So far we've been discussing to use pre-built binaries to optimize our caching mechanism. There are different ways how to realize a pre-built binaries optimization. Among others, the following three:
We embed the pre-built binaries into our tool and the first time ocaml-platform
is run, it pre-populates our cache with those binaries.
ocaml-platform
for a certain architecture and operating system, we embed the correspondent tool binaries only)ocaml-platform
binaryA similar approach to 1. (we pre-populate our cache first time ocaml-platform
is run), but instead of embedding the pre-built binaries into ocaml-platform
, we keep them somewhere online on some repo or similar.
ocaml-platform
binaryIn addition to our local cache, we add another cache in the following way: we have an online cache somewhere with the pre-built binaries and ocaml-platform
first has a look at the online cache and installs all tools found there; then it has a look at the local cache and does the rest (one possible way to implement that approach would be by writing an online opam repo with the meta-information for all pre-built tools and adding that opam repo to the users opam state. that would be similar to option 1. in https://github.com/tarides/ocaml-platform/issues/11 but remote).
ocaml-platform
binaryHave I missed any upsides or downsides? And/or are there more approaches anyone would like to discuss?
(btw, if anyone feels down for making a table out of this bad bullet point overview, don't hesitate! :))
Currently, the CI is run by github actions.
Port CI to OCurrent and use OCluster to run it.
It needs to support at least the current features of the CI: testing and releasing for linux/amd64, linux/arm64 and macos/amd64 (maybe more in the future).
We must temporarily set the global variable sys-ocaml-version
in order to install the ocaml-system
package in the sandbox switch. This variable can't be "local" it seems.
@dra27 suggested that the local value don't shadow the global value might be a bug in Opam and that we can re-define ocaml-system
to not depend on this variable.
Related issue on Opam: ocaml/opam#4923
Issue https://github.com/tarides/ocaml-platform/issues/11 talks about using a cache to install the platform tools, but for that of course we need a cache. There are different approaches how to populate the cache. One approach that has been discussed a lot is via a sandboxing workflow. Concretely that could mean something like: If a tool isn't in the cache when isntalling it, then:
Of course, that's just an example of how it could be implemented concretely and probably I'm missing details :)
The advantages of a sandbox workflow are the following:
The disadvantage of a sandbox workflow is that installing the platform will be very slow (if we don't optimize it e.g. by some pre-built binaries).
I tried installing using the bash script but tweaked installer.sh to change the PREFIX to look under opt/homebrew/bin/opam
instead because that's where my opam binary lives. I get the following error
❯ sudo bash < <(curl -sL https://github.com/tarides/ocaml-platform-installer/releases/latest/download/installer.sh | sed 's;/usr/local;/opt/homebrew;g')
Password:
=> Download ocaml-platform-0.1.0-macos-x86_64.tar
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 2820k 100 2820k 0 0 2254k 0 0:00:01 0:00:01 --:--:-- 11.8M
=> Install into /opt/homebrew/bin
Installation is done. Please run 'ocaml-platform' to install the Platform tools inside your set switch.
❯ ocaml-platform
Inferring tools version...
Creating a sandbox to build the tools...
Building odoc...
Building merlin...
Building dune-release...
ocaml-platform: [ERROR] Command 'opam install dune-release --yes -q --color=never --switch
/var/folders/d0/c300r25n785405hhz1typhkr0000gn/T/ocaml-platform-sandbox-3ccb20
--root /Users/koonwen/.opam' failed: exited with 31
ocaml-platform: [ERROR] Command 'opam install merlin --yes -q --color=never --switch
/var/folders/d0/c300r25n785405hhz1typhkr0000gn/T/ocaml-platform-sandbox-3ccb20
--root /Users/koonwen/.opam' failed: exited with 31
ocaml-platform: [ERROR] Command 'opam install odoc --yes -q --color=never --switch
/var/folders/d0/c300r25n785405hhz1typhkr0000gn/T/ocaml-platform-sandbox-3ccb20
--root /Users/koonwen/.opam' failed: exited with 31
There is no LICENSE file.
What kind of license do we want? GPLv3?
The sandbox can take the form of a switch in a separate opam root, and the tools can be installed with opam install
.
This repository currently contains the code for the installer of the OCaml platform tools. Would be nice to reflect this in the repo name and in the README. The "Platform" name is overused everywhere, so let's try to not add confusion :-)
Currently, we don't do any meaningful logging at all yet. It would be good to implement the UserInteractions
module and to use it for logging, prompting and error reporting. For that, a first important step would be
That will be a good base to discuss the questions:
With #31 it is <opam_root>/plugins/ocaml-platform/platform-cache
fot the repo, <opam_root>/plugins/ocaml-platform/archives
for the sources, <name of tool>+cached
for the name of the binary packages.
This shoud be discussed as we will need to keep the names after the first release to ensure compatibility.
There are a couple of places in which we currently use the opam binary , for example, to init opam and to install the non-opam platform tools. We should use the opam library instead
In fact, it would be great not to use the opam binary at all. As a later step, we can then totally get rid of installing the opam binary and wrap the opam CLI into our tool instead.
Various parts of ocaml-platform
rely on temporarily modifying the opam state.
We should make sure that these temporary things are cleaned as much as possible.
As it might not be possible to clean them in all situation (eg a sudden shutdown of the computer), we should also clean them at the next run of the binary, and not rely on getting a clean state.
The temporary things are:
The program interuption that can happen are
Sys.catch_break
is true
by default)opam
root leaves a better state for the user.Related to https://github.com/tarides/ocaml-platform/issues/50
Opam 2.1 deprecates opam config
and its sub-commands related to variable in favor of opam var
.
Potential solutions are:
Generated opam files contains
install: [
[ "cp""-pPR"".""%{prefix}%" ]
]
In opam 2.1, there is an optimisation of install directory tracking (to record changes for removal), ocaml/opam#4494: if there is an install file and no install command, no tracking is done: recorded changes are the one in the install file.
Installers written in sh
have a number of advantages, but one notable disadvantage is that there is no detection whether the script is complete. If anything happens during transfer of the installer the script might get truncated, but it still continues to be a valid shell script.
rm /home/user/.opam
could become
rm /home/user
as described in this blog post by sandstorm. It also mentions a fix they use in their own install.sh
:
however, this can be mitigated by wrapping the whole script in a bash function executed on the last line, as we do
Checking their [install.sh](https://install.sandstorm.io/install.sh)
this is achieved by wrapping their script into
_ {
# shell code here
}
_ "$0" "$@"
They mention a blog post in their script which stopped to exist but an archive copy can be found here.
If deemed a sensible addition I'll gladly contribute a PR to equip the installer with a guard.
On macOS the experience is not super smooth yet it seems :p
First try: I was confused because nothing was happened - but actually the output is mangled (notice where Password
is displayed):
❯ sudo bash < <(curl -L https://github.com/tarides/ocaml-platform-installer/releases/latest/download/installer.sh)
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Password:
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 4488 100 4488 0 0 2347 0 0:00:01 0:00:01 --:--:-- 5212
Second try: it goes a bit further but then fail because of a missing sha512sum
:
❯ sudo bash < <(curl -L https://github.com/tarides/ocaml-platform-installer/releases/latest/download/installer.sh)
[...]
=> Download ocaml-platform-0.1.0-macos-x86_64.tar
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 2820k 100 2820k 0 0 1236k 0 0:00:02 0:00:02 --:--:-- 1674k
bash: line 87: sha512sum: command not found
Issue https://github.com/tarides/ocaml-platform/issues/5 talks about that there's some information needed about each tool when installing it, such as the compiler version the tool needs to be compiled with (for most tools) and the tools version (for ocamlformat). So the list of platform tools also needs to embed that information: e.g. it could be a function taking something like OpamStateTypes.switch_state
from the opam library and the ocamlformat version to return all the info needed.
Currently, we only have a placeholder for the Opam.is_initialized function. I propose to actually implement it :P
The reason we need that function is because we want to include folks who dont have OCaml up and running yet into our target. For those ocaml-platform will do the following:
Since we want to install the platform tools in the currently selected switch and most platform tools are compiler dependent, that only works if the compiler is already installed in the switch (or: if we allowed to pass in a CLI argument for the compiler version, but we want to keep the CLI as simple as possible...). So we should add a step to the tool that checks if the compiler is already installed after checking if opam is inited and before installing the platform tools in the currently selected swtich.
As pointed out in https://github.com/tarides/ocaml-platform/issues/19 (particularly in Difference to the current plan), there are different opinions on the question if ppxlib
should be in the Platform or not. Maybe we can discuss that here on the issue? I've explained in Defining the current/initial state of https://github.com/tarides/ocaml-platform/issues/19 why, without knowing the counterarguments for ppxlib
being in the OCaml, I think it should be in the Platform.
What are the counterarguments for ppxlib
being in the Platform? Is the counterargument that ppxlib
-among others- is a library? If so: I see ppxlib
as a tool set that provides everything needed for meta-programming in OCaml. Of course, that also includes a library to write ppx rewriters, but it also includes the ppxlib
driver, which is a tool. Do you agree?
Or is it because ppxlib
can't be generally installed by ocaml-platform
? In that case, we should discuss if we want ocaml-platform
to be the base for deciding which tools are in the OCaml Platform or if we want to define and shape the OCaml Platform and write ocaml-platform
to consolidate the OCaml Platform whereever it makes sense.
@tmattio, what are your reasons to remove ppxlib
from the Platform? Do you agree with https://github.com/tarides/ocaml-platform/issues/19 in general?
Currently the non-opam platform tools are simply installed running opam install <tool>
. To optimize that, we should first check if they already are in a cache and, if so, install them from there. There are different approaches how to get the binaries from the cache to the switch, for example:
PD: We don't have the cache yet, but we can already work on this issue by assuming we have some cache directory and use it as if it was a populated cache.
Add a benchmark to see how long each operation takes.
The first step will be to verify that the binary for amd64 runs well on macos/arm64 using rosetta2.
The second step is to build a native binary for macos/arm64.
opam
has a --cli
option to have a stable api that we should use (there are already differences between 2.0 and 2.1 and we have warnings about this currently).
The installer currently require that a switch exist:
ocaml-platform: [DEBUG] Running: 'opam' 'config' 'var' 'ocaml:compiler' '--yes' '-q' '--color=never' '--root' '/home/.opam'
ocaml-platform: [INFO] Error in 'opam' 'config' 'var' 'ocaml:compiler' '--yes' '-q' '--color=never' '--root' '/home/.opam':
[WARNING] var was deprecated in version 2.1 of the opam CLI. Use opam var instead or set OPAMCLI environment variable to 2.0.vv
[ERROR] No switch is currently set. Please use 'opam switch' to set or install a switch
ocaml-platform: [ERROR] Command 'opam config var ocaml:compiler --yes -q --color=never --root
/home/.opam' failed: exited with 50
It is failing when querying config from that switch but it's certain that subsequent operations have the same implicit requirement.
This situation is caused by a failing opam init
, potentially indirectly called by ocaml-platform
, due to an error while creating the default switch.
I suggest detecting this case and exiting earlier with a friendlier error message.
I think silently and slowly creating a default switch is not a good idea if we are encouraging the use of local switches.
installer.sh uses some parts of opam installer script.
To avoid code/hashes duplication and double maintenance, maybe worth calling opam installer with some options: --download-only
that only downloads and validate sha, and --version
.
Currently the (unmerged) CI does not check if the code is ocamlformat-idempotent.
Currently we don't have any tests (at least no "reasonable" test). @panglesd has already implemented a very nice CI to test if the whole installation works, which we should merge asap.
Apart form that, it might also be nice to have some finer-grained end-to-end testing, such as a cram test workflow. In our case, we can't simply use dune cram tests since we want to test the installer in an environment in which dune doesn't necessarily exist. Still, it would be nice to have a cram test workflow, for example to test logging and error reporting. What do you think?
Hello, thanks for this great tool!
I recently took it for a spin with a 5.0.0~alpha1 switch and it seems things go a little askew when looking for a system compiler. The context of this was building in a dockerfile.
FROM ocaml/opam:debian-11-ocaml-5.0@sha256:8ef27313193b5e003fce1c79624f1b5796508c672abdc6c2d24b4698ad92f398
RUN curl -sL https://github.com/tarides/ocaml-platform-installer/releases/latest/download/installer.sh > ./installer.sh
RUN chmod +x installer.sh
RUN sudo bash ./installer.sh
And then running ocaml-platform -v -v
which resulted in the following.
opam@de7fead58b3f:~/workshop$ ocaml-platform -v -v
ocaml-platform: [DEBUG] Running: 'opam' 'show' 'ocaml' '-f' 'installed-version' '--normalise' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
* Inferring tools version...
ocaml-platform: [DEBUG] Running: 'opam' 'show' 'dune' 'dune-release' 'merlin' 'ocaml-lsp-server' 'odoc' 'ocamlformat' '-f' 'name,installed-version' '--normalise' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'available-versions' 'dune' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'dune.3.4.1' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
-> dune will be installed as dune.3.4.1-ocaml5.0.0
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'available-versions' 'dune-release' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'dune-release.1.6.2' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
-> dune-release will be installed as dune-release+bin+platform.1.6.2-ocaml5.0.0
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'available-versions' 'merlin' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'merlin.4.6.1~5.0preview' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
-> merlin will be installed as merlin+bin+platform.4.6.1~5.0preview-ocaml5.0.0
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'available-versions' 'ocaml-lsp-server' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocaml-lsp-server.1.13.2~5.0preview' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
-> ocaml-lsp-server will be installed as ocaml-lsp-server+bin+platform.1.13.2~5.0preview-ocaml5.0.0
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'available-versions' 'odoc' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'odoc.2.1.1' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
-> odoc will be installed as odoc+bin+platform.2.1.1-ocaml5.0.0
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'available-versions' 'ocamlformat' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocamlformat.0.24.1' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
-> ocamlformat will be installed as ocamlformat+bin+platform.0.24.1-ocaml5.0.0
* Building the tools...
-> Creating a sandbox...
ocaml-platform: [DEBUG] Running: 'opam' 'config' 'var' 'prefix' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
ocaml-platform: [DEBUG] Error in execution: [WARNING] var was deprecated in version 2.1 of the opam CLI. Use opam var instead or set OPAMCLI environment variable to 2.0.
ocaml-platform: [INFO] Creating sandbox switch for building the tools
ocaml-platform: [DEBUG] Running: 'opam' 'switch' 'create' '--no-switch' '/tmp/ocaml-platform-sandbox-cba4ba' '--empty' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'config' 'var' 'sys-ocaml-version' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
ocaml-platform: [DEBUG] Error in execution: [ERROR] Variable sys-ocaml-version not found in global config
[WARNING] var was deprecated in version 2.1 of the opam CLI. Use opam var instead or set OPAMCLI environment variable to 2.0.
ocaml-platform: [DEBUG] Running: 'opam' 'config' 'set-global' 'sys-ocaml-version' '5.0.0' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
ocaml-platform: [DEBUG] Error in execution: [WARNING] set-global was deprecated in version 2.1 of the opam CLI. Use opam var instead or set OPAMCLI environment variable to 2.0.
ocaml-platform: [DEBUG] Running: 'opam' 'install' 'ocaml-system' '--yes' '-q' '--color=never' '--switch' '/tmp/ocaml-platform-sandbox-cba4ba' '--root' '/home/opam/.opam'
ocaml-platform: [DEBUG] Error in execution: [ERROR] ocaml-system unmet availability conditions, e.g. 'sys-ocaml-version = "4.14.0"'
ocaml-platform: [DEBUG] Running: 'opam' 'config' 'unset-global' 'sys-ocaml-version' '--yes' '-q' '--color=never' '--root' '/home/opam/.opam'
ocaml-platform: [DEBUG] Error in execution: [WARNING] unset-global was deprecated in version 2.1 of the opam CLI. Use opam var instead or set OPAMCLI environment variable to 2.0.
ocaml-platform: [ERROR] Command 'opam install ocaml-system --yes -q --color=never --switch
/tmp/ocaml-platform-sandbox-cba4ba --root /home/opam/.opam' failed: exited with 5
Looks like (although I'm not familiar with the internals) it tries to install a 5.0.0 system compiler and fails as there is none at the moment. A pretty niche bug, but perhaps it would be good to catch and report this error. At the moment it looks like it fails silently and then goes on to try and do something with the switch that was never made and only then realises something has gone wrong.
The binary store takes the name of a package, its version, and eventually metadata such as the ocaml version used to compile the binary, and returns a list of cached installation files.
An interface for the binary store could look like
val cache : ~package:string -> ~version:string -> ~files:string list -> unit
val get : ~package:string -> ~version:string -> string list option
suggested by @samoht
At least one workflow (cygwin/WSL/native/..., I'll know more about that after investigating).
This includes:
Currently (but it is not merged), linux/amd64 and macos/amd64 tests are run whenever a commit is pushed, while linux/arm64 is run only is case of a release.
We would like to run CIs only when they are necessary to avoid costly runs.
A few possibilities:
Reported by @samoht
❯ ocaml-platform -vv
ocaml-platform: [DEBUG] Running: 'opam' 'config' 'var' 'ocaml:compiler' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [INFO] Error in 'opam' 'config' 'var' 'ocaml:compiler' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam':
[WARNING] var was deprecated in version 2.1 of the opam CLI. Use opam var instead or set OPAMCLI environment variable to 2.0.
* Inferring tools version...
ocaml-platform: [DEBUG] Running: 'opam' 'show' 'dune' 'dune-release' 'merlin' 'ocaml-lsp-server' 'odoc' 'ocamlformat' '-f' 'name,installed-version' '--normalise' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
-> dune is already installed
-> dune-release is already installed
-> merlin is already installed
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'available-versions' 'ocaml-lsp-server' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocaml-lsp-server.1.13.2~5.0preview' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocaml-lsp-server.1.13.1' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocaml-lsp-server.1.13.0' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocaml-lsp-server.1.12.4' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocaml-lsp-server.1.12.3' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocaml-lsp-server.1.12.2' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocaml-lsp-server.1.12.1' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocaml-lsp-server.1.12.0' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocaml-lsp-server.1.11.6' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocaml-lsp-server.1.11.5' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocaml-lsp-server.1.11.4' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocaml-lsp-server.1.11.3' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocaml-lsp-server.1.11.2' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocaml-lsp-server.1.11.1' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocaml-lsp-server.1.11.0' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocaml-lsp-server.1.10.6' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'show' '-f' 'depends:' 'ocaml-lsp-server.1.10.5' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
-> ocaml-lsp-server will be installed as ocaml-lsp-server+bin+platform.1.10.5-ocaml4.13.1
-> odoc is already installed
-> ocamlformat is already installed
* Building the tools...
-> Creating a sandbox...
ocaml-platform: [DEBUG] Running: 'opam' 'config' 'var' 'prefix' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [INFO] Error in 'opam' 'config' 'var' 'prefix' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam':
[WARNING] var was deprecated in version 2.1 of the opam CLI. Use opam var instead or set OPAMCLI environment variable to 2.0.
ocaml-platform: [INFO] Creating sandbox switch for building the tools
ocaml-platform: [DEBUG] Running: 'opam' 'switch' 'create' '--no-switch' '/var/folders/88/jkblgplx33db3c6606wyy7rc0000gn/T/ocaml-platform-sandbox-614ef7' '--empty' '--yes' '-q' '--color=never' '--root' '/Users/thomas/.opam'
ocaml-platform: [DEBUG] Running: 'opam' 'repository' 'add' '--this-switch' '-k' 'local' 'platform_sandbox_compiler_packages' '/Users/thomas/.opam/plugins/ocaml-platform/platform_sandbox_compiler_packages' '--yes' '-q' '--color=never' '--switch' '/var/folders/88/jkblgplx33db3c6606wyy7rc0000gn/T/ocaml-platform-sandbox-614ef7' '--root' '/Users/thomas/.opam'
ocaml-platform: [INFO] Error in 'opam' 'repository' 'add' '--this-switch' '-k' 'local' 'platform_sandbox_compiler_packages' '/Users/thomas/.opam/plugins/ocaml-platform/platform_sandbox_compiler_packages' '--yes' '-q' '--color=never' '--switch' '/var/folders/88/jkblgplx33db3c6606wyy7rc0000gn/T/ocaml-platform-sandbox-614ef7' '--root' '/Users/thomas/.opam':
[NOTE] Repository platform_sandbox_compiler_packages has been added to the selections of switch /private/var/folders/88/jkblgplx33db3c6606wyy7rc0000gn/T/ocaml-platform-sandbox-614ef7 only.
Run `opam repository add platform_sandbox_compiler_packages --all-switches|--set-default' to use it in all existing switches, or in newly created switches, respectively.
ocaml-platform: [DEBUG] Running: 'opam' 'install' 'ocaml-system' '--yes' '-q' '--color=never' '--switch' '/var/folders/88/jkblgplx33db3c6606wyy7rc0000gn/T/ocaml-platform-sandbox-614ef7' '--root' '/Users/thomas/.opam'
ocaml-platform: [INFO] Error in 'opam' 'install' 'ocaml-system' '--yes' '-q' '--color=never' '--switch' '/var/folders/88/jkblgplx33db3c6606wyy7rc0000gn/T/ocaml-platform-sandbox-614ef7' '--root' '/Users/thomas/.opam':
[ERROR] The compilation of ocaml-system.5.0.0~alpha1 failed at "ocaml gen_ocaml_config.ml".
#=== ERROR while compiling ocaml-system.5.0.0~alpha1 ==========================#
# context 2.1.3 | macos/arm64 | | file:///Users/thomas/.opam/plugins/ocaml-platform/platform_sandbox_compiler_packages
# path /private/var/folders/88/jkblgplx33db3c6606wyy7rc0000gn/T/ocaml-platform-sandbox-614ef7/_opam/.opam-switch/build/ocaml-system.5.0.0~alpha1
# command ~/.opam/opam-init/hooks/sandbox.sh build ocaml gen_ocaml_config.ml
# exit-code 1
# env-file ~/.opam/log/ocaml-system-1131-691ec0.env
# output-file ~/.opam/log/ocaml-system-1131-691ec0.out
### output ###
# ERROR: The compiler found at /var/folders/88/jkblgplx33db3c6606wyy7rc0000gn/T/ocaml-platform-system-compiler-10a6bb/ocamlc has version 4.13.1,
# and this package requires 5.0.0~alpha1.
# You should use e.g. 'opam switch create ocaml-system.4.13.1' instead.
The former state can be restored with:
/opt/homebrew/bin/opam switch import "/private/var/folders/88/jkblgplx33db3c6606wyy7rc0000gn/T/ocaml-platform-sandbox-614ef7/_opam/.opam-switch/backup/state-20220908064624.export"
ocaml-platform: [ERROR] Command 'opam install ocaml-system --yes -q --color=never --switch
/var/folders/88/jkblgplx33db3c6606wyy7rc0000gn/T/ocaml-platform-sandbox-614ef7
--root /Users/thomas/.opam' failed: exited with 31
Note that the switch contained other opam repositories:
<><> Repository configuration for switch default ><><><><><><><><><><><><><><><>
1 alpha git+https://github.com/kit-ty-kate/opam-alpha-repository.git
2 dune-universe git+https://github.com/dune-universe/opam-overlays.git
3 default https://opam.ocaml.org/
But that the error was reproduced on a fresh switch.
It might imply a smaller footprint for opam
users. See #39.
I think we should decide if we want that before the first release to ensure retro-compatibility more easily!
Once we only use the opam library and not the opam binary anymore (see issue https://github.com/tarides/ocaml-platform/issues/4), we could entirely get rid of installing the opam binary and instead wrap the opam CLI into ocaml-platform
.
Opam is of quite a different nature than all the other tools, so I don't think only wrapping this one CLI would be incoherent. Also, for opam it seems like wrapping the CLI would probably be quite a bit simpler than for most of the other tools. In fact, Patrick and Thibaut don't seem to have run into blockers concerning opam when trying out the "fat binary approach" of using the platform tool's libraries only.
As mentioned very briefly at the end of https://github.com/tarides/ocaml-platform/issues/19, we need to discuss what to do about the situation that both dune-release
and opam-publish
are in the Platform. The current suggestion in our state of the Platform docs is simply to demote opam-publish
to the Sustained category and to promote dune-release
to Active. Of course, that would be the simplest solution. Do you think the community in general would be behind that decision?
As far as I can tell, the differences between the two tools are:
dune-release
is more opinionated that opam-publish
opam-publish
is build system agnostic, whereas the usual/recommended dune-release
workflow relies on dune
being used (it's possible to use dune-release
to do the same as opam publish
does, but that's quite a bit more tedious and it's not the "dune-release philosophy" to do so)dune-release
by default requires a changelos fulfilling a certain standarddune-release
by default requires using githubdune-release
runs dune subst
which can be considered invasive/unreliable by somedune-release
has a wider scope than opam-publish
opam-publish
only has a minimal linting mechanism linting the generated opam filedune subst
)x-commit-hash
field to the opam file, which is good for vendoringopam-publish
can be more convenient wrt certain workflows and problems
opam-publish
workflow it's way more straight-forward to force push a tag and then publish the modified release on opam. (in the future, hopefully that problem will be solved in dune-release
by its WIP draft release feature)opam-publish
currently needs less configuration (is that correct?)I don't have an opinion about what to do with these two tools myself yet. So far, I've just been working out the differences between the two tools. I'm discussing with the maintainers what they think and if there might be implementation parts/ideas of opam-publish
that could serve dune-release
as well and in that sense it would make more sense to "merge" the two tools instead of moving one of the tools on the path to get deprecated and keeping only one without collaboration. If anyone here has an opinion or more context on this, please let me know!
Currently --help
displays almost only information about Logs
cli.
It should display the information what ti is for, which binary it installs and some information about each installed binary.
It should also list the error codes it can fail with.
Of course this isn't a short-term or even medium-term issue, but I think that for some people, particularly for @tmattio, this is still the longer term final goal, so I think it's good to have an issue for it.
Of course, if this is feasible, it would mean an incredible improvement in user experience, so it would be worth overcoming some issues for it, both technical and non-technical. But still, I think there are quite a few quite significant issues with it, so I also think it's important to discuss them before getting back to the idea.
Of course, to start with there are all the technical issues we had last time.
But also, unless I'm missing something (in that case, please let me know!), every tool using the compiler's Marchal
module would only work on the same compiler ocaml-platform
was compiled with. That would at least include merlin
, ocaml-lsp-server
, odoc
and ocamlformat
I think. Is that right or am I missing something? If that's right: How would we solve that? Would we only expose a CLI for the tools that don't use the Marchal
module (I think for me that would be an ok-ish compromise)? Or would we be ok with ocaml-platform
only supporting one compiler? I think that would mean a hard pay-off for increased user experience, particularly taking into account two things: one, we're in the middle of a major compiler release, so there will probably keep on existing projects only working on the 4.x compiler for some time; and two, we'd need to really make sure to have a very good update story for ocaml-platform
itself.
And then there are a few more general fat binary downsides we at least need to consider such as having a fat binary will probably slow down starting the servers of some tools.
Edit: Another technical issue that we've now come across trying to embed the opam
CLI is: if the user has installed another version of opam
than the version whose library we're embedding, opam
will notice that and fail.
Who would maintain such a huge CLI? Would we invite a maintainer of every platform tool to co-maintain ocaml-platform
(and possibly to duplicate their work by maintaing both their CLI and our CLI)? Or would we ask the platform tool maintainers to port their CLI to Cmdliner
(if that isn't done already) and expose it? I think the latter might not be possible for all tools (but with that we're getting into technical issues again...).
Including an utop
binary does not allow to do dune utop
which makes it loses some of its advantages.
What should we do about it? There are several possibilities:
ocaml-platform
opam install utop
to be able to run dune utop
opam install utop
(but this will install other dependencies as well, and seems not that good...).Anyway, everything should changes when (if) we have a relocatable compiler. We can rethink the solution adopted at this point.
I might tend toward a pure removal of utop
in the packaged installed by ocaml-platform
, but I am not sure!
The Platform.t
type represents a non-opam platform tool and should contain the information needed to know how to install that tool. It's currently defined in a very "first version" way. To improve it, we should do at least the following two things:
OpamPackage.Name.t
from the opam library.I'm trying to easily install the platform in a debian Github Codespace (https://github.com/features/codespaces).
The platform installer is not working as expected. When running it, it fails with the following error message:
@TheLortex ➜ /workspaces/ocaml-codespace (main) $ sudo bash < <(curl -L https://github.com/tarides/ocaml-platform-installer/releases/latest/download/installer.sh)
[...]
=> Install into /usr/local/bin
Installation is done. Please run 'ocaml-platform' to install the Platform tools inside your set switch.
@TheLortex ➜ /workspaces/ocaml-codespace (main) $ ocaml-platform
ocaml-platform: [ERROR] Command 'opam show ocaml -f installed-version --normalise --yes -q
--color=never --root /home/codespace/.opam' failed: exited with 50
When running the opam command manually I get [ERROR] Opam has not been initialised, please run 'opam init'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.