equinor/komodo

Komodo is a software distribution system used internally in Equinor

Komodo

Komodo is a software distribution system.

The purpose of Komodo is to automatically, reproducibly, and testably create a software distribution. Automatic deploy of new releases, as well as nightly deploy and the option of automatically moving the testing stage, is supported.

Install

The tool is not hosted on PyPI but can be installed with pip directly from GitHub:

pip install git+https://github.com/equinor/komodo.git

Documentation

The documentation is online. Developers can build it by installing with dev-dependencies by running pip install ".[dev]" and running make html in the docs directory.

Basic usage

We have a 'repository' of packages described in a YAML file. Each package contains a list of one or more versions. Each version contains:

• Build information.
• Maintainer.
• Source, e.g. PyPI or a GitHub repository, if required.
• Dependency list, if any.
• Other metadata, depending on the type of package.

For example, we may have a repository.yml like this:

python:
  3-builtin:
    make: sh
    makefile: build__python-virtualenv.sh
    maintainer: [email protected]
    makeopts: --virtualenv-interpreter python3

treelib:
  1.6.1:
    source: pypi
    make: pip
    maintainer: [email protected]
    depends:
      - python

Note that build__python-virtualenv.sh is a script that comes with komodo (in komodo/data); it will use the system Python in the environment it builds.

Now a 'release', e.g. stable, is defined in another YAML file, e.g. stable.yml, containing some or all of the packages in the repository file:

python: 3-builtin
treelib: 1.6.1

A full software distribution can then be built and deployed to a specified path, e.g. ./builds/stable-0.0.1, with the following command:

kmd stable.yml repository.yml --prefix builds --release stable-0.0.1

To use this environment, type source builds/stable-0.0.1/enable.

Other komodo commands

As well as the kmd command, this package installs several other commands, each with its own options:

• komodo-check-symlinks — Verify symlinks for komodo versions are according to a given config
• komodo-check-pypi — Checks if pypi packages are up to date
• komodo-clean-repository — Clean up unused versions in the repository file based on a set of releases
• komodo-create-symlinks — Create symlinks for komodo versions
• komodo-extract-dep-graph — Extracts dependencies from a given set of packages
• komodo-insert-proposals — Copy proposals into release and create PR
• komodo-lint — Lint komodo setup
• komodo-lint-maturity — Lint the maturity of packages
• komodo-lint-package-status — Lint the package status file
• komodo-lint-upgrade-proposals — Lint the upgrade proposals file against the repository file
• komodo-non-deployed — Outputs the name of undeployed matrices given an installation root and a release folder
• komodo-post-messages — Post messages to a release
• komodo-reverse-deps — Extracts dependencies from a given set of packages
• komodo-show-version — Return the version of a specified package in the active release
• komodo-snyk-test — Test a release for security and license issues
• komodo-suggest-symlinks — Returns a pull request if the symlink configuration could be updated
• komodo-transpiler — Build release files

Run tests

In a virtual environment:

git clone https://github.com/equinor/komodo.git
cd komodo
pip install ".[dev]"
pytest tests

komodo's Issues

Release maturity should not supersede tag maturity

I believe that the maturity of a release should not supersede the maturity of the tags in the release. This is also reflected in our release documentation (https://github.com/equinor/scout/pull/173). This can be, and hence should be, automated.

Examples

• 2020.05.a0 can contain any kind of tag, including 0.6a0, 1.0b0, 1.6rc1, 2.1.0 and 0.0.1.
• 2020.05.b0 can contain the tags 1.0b0, 1.6rc1, 2.1.0 and 0.0.1, but not 0.6a0.
• 2020.05.rc0 can contain the tags 1.6rc1, 2.1.0 and 0.0.1, but not 0.6a0 nor 1.0b0.
• 2020.05.00 can contain the tags 2.1.0 and 0.0.1, but not 0.6a0, 1.0b0 nor 1.6rc1.

Suggested solution
An entry point komodo-lint-maturity that given a release file and a repository validates that all tags in the release file respects the maturity level of the release file itself. The exit code should reflect whether this is indeed the case or not and if not, print the tags not respecting the release maturity.

We probably need an exclude flag as some of the packages are not using semver, i.e. komodo-lint-maturity releases/2020.05.06-py36.yml repository.yml --exclude=opm-common.

Usage
I would like to see this applied to all releases during our CI to ensure that the maturity level is indeed respected.

NB: I know we want to merge the many command line tools in Komodo, but there are already 8 of them and unification is independent of this task.

Stop copy-paste, use git properly

A major problem that plagues komodo is that we keep multiple versions of every package and release, requiring us to copy-paste a non-insignificant amount of text every time we want to add a new version or komodo release.

I propose that we instead use git branches and stop having duplicate versions of releases.

Changes:

• master branch is for upgrade proposals only
• bleeding branch is used to build bleeding only
• Release branches (eg. 2020.06) are created by each respective RM, basing off of master.
• Only two release YAML files exist in any branch -- py27.yml and py36.yml.
• Bumping a package is just changing the values in repository.yml and the two release files.
• A new git tag is a new release that Jenkins picks up and builds.
  • Alternatively, there is a RELEASE_PREFIX file at the root of each branch that contains eg. 2020.06.rc2 -- the name of the release. This string, combined with the YAML release file (eg. py27.yml) is used to create the full name: 2020.06.rc2-py27.
• symlinks.json can be read from master branch, idk.

This proposal will solve at least the following problems of komodo:

• No more having to duplicate entries in repository.yml in order to keep the old versions
• Minimise danger of incorrectly copy-pasting earlier entries (libres in 2020.03)
• Proper diffs in pull-requests
• More organised git history

Misleading slack notification when running symlink suggester on older build

https://ci.equinor.com/komodo/job/komodo-suggest-symlink/128/console succeeded, and sent the following slack msg:

Suggested new symlink (2020.02.02-py27 - unstable): (<|Open>)

The script should fail if not suggesting. Likely now it is not failing, but not suggesting (rather suggesting the empty string).

create kmd --extract-dep-graph repo pkg

usage

    kmd --extract-dep-graph repo [pkg …] [-o file]

For all packages in [pkg …], which are YAML files in the format pkg: <version>\n, use repo to recursively find all dependencies of the packages in [pkg …] and create an output file called extracted-repository.yml (if no -o defined) wherein these dependencies are put.

Ambiguities shall cause the script to fail with an exit code, and the cause of the ambiguity shall be printed to stderr (e.g. foo -> [bar: 1.2.2 | bar: 2.5.0]).

Validate release names

The current pipeline requires each release file to be postfixed with python version, i.e. -py27 or -py36. However, this is not validated before merging. The auto tooling then just silently ignores the newly added release.

Automatic unit testing of _main_ packages in komodo

We want to unit test each of the following packages for some $RELEASE_NAME:

• ERT @sondreso equinor/ert#679
• libres @ManInFez equinor/libres#900
• libecl @jondequinor equinor/resdata#693
• semeio @ManInFez equinor/semeio#40
• equilibrium @ManInFez equinor/equilibrium#10
• everest @jondequinor https://github.com/equinor/everest/pull/1199
• spinningjenny @ManInFez equinor/spinningjenny#192

For consistency, the jenkins file should be named ci/jenkins/Jenkinsfile-komodo.

Automatic removal of dependencies

When removing a package from a release file, it would be nice to also at the same time automatically be able to remove other packages being dependencies (direct + indirect) which are not used by other packages in the same release dependency tree.

Version tags becomes dirty when using setuptools_scm

Given a repository tag defined in the repository.yml, the correct tag is downloaded as expected.

git clone -b 0.0.1 -q --recursive
Note: checking out 'e06b..'.

You are in 'detached HEAD' state. You can look around, make experimental...

and later installed,

However, the written version.py has a dirty tag, as if something in the repo had been changed:
0.0.2.dev0+ge06bf49.d20200123

"Prettier" option

Common CI tools for Python are black for code style formatting, and pylint for static code analysis. JavaScript has a similar quite standard pair: prettier for code style formatting, and eslint for static code analysis.

black / prettier automatically change the code style to comply with some predefined style. Static code analysis tools however reports error, and typically never try to fix them automatically (as that would in many cases involve some assumptions that could change the program behavior).

komodo has "static analysis" which can report missing version definitions, undefined dependency, undefined maintainer etc.

Would also be nice with an automatic "style formatter" ensuring that e.g. the lists are indeed alphabetically sorted, empty lines are consistent, no space before : in yaml dictionaries, no '' around strings, consistently use two space indentation etc. This style formatter could, like black and prettier, have a --check option which could be used in CI to ensure that also a preferred style is followed.

Add vars/komodoTest.groovy

This is a Jenkins pipeline which will:

1. Check if ${sha1} is set. If it is, then it's PR and not komodo integration test.
2. Checkout desired repository
3. Use komodoenv to create a virtualenv
4. Source ci/jenkins/testkomodo.sh
   1. Run either test_pr or test_integration

Optionally, determine if test_requirements.txt exists, pip install it. Or install pytest / tox. Or whatever.

IPython

Hola! Seems like iPython is not working - something worth fixing?

Add tool for dumping library dependencies for elf-files (ldd)

Determine if external repos should provide their own komodotest.sh

We aren't sure if eg. xtgeo should have its own komodotest.sh or if it should exist solely in komodo-releases.

Argument against xtgeo having this is that if they change their testing procedure, they'll notice when .travis.yml complains, but they won't notice that komodotest.sh also needs updating, because it's not being automatically run against their PRs.

Should we allow them to use komodo-integration PRs?

The alternative is that they have a komodotest.sh that doesn't do anything until we complain because bleeding stopped passing tests.

Bash's enable isn't `set +u`-friendly

set +u tells bash to give up if it tries to read from an undefined variable rather than assuming it's an empty string.

Komodo's enable script therefore fails due to MANPATH. These should be replaced with eg. ${MANPATH:-}, where the :- means replace with whatever is next if the variable is undefined.

The version of cwrap reported when used from komodo is wrong (default):

CMake Warning at python/cmake/Modules/find_python_package.cmake:68 (message):
  Python package cwrap too old.  Wanted 1, found Loading cwrap

  0.0.0

Create disable script

Create disable script such that the environment variables set by the enable script is reset to their original values. (Similar to how activate and deactivate works for virtualenv)

Make Komodo portable

Calls to bash, rsync, pip via komodo.shell are hard to debug. STDOUT is swallowed, and it is non-trivial to spit this out again. The assumption here is that all useful info goes over STDERR, which is not always true.

These calls also represents a very turbulent boundary between textual interfaces and Python. You send in strings and get strings back. It in most cases this prevents Komodo from doing any thing intelligent to—or reasoning about why—something failing. Exceptions > error codes.

And it has value that it is easy to run komodo on non-Redhat machines. In the cloud you might use Alpine which ships busybox (subset of normal linux utils). If you run on Debian (and derivatives), the default renamer will crash.

Testing should be easy
At what level should you mock? There's only one available, and that's komodo. And most often komodo is a thin layer on top of a bunch of binaries. Does komodo need testing? Yes.

Komodo incompatible with GNOME on RHEL7

Komodo is incompatible with GNOME on RHEL7 if sourced in a .*rc-file, because that is sourced from .*_profile, and the python version overwrites the system default.

Known issues:

• gnome-tweaks
• frozen login screen

Symlink job fails on check

+ komodo-check-symlinks symlink_configuration/symlink_config.json
Traceback (most recent call last):
  File "/var/jenkins/jenkins-komodo/workspace/komodo-symlink/.venv/bin/komodo-check-symlinks", line 11, in <module>
    load_entry_point('Komodo==1.0', 'console_scripts', 'komodo-check-symlinks')()
  File "/var/jenkins/jenkins-komodo/workspace/komodo-symlink/.venv/lib/python3.6/site-packages/komodo/symlink/sanity_check.py", line 108, in sanity_main
    assert_root_nodes(input_dict)
  File "/var/jenkins/jenkins-komodo/workspace/komodo-symlink/.venv/lib/python3.6/site-packages/komodo/symlink/sanity_check.py", line 85, in assert_root_nodes
    "The roots in the link-tree is not matching the roots defined in link_roots in dict"
AssertionError: The roots in the link-tree is not matching the roots defined in link_roots in dict```

Version for implicit dependencies

For projects that provide a requirements.txt file, those requirements are pip installed when building a komodo release. This can cause at least 2 issues:

• we can get different versions in different builds of the same release (this hit us hard on Everest)
• if the requirements.txt file asks for specific versions of a library, we can get version conflicts

Verify dependency graph tooling

Every package in the repository.yml is required to have dependencies listed, but this is currently not checked in any way.

Installation through pip is used with --no-deps, and we therefore risk the scenario that a dependency is missing if it is not included manually in the repository.yml.

Consider tooling that will verify all dependencies listed (See e.g. https://pypi.org/project/pipdeptree/)

Extracting dependencies does not create pretty releases

Used komodo-extract-dep-graph in a Python 3.7 environment to create the release file in https://github.com/equinor/komodo-releases/pull/249. It was built based on the release 2020.03.rc0-py27 with ert-statoil as the main package to include in the release.

However, the resulting release file was not deemed pretty and hence I had to reformat it afterwards to get the CI pipeline to pass 🤷‍♂

Check outdated packages

Automatic tooling to check for outdated packages.

pip list —outdated provides information about outdated packages.

It might be necessary to differentiate between the behaviour for bleeding versus a release. We probably don't want to check all releases either, as they will become progressively outdated over time.

Sorely needed cleanup of komodo (and komodo-releases)

• move all custom build scripts into komodo-releases/build_scripts or whatever. They do not belong logically in komodo as they are referenced in komodo-releases?
• move all activators (enable, local/enable) to komodo-releases/activators. I do not see any value of having these in komodo. Komodo could take an --activators-dir argument.
• remove minutes
• remove build scripts that are not referenced

Run komodo PR tests also for Python 3

At the moment we are only running integration test for PR agains bleeding-py27, we should also test agains bleeding-py36

komodo dependencies defined twice

Currently komodo has dependencies defined both in requirements.txt and setup.py.

This is a source of errors and confusion. Move to one single source of dependency definitions.

I guess two possible solutions exist:

1. Remove requirements.txt.
2. setup.py reads the content of requirements.txt and use that in install_requires.

Make komodo pip-installable from github

Currently linting scripts etc are not installed properly

Separate kmd' environment from target python environment in Jenkins

We do this for Travis already, but should also do it for the actual Jenkins deploys as well. Travis uses Python 3.6 for both integration tests, so:

• To compile Python 2.7.14 komodo release, kmd uses Python 3.6
• To compile Python 3.8.3 komodo release, kmd uses Python 3.6

However, due to the large amounts of gluten present in the Jenkins code, as well as my general lack of Groovy knowledge, I haven't done this yet for the Jenkins scripts, so kmd there uses the target python version too.

To fix this, I propose adding a new bash script called bootstrap-kmd.sh, that does the following:

• Set up komodo environment:

python3 -m venv kmd-venv
kmd-venv/bin/pip install --upgrade pip
kmd-venv/bin/pip install git+https://github.com/equinor/komodo.git
kmd-venv/bin/pip install virtualenv

• Set up target python build environment:

kmd-venv/bin/virtualenv -p${TARGET_PYTHON} build-venv

• Set up binutils:

mkdir binutils
ln -s /opt/rh/devtoolset-8/root/bin/gcc binutils/gcc
ln -s /opt/rh/devtoolset-8/root/bin/g++ binutils/g++
# etc
ln -s /usr/bin/cmake3 binutils/cmake
ln -s build-venv/bin/pip binutils/pip
ln -s /prog/sdpsoft/git-2.8.0/bin/git binutils/git

Then you may start a komodo build:

export PATH=$PWD/binutils:$PATH
kmd-venv/bin/kmd --download --build --install --cry --get-a-hug --prefix=/prog/res/komodo --etc --etc

Replace compiled python with virtualenv python (builtin)

Since RHEL7 python and komodo python share major and minor versions, they will conflict. This conflict happens due to the Python .so present in LD_LIBRARY_PATH.

Disallow user python site-packages

Komodo uses a vanilla Python interpreter, which by defaults adds ~/.local/lib/pythonX.Y/site-packges to sys.path, overriding stuff from Komodo. This will cause us headaches, such as user pip install --user a package, never updates it. We add this package to Komodo, user complains when everything breaks.

This already broke komodoenv for me, because I have an incompatible version of virtualenv in my .local/lib/pythonX.Y/site-packages. If any users expect Komodo to work like this, then we should inform them that they are doing it wrong.

Jonas' virtualenv approach already has breaking changes wrt. this: virtualenv explicitly does not add user site-packages. Depending on how long we'll keep shipping our own python binaries, we should consider disabling user site-packages via compile-flags.

Tagging @markusdregi @jondequinor and then running away to 🏖️ .

Automatic removal of releases

For all releases that are not linked to by some root (unstable, stable, testing, etc.) should be removed from the releases folder.

`kmd diff` tool

I want to write kmd diff 2020.02.01 2020.03.01 in a releases directory or kmd diff stable-py2 testing-py2 --root=/prog/res/komodo for deployed and have a nice diff of changes, with colours and emojis. The output could be divided into four sections. A is the first argument, B is the second:

• Updates: Exist in both; version in A is lower than in B
• Downgrades: Exist in both; version in A is higher than in B
• Additions: Exist in A; don't exist in B
• Deletions: Don't exist in A; exist in B

Additionally, adding a --markdown (-m ?) flag would print markdown output that can be copy-pasted into Slack or Github.

Drop kmd support for Python 2

There's no technical reason for komodo to need Python 2 in order to build a Python 2 release. We should drop it, since this codebase already doesn't support py2.

To do so, we must make the following changes:

• Komodo's Jenkins job must create a py3 virtualenv (found in eg. KOMODO_PYTHON. Defaults to python3)
• It must create a separate virtualenv for the release we're building.
• The job can start komodo with <absolute path komodo venv>/bin/kmd -build [...]
• Komodo can run pip and stuff with <absolute path to release venv>/bin/pip and so on.

At no point must any virtualenv be sourced.

Testing this might prove a challenge, but I think it's possible. I reckon we want to have a fuller test suite anyway since we're not going to get rid of komodo for some time.

Move build-scripts

Move build scripts to separate folder on root in order to force users to be conscious about the location where the scripts are picked from.

Try bulding a komodo release with Python 3

Add deactivate option

Would be nice to have a deactivate option, could be something akin to virtualenv:

deactivate () {
   if ! [ -z "${_OLD_VIRTUAL_PATH:+_}" ] ; then
        PATH="$_OLD_VIRTUAL_PATH"
        export PATH
        unset _OLD_VIRTUAL_PATH
...
}

Make embarrassingly parallelizable builds build in parallel

Make embarrassingly parallelizable builds build in parallel. This means that for each level in the dependency graph, all nodes at this level should be built in parallel.

This means that if the graph looks like this:

                  python
                    / \
                   /   \
                  /     \
                 /       \
           setuptools   wheel
               / \        /\
              /   \      /  \
             /     \    /    \
            /       \  /      \
           /         \/        \
     contextlib2  cachetools  enum34

three batches of build will occur:

1. python
2. setuptools and wheel
3. contextlib2, cachetools and then enum34

etc.

If, for any reason, some build is not parallelizable, it should be explicitly marked as such in repository.yml.

pyscal:
  v0.5.1:
    source: git://github.com/equinor/pyscal
    fetch: git
    make: sh
    makefile: src/setup-py.sh
    maintainer: havb
    depends:
      - numpy
      - pandas
      - python
      - …
    build_isolation: yes

So, if a package is for some reason not possible to build in parallel with other packages, it can get build_isolation—which currently means that it is built not in parallel with other packages.

Create a "motd" or similar to inform users about Komodo when they enable it

It'd be nice to be able to inform users of changes to Komodo. The current three things that we can inform users about are:

• -py27 is deprecated, please update your jobs to work with -py36.
• Please remove SDPSoft
• You are using an old version of FMU environment, please update. (Automatic after eg, 3 months after release)

These can be done by simply having a file called motd in each release directory, and if it exists during sourcing, print it. We can have a Jenkins job generate these files.

Alternatively, using the slightly more advanced Linuxy route, have a /prog/res/komodo/motd.d directory with bash scripts, that are executed in order. This method will allow us to check if SDPSoft is sourced, for example, but it's has a much bigger risk of failure if we screw up the scripts somehow.

When updating stable, deprecated should also be updated

deprecated points to the previous stable release. We should update both when symlink mode is stable.

Symlink job doesn't play nice when there is a release for multiple pythons

Suppose you're releasing 2020.03.rc0 for both py27 and py36. Once both are built, there will be two PRs.

-py27

        "2020.02-py36": "2020.02.rc8-py36",
+       "2020.03-py27": "2020.03.rc0-py27",
        "bleeding": "bleeding-py3",

-py36

        "2020.02-py36": "2020.02.rc8-py36",
+       "2020.03-py36": "2020.03.rc0-py36",
        "bleeding": "bleeding-py3",

---

First you merge the py27, then try to rebase and merge py36. However, it's ambiguous for git whether it should be

        "2020.02-py36": "2020.02.rc8-py36",
        "2020.03-py27": "2020.03.rc0-py27",
+       "2020.03-py36": "2020.03.rc0-py36",
        "bleeding": "bleeding-py3",

...or...

        "2020.02-py36": "2020.02.rc8-py36",
-       "2020.03-py27": "2020.03.rc0-py27",
+       "2020.03-py36": "2020.03.rc0-py36",
        "bleeding": "bleeding-py3",

That is, it's ambiguous for git whether the user intended to add a new line or replace it because the lines are so similar.

---

I have several proposals on how to fix this:

1. Symlink job can wait until both deploys are complete before offering to rename symlinks
2. Split up the symlinks somehow so that merge conflicts don't occur
3. Wait and see if this is a problem. (I had to fight this on Monday, but it wasn't impossible to resolve manually)

Implement komodo as an explicit state machine

There's an implicit state machine in komodo. Make it explicit.

(Assuming we are to expand komodo and do a major refactor)

Rationale
It might be hard to reason about some flows through komodo, especially the file operations in the end (rename, rsync, .delete, etc). Also, if one of these steps fail for some trivial reason, the most safe thing is to re-build from scratch.

So, we might have easier-to-understand code, possibility to more precisely know the state of the build, thus maybe retry from some stage.

The state machine code should not be written by us.

setup-py.sh downloads dependencies, potentially preventing reproducable builds

Through python setup.py … install, komodo will download anything in order to satisfy dependencies.

This means the komodo dependency graph does not need to be complete, and builds are not necessarily reproducible.

Noticed by 🕵️‍♂️ @sondreso @oysteoh

Add tool for handling single release file for py2, py3, rhel6 rhel7 builds

The new release file use the folling format:

# non py specific
lib-something: 1.0
# non rhel specific
lib:
  py2: 1.0
  py3: 1.0+py3
# rhel specific
otherlib:
  rhel6:
    py2: 1.0
    py3: 1.0+py3
  rhel7:
    py2: 0.9
    py3: 0.9+py3

The new format should be then transpiled into release files to be built.

Add ($KOMODO_RELEASE) to the csh prompt

When you source Komodo in bash today you get the release as a prefix for your PS1, it would be nice to have the same in csh.

For bash, it is essentially

export PS1="(${KOMODO_RELEASE}) ${PS1}"

See the files enable.in (bash) and enable.csh.in (csh).

Format of repository yaml file too verbose

The format of the repository file is unnecessarily verbose. In most cases, different versions of the same software use exactly the same settings. As a user, I would appreciate to be able to add a new version of a software without having to duplicate the entire configuration

Remove repository.yml, use Recipes

Package managers such as nix and Homebrew specify their packages programmatically.

Homebrew:

• Advanced: https://github.com/Homebrew/homebrew-core/blob/master/Formula/python.rb
• Simple: https://github.com/Homebrew/homebrew-core/blob/master/Formula/libxkbcommon.rb

nix example: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/python-modules/numpy/default.nix

---

Komodo recipes example:

Manual example:

class Python3(Recipe):
    versions = [
        "3.6.9-src"
    ]

    @property
    def source(self):
        version = self.version
        self.from_source = "-src" in version
        if self.from_source:
            version.replace("-src", "")
       
        if self.from_source:
            fmt = "https://www.python.org/ftp/python/{version}/Python-{version}.tgz"
        else:
            fmt = "/project/res/komodo/repository/python-{version}/root"
        return fmt.format(version=version)

    def make(self):
        # CWD is the extracted/git cloned repository
        if self.from_source:
            sh(["./configure", "--prefix="+self.prefix, "--enable-shared,  "--enable-optimizations", "--with-ensurepip=install", "LDFLAGS=-Wl,-rpath={}/lib".format(self.prefix)])
            self.make("install")
        else:
            self.rsync(self.source, self.dest)  # haven't thought through this

Advanced example:

class Libres(CMakeRecipe):
    source = "git+https://github.com/equinor/libecl"
    versions = [
        "4.0.0",
        "3.3.0",
        "3.2.2",
        { name: "3.2.2-overrided-example", source: "https://somewhere.else" }
    ]

    @property 
    def makeopts(self):
        if self.python2:
            return [f"-DSITE_CONFIG_FILE={self.prefix}/etc/ERT/site-packages",
                       "-DINSTALL_ERT_LEGACY=ON",
                       "-DUSE_RPATH=OFF"]
        if self.python3:
            return ["-DUSE_RPATH=OFF"]

    @property
    def dependencies(self):
        extra = []
        if self.python2:
            extra = ["functools32", "futures"]
        return ["PyYAML", "configsuite", "cwrap", "future", "jinja2", "libecl", "numpy", ]

Simple example:

class Numpy(PipRecipe):
    # no source -- default to PyPI
    versions = [
        "1.17.2",
        "1.16.4"
    ]
    dependencies = ["setuptools", "python"]

komodo-check-symlinks fails with legacy releases

Currently this script checks that the root links (ie, symlinks that no other symlink points to, eg. testing-py2) on harddisk match the ones in the config file. We're getting errors because nothing points to the 2020.02-py36 link -- therefore it's a root link -- but it's not in the config.

I am not certain why this script doesn't show 2019.12 as a root link, since it also has no parent links. (via printf-debugging)

PS: Also this script does the graph traversal in O(n^2) time, rather than amortised O(n). We're literally bleeding compute money here.

Inject lib/python/site-packages into PYTHONPATH

As of current, the recommendation for sourcing komodo (for fmu setups?) is to source the SDP_bashrc script first, then komodo.

SPD_bashrc injects things into PYTHONPATH, which takes precedence. Packages that follows the komodo python binary that are also in PYTHONPATH are effectively gone.

A solution to this is to simply inject komodo's site-packages into PYTHONPATH.

Cache option should function as a cache between repeated builds

Right now, arbitrary things are done to the cache files. They should be treated as read-only so to be re-used.

And every fetch method should check for existence before trying to (re-)download anything.

Consider improving CLI API

Current:
--tmp has no default, is passed onto komodo.build as builddir but is largely ignored by the builders
--cache has no default, becomes outdir in komodo.fetch (which defaults to . and is created). It is also cd-ed into to ease tar and pip calls I assume. Also passed onto komodo.build as dlprefix, which is used to find e.g. find downloaded packages.
--workspace has no default, but will just become the directory from where you invoked kmd. The activators are assumed to be here if the folder exists. If the folder does not exists, the activators are copied over.
--fakeroot has no default, but becomes fakeroot = os.path.abspath(args.release) which probably has its reasons. However, in komodo.build it is either set to prefix via the cli, or to .. This is outright confusing. It's also a bool option.

cache is also not a cache the way I understand it, as things are built/moved around in there after fetching it. I think tmp is better suited to do temporary things pertaining to the build, whereas the cache could serve multiple builds.

In conjunction with #124, this setup makes komodo unnecessarily hard to understand. This is aggrevated by a bunch of cd operations.

In conjunction with #122, preparatory steps for a komod build involves copying stuff from here to there.

Can you even build komodo without cloning komodo (and of course komodo-releases)?

Maybe komodo could take a repository file, a release file, and a working directory (in addition to a prefix)? All other things should live inside komodo and the komodo pip package.