praekeltfoundation/travis-pyenv

Set up pyenv to use in Travis CI

travis-pyenv

This project is no longer used by Praekelt.org and won't be maintained. It will remain here if you'd still like to use it. Feel free to adopt it if you want to make some changes.

Set up pyenv to use in Travis CI builds.

Setting up pyenv properly in a Travis CI build environment can be quite tricky. This repo contains a script (setup-pyenv.sh) that makes this process much simpler.

Use cases for this include:

• Install an up-to-date version of PyPy. The Travis CI build images currently contain a very old version of PyPy which breaks some common Python modules.
• Install an exact version of CPython or some other lesser-known distribution that Travis CI doesn't support.
• Install Python on macOS builds.

Usage

1. Set the $PYENV_VERSION environment variable to the Python to install.
2. Tell Travis to cache the $HOME/.pyenv_cache directory.
3. Download and source the setup-pyenv.sh script in before_install.
4. Build your project and run your tests as usual.

There are a few install options that can be set via environment variables:

• PYENV_VERSION The pyenv to install [required]
• PYENV_VERSION_STRING String to grep -F against the output of python --version to validate that the correct Python was installed (recommended) [default: none]
• PYENV_ROOT Directory in which to install pyenv [default: ~/.travis-pyenv]
• PYENV_RELEASE Release tag of pyenv to download [default: clone from master]
• PYENV_CACHE_PATH Directory in which to cache pyenv's Python builds [default: ~/.pyenv_cache]
• VIRTUALENV_EXTRA_ARGS Extra arguments to be used when creating the virtualenv

Example travis.yml

language: python
dist: trusty
matrix:
  include:
    - env: PYENV_VERSION='2.7.13' PYENV_VERSION_STRING='Python 2.7.13'
    - python: '3.5'
    - env: PYENV_VERSION=pypy2.7-5.8.0 PYENV_VERSION_STRING='PyPy 5.8.0'
    - env: PYENV_VERSION=pypy3.5-5.8.0 PYENV_VERSION_STRING='PyPy 5.8.0-beta0'
cache:
  - pip
  - directories:
    - $HOME/.pyenv_cache

before_install:
  - |
      if [[ -n "$PYENV_VERSION" ]]; then
        wget https://github.com/praekeltfoundation/travis-pyenv/releases/download/0.4.0/setup-pyenv.sh
        source setup-pyenv.sh
      fi

script:
  - py.test my_project

Notes

• Some recent PyPy versions and all recent "Portable PyPy" versions require Travis' Trusty CI build environment. See pyenv/pyenv#925.
• Installing pyenv by downloading a release tag rather than cloning the git repo can make your builds a bit faster in some cases. Set the PYENV_RELEASE environment variable to achieve that.
• If you want to use $PYENV_CACHE_PATH, you must also set up Travis to cache this directory in your Travis configuration. Using the cache is optional, but it can greatly speed up subsequent builds.
• The $PYENV_ROOT defaults to ~/.travis-pyenv, rather than the usual ~/.pyenv. This is because some of Travis' Trusty build environments already have a pyenv install in this location.
• pyenv fails to install properly if the $PYENV_ROOT is already present, even if the directory is empty. So if you set Travis to cache any directories within the pyenv root, then you will probably break pyenv. For this reason, Python builds are cached outside the pyenv root and then linked after pyenv is installed.

travis-pyenv's Issues

Improve logging for `verify_python` function

When the version string mismatches, we don't log the output of python --version (because reasons...) which makes mismatches difficult to debug.

installation with PYENV_VERSION=3.5.3 failed

Just trying travis-pyenv for the first time and hit this:

2017-11-20 21:17:21 (131 MB/s) - ‘setup-pyenv.sh’ saved [5006/5006]
**** Installing pyenv.
mkdir: cannot create directory ‘/opt/pyenv’: File exists
**** Trying to find and use cached python 3.5.3.
No cached python found.
**** Installing python 3.5.3 with pyenv now.
pyenv: /opt/pyenv/versions/3.5.3 already exists
continue with installation? (y/N) 

[...and now my build is stuck]

I also specified PYENV_RELEASE="v1.1.5" (latest, and recent, pyenv release)

Full build log at https://travis-ci.org/jab/bidict/jobs/304959775

In general, ideally there'd be some way to make sure this can run in completely non-interactive mode, no matter what questions it might ever have.

Can't set extra arguments to virtualenv

My actual use case is wanting to set --system-site-packages which I think might do horrible things, but I can imagine some (fairly rare) cases where users have less awful ideas for this

Travis trusty environments already have pyenv installed

...which breaks our installation.

We should either:

• Install our copy of pyenv in a different-from-default location
• Try update Travis' installation

Installation is broken with PyPy 3.5-5.7.0 (portable)

PyPy 3.5-5.7.0 does not seem to include a python symlink in it's bin/ folder, which breaks the assumptions from this script, making the installation fail.