Please be aware that DYE is not stable yet - it is a rework of our internal deployment scripts to make them easier to reuse, and the rework is a work in progress. Please have a play, test against your dev server and give us feedback. But don't download and immediately run against your production server.
DYE is a bacronym for "Deploy Your Environment" - a set of scripts and functions to deploy your web app along with the required python libraries in a virtualenv, either locally or on a remote server. It is built on fabric. It is most well developed for Django web apps, but we have used it for PHP projects aswell.
You can create a compatible project using cookiecutter.
See readme-cookiecutter.rst for details on how to do this.
A bare bones project structure would be:
/apache <- contains config files for apache, for different servers
staging.conf
production.conf
/deploy <- contains settings and scripts for deployment
bootstrap.py
fab.py
localfab.py <- optional
localtasks.py <- optional
pip_packages.txt <- list of python packages to install
project_settings.py
tasks.py
ve_mgr.py
/django
/website <- top level for Django project
manage.py <- a modified version of manage.py - see examples/
.ve/ <- contains the virtualenv
local_settings.py <- a link to the real local_settings.py.env
local_settings.py.dev
local_settings.py.staging
local_settings.py.production
manage.py <- our modified version
private_settings.py <- generated by these scripts
settings.py <- this will import local_settings.py
urls.py
/wsgi <- holds WSGI handler
wsgi_handler.py
A certain amount of the directory structure can be overridden in project_settings.py but that is not well tested currently.
tasks.py is designed to make it easy to get your development environment up and running easily. Once the project is set up, getting going should only require:
cd deploy
./bootstrap.py
./tasks.py deploy:dev
cd ../django/django_project
./manage.py runserver
bootstrap.py will create the virtualenv and install the python packages required
tasks.py deploy:dev will:
- generate a .private_settings.py. (random database password and Django secret key)
- link to one of your local_settings files (selects database etc)
- init and update git submodules (if any)
- create the database (if using MySQL at least) and run syncdb and migrations
Your Django application will then be good to go.
tasks.py includes a number of other tasks that should make your life easier. It also makes it easy to add your own tasks and integrate them into the deploy task by using:
This is a file where you can define your own functions to do stuff that you need for your project. You can also override functions from tasklib.py simply by defining a function with the same name as the function in tasklib.py
You can override the main deploy() function, but you might lose out if the
deploy function starts to do more. Generally a better strategy is to define a
post_deploy() function - when you run ./tasks.py deploy then after the other
tasks have completed it will check if there is a function called post_deploy()
in localtasks.py and if so it will call it.
We use a modified version of manage.py that knows about the virtualenv in the
.ve/ directory. So when you run manage.py it will automatically relaunch itself
inside the virtualenv, so you don't have to worry about activating/deactivating.
It also knows about the list of packages in pip_packages.txt - if that is
updated without the virtualenv being updated (or if the virtualenv doesn't
exist) then manage.py will complain. You can then create/update the virtualenv
with:
../../deploy/bootstrap.py
Note that this will only update the virtualenv if it's required, though you can
use --force to do it anyway. Also note that by default it will only update the
packages that need updating - use --full-rebuild to force it to delete the
virtual env and then rebuild. To see all options use --help.
We have developed a set of fabric functions for our standard project layout. In order to avoid violating the DRY principle, we delegate the work to tasks.py where possible. Our standard fab deploy will:
- check if you have made any local changes to the server. If it finds any it will alert you to them and give you the choice of whether to continue or not.
- if using git it will check the branch set in project_settings.py, the branch currently on the server and the branch you are currently on locally. If there is a mismatch it will ask you to confirm what branch you want to use.
- create a copy of the current deploy in a directory called
next/- or create the directory on the server (if this is the first deploy) - checkout or update the project from your repository (git, svn and CVS
currently supported). It uses the
next/directory so that your running website is unaffected. - remove all
*.pycfiles. (If x.py is removed by your VCS but x.pyc remains, then as far as python is concerned, x.py is still present). - ensure the virtualenv is created and packages installed (as bootstrap.py does)
- unlink the webserver config and reload the web server (effectively turning off the site)
- do a database dump in the current directory
- move the current directory to a subdirectory of
previousso that a rollback can occur, and then movenext/to where the current directory is. - call
tasks.py deploy- which ensures settings are ready and does all the database related stuff. - relink the webserver config and reload the web server (effectively turning on the site again)
- delete excess copies in
previous/(by default 5 copies are retained).
As with tasks.py you can add extra functions and override the default behaviour by putting functions in:
Dye will create /var/django/project_name and in that directory will be:
dev/ <- contains the checked out code
previous/ <- copies for rollback, with directories named by timestamp
During a deploy there will also be:
next/ <- only temporary
You can override the project root with the server_home variable in project_settings.py
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent