twisted-infra / braid Goto Github PK
View Code? Open in Web Editor NEWAutomation scripts for twistedmatrix.com
License: Other
Automation scripts for twistedmatrix.com
License: Other
Is there something as convenient as .format, but that properly handles
shell quoting?'from pipes import quote'. You'll need to write a function that combines that with 'format', of course.
Note that there are a couple of places where we currently depend on spaces in interpolated strings causing arguments to be separated. (Nothing that isn't fixable, but something to look out for an fix, when addressing this.)
froztbyte commented:
I tend to use absolute paths for system utils just in case there's PATH fuckery (a good recent example I've run into is sensible-browser vs xdg-open vs not having either of those), and would suggest perhaps just being a bit more specific about the path layout of all the various bits which braid sets up.
The braid.bootstrap step fails hard if apt-get install ...
fails.
It could run apt-get update
first.
$ fab -f fabfile.py t-names.start
Fatal error: The following specified roles do not exist:
nameserver
Aborting.
If using virtualenv, this means sourcing the virtualenv and ~/bin
.
Otherwise ~/bin
and ~/.local/bin
at least (and perhaps ~pypy/bin
if appropriate).
trac will need this, and twisted-buildbot-configuration needs this (it actually needs it during install to run buildbot upgrade-master
).
Both pyOpenSSL and Twisted installation on PyPy fails:
[54.242.142.31] sudo: ~pypy/bin/pip install svn+svn://svn.twistedmatrix.com/svn/Twisted/trunk/
[54.242.142.31] out: Wheel installs require setuptools >= 0.8 for dist-info support.
[54.242.142.31] out: pip's wheel support requires setuptools >= 0.8 for dist-info support.
[54.242.142.31] out: Storing debug log for failure in /home/ubuntu/.pip/pip.log
[54.242.142.31] out:
fail2ban is now installed into the dornkirk mailman configuration but its deployment is not automated. We should automate that.
Currently, tasks which support multiple target hosts, are run in sequence. It is possible to run them in parallel by passing the -P
command line flag when invoking fabric.
We can make this behavior the default by wrapping the generated tasks in the parallel
decorator inside Service.getTasks
.
The data that needs to be preserved is the db, ~/attachments
and htpasswd
.
Rather than having each class link ~/config/start
to ~/bin/start
, have the base class do this, and also do it for stop
if that exists, rather than using the default one.
Also, we could perhaps generate a typical start
, and we should create a restart
as well.
mailman's wrapper (used by the cgi script, limits the groups that can access it.
Currently we just have eggs in version control, rather than the sources.
The monitor can be a separate service in the same tac file, that just triggers trac (but not the post-commit-hook) to restart.
Packages are nice, we should use packages.
Among other things, the current PyPy deployment situation means that we can't deploy on an i386 VM for testing.
The code probably wants to be factored into a function for generating the readme, as well as a generic function like fabric.contrib.file.upload_template
.
By creating a specific user to administer services, and using its .ssh/authorized_keys as target for the Match Group directive, we can assign different privileges to users (install/setup for root only and restart/update for every service admin).
Functions to add/remove keys to the authorized_keys file should be provided.
The current deployment uses a local branch, but it should be a seperate repo.
It probably wants to live in ~/private
My notes indicate that this is currently ~/twisted/Releases
and ~/distributable
.
There should be commands to dump+restore that directory, as well.
Following the instructions in the ReleaseProcess[1] on the wiki, I run into an error on the final step:
(braid)red ~/braid/ (master) $
cd ~/braid; fab config.production t-web.uploadRelease:13.2.0,~/release.tar.bz2
Warning: Command(s) not found:
t-web.uploadRelease
<rest of available commands here>
I'm not sure if anything is in a branch, so if it is, the ReleaseProcess document should match it. If the functionality doesn't exist, it should be reverted to the old version.
(just keeping track of things to discuss over here)
Virtual environments should probably still be preferred to user local installations as they provide more isolation from the system site-packages, allow to run different versions of python and don't require scripts to be invoked by specifiying the correct interpreter.
We could also activate the correct environment in each user's .profile in order to have it available for each command.
Currently 3 dependencies are listed:
It would be nice to eliminate this documentation and, for each of these, respectively:
This is all the directories in ~/data
plus state.sqlite
there.
dependencies
is a special case, but does need to be handled as well.
t-web configuration redirects to twistedmatrix.com so it's hard to load content just from an alternate testing host
.local/bin/trac-admin config/trac-env/ permission add <username> wiki_editors
for example
fab -u somebody ... trac.install
still logs in to the server as root
if somebody
has sudo permission that should be sufficient.
so trac can't actually be started just from braid setup, a trac environment needs to be copied from somewhere else or otherwise manually initialized
The website has moved, and #18 is moving the subdirectory where the releases live.
They should say what the service is, pointers to the config file, additional paths used, fabric commands available, dump/restore procedure, etc.
but there is no such thing. t-web.install installs twisted with --user
putting twistd at ~t-web/bin/twistd
This probably means on wolfwood. One tricky bit, is how to make it accesible to the service user.
Either there needs to be a shared user on wolfwood for accessing it, or people need to put their user in ~/.config/braid/production.env
somehow. (But the latter means the remote won't be correct).
Currently environment shortcuts (test and prod) have to be imported manually into all service fabfiles which want to make use of them. We should eventually get rid of them completely or make them a generic single import.
A possible solution can be to move them to the braid.env
package and make the environment
task the default one. This would have as consequences a slightly longer but more explicit command line syntax: fab env.test myTask
Code used to deploy to production should be recorded, so we should ensure that that is being done.
buildbot and trac will actually addtional things to go there (see #23).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.