yaybu/yay

yay ain't YAML

Yaybu

Yaybu is a push based configuration management tool written in Python with the goal of helping you tame your servers. You describe your infrastructure in a simple and flexible YAML-like language and Yaybu works out what needs to happen to deploy your updates.

All the documentation is available on http://docs.yaybu.com.

Yaybu is very new software, please subscribe to the yaybu-dev mailing list and let us know if you are using it, we'd love to hear from you. You can find the mailing list at http://lists.yaybu.com.

You can also find us in #yaybu on irc.oftc.net.

yay's Issues

Improve protected string support

Rename to yay.string.String, and deprecate ProtectedString

Ensure that constructor actually calls self.add() so that raw strings passed to a ProtectedString are wrapped

To support more uses cases like building a command line as a list, support accessing an unprotected and protected list

class String:

    @property
    def protected_parts(self):
        """ Return a list of all string parts with obfusication """

    @property
    def unprotected_parts(self):
        """ Return a list of all string parts with no obfusication """

Bug in macro variable handling

The following does not work:

macro m:
    foo: {{arg}}

bar: baz

r:
    call m:
        arg: {{bar}}

YEP2: Default values

Abstract

This YEP proposes a method to have default values.

Problem

In Yaybu you have might normally deploy a file into /var/local/www but 1 in a 100 times you want to override that value and use /mount/some_nfs/www. It would be annoying to specify the common case every single time.

Proposed Solution

We will add an else keyword to the expressions handling. The else keyword will be parsed like an or is processed, but the alternative will only be chosen if processing the preceeding option results in a NotFound exception.

environments.append:
  - name: www-foo-com
  - name: www-bar-com
  - name: www-baz-com
     checkoutdir: wibble

resources.foreach e in environments:
  - Checkout:
        name: /var/www/${e.checkoutdir else e.name}

Scope

This adds additional behaviours to ${} expressions.

Risks

N/A

Transition

We will need to improve KeyError handling and make sure that appropriate NotFound errors are raised.

CycleError when mixing yaybu.options and yaybu.searchpath

I instrumented the CycleError detector and got this:

want (<bound method LazyPredecessor._expand of <LazyPredecessor 41139664>>, ())
(<bound method AttributeRef._expand of <AttributeRef 40565968>>, ())
(<bound method AttributeRef._get_key of <AttributeRef 40565968>>, ('searchpath',))
(<bound method PythonicWrapper._get_key of <root>/yaybu>, ('searchpath',))
(<bound method AttributeRef._expand of <AttributeRef 40566160>>, ())
(<bound method PythonicWrapper._expand of <root>/yaybu/searchpath>, ())
(<bound method Include._expand of <Include 41070416>>, ())
(<bound method Include._expand of <Include 41123792>>, ())
(<bound method Include._expand of <Include 41136656>>, ())
(<bound method Directives._expand of <Directives 41070224>>, ())
LOOP (<bound method LazyPredecessor._expand of <LazyPredecessor 41139664>>, ())
(<bound method AttributeRef._expand of <AttributeRef 40611344>>, ())
(<bound method AttributeRef._get_key of <AttributeRef 40611344>>, ('options',))
(<bound method PythonicWrapper._get_key of <root>/yaybu>, ('options',))
(<bound method AttributeRef._expand of <AttributeRef 41138320>>, ())
(<bound method PythonicWrapper._expand of <root>/yaybu/options>, ())
<ROOT>

In master calling expand() on a dict will chain expansion to predecessors. This ensures that Include's happen in the right order. So accessing yaybu.options will cause all the imports to happen. Unfortunately the includes need yaybu.searchpath, so this call will call expand on the dicts too, triggering cycles.

You can move the exception arround by moving yaybu.options, but it will still trigger.

Appending a dict to a list causes horrific traceback

go on just try it.

Support .d directorys

Extends mechanism should be able to include all files in directory:

.include:
        - file:///etc/yay.d/*.yay

Test error when installing

Processing triggers for python-support ...
Compiling /usr/lib/pymodules/python2.5/yay/doctests/test_documentation.py ...
File "/usr/lib/pymodules/python2.5/yay/doctests/test_documentation.py", line 32
globs = globs

support multiple imports of the same file

only process the file once!

Doesn't like blank files

Had a config that included multiple files at the end, some of which are currently blank. That seemed to block data from showing up in final output.

YEP1: Reusable chunks

Abstract

This YEP proposes to provide reusable blocks of config that can be inserted with different parameters.

Problem

When using Yay in Yaybu we often repeat the same things over and over again. While you can work around this to some extend using hooks like this:

things: []

resources.append:
  .foreach thing in things:
    Execute:
      name: do-stuff-to-${thing}

things.append:
  - workaround

But you sacrifice control over when execution occurs. This can lead to 'cannot forward declare' errors.

Proposed Solution

You define a reusable chunk with the .define

.define Virtualenv:
      - Directory:
          name: ${directory}
          owner: ${owner}
          mode: 0755

      - Execute:
          name: virtualenv-${directory}
          command: virtualenv --no-site-packages ${directory}
          user: ${owner}
          creates: ${directory}/bin/pip

      - Execute:
          name: pip-${directory}
          command: ${directory}/bin/pip install -r requirements.txt
          user: ${owner}
          cwd: ${checkout}

You can call it with the @ decorator:

.foreach ev in envs:
    - Checkout:
        name: /var/loca/checkouts/zap

    - @Virtualenv:
        directory: /var/local/envs/zap
        hoo: ha

.define Virtualenv:
  File:
    name: /var/local/sites/${name}

# Here, the Virtualenv definition is captured from the foreach
resources.append:
  .foreach name in the_big_book_of_names: ${@Virtualenv}

# Here it name is not from the foreach, the value that is passed in explicitly is used.
resources.append:
  .foreach name in the big_book_of_names:
    - @Virtualenv:
        name: this has priority

# Your chunks can yield mappings, sequences or literals
.define LoveYou: Hi ${name}, you are amazing.

markov.append:
  .foreach name in the big_book_of_names: ${@LoveYou}

markov.append:
  .foreach name in the_big_book_of_names:
    @LoveYou:
      name: Mr Eeejit

Scope

This changes the behaviour of mapping keys and adds additional behaviours to ${} expressions.

Risks

Any string starting @ will become a reserved word.

Transition

This introduces new language features - it shouldn't change any existing language behaviour, but might affect any configs that use "@" in their keys.