vghn/puppet

Vlad's Puppet Control Repo

Puppet Control Repo

Development status

This project is still in a prototype development stage.

Overview

Vlad's Puppet Control Repo.

Description

bin/

Contains various executable scripts.

data/

Contains the hiera data files. It's intended to serve as a base only, for public data, with sane defaults. It should be overwritten or amended with data from private sources.

dist/

Contains organization-specific roles and profiles. This directory is specified as a modulepath in environment.conf Designing Puppet – Roles and Profiles.

manifests/

Contains Puppet's manifests:

• site.pp: the main manifest

spec/

Contains test definitions.

.env

Contains private environment variables.

environment.conf

This file can override several settings whenever the Puppet master is serving nodes assigned to that environment. Config Files: environment.conf

envrc

This file contains global variables. All variables declared here are public

hiera.yaml

This file configures Hiera per environment

Puppetfile

r10k needs this file to figure out what component modules you want from the Forge. The result is a modules directory containing all the modules specified in this file, for each environment/branch. The modules directory is listed in environment.conf's modulepath.

Testing

Prerequisites

• Vagrant: https://www.vagrantup.com/
• Docker: https://www.docker.com/
• RVM: https://rvm.io/
• GIT: https://git-scm.com/

cd dist/profile
bundle install

Unit testing

cd dist/profile
bundle exec rake test

Acceptance testing

Run default set for the first time (default is just the base profile):

PUPPET_INSTALL_TYPE=agent ORDERING=manifest BEAKER_destroy=no bundle exec rake integration

Subsequent runs:

PUPPET_INSTALL_TYPE=agent ORDERING=manifest BEAKER_destroy=no BEAKER_provision=no bundle exec rake integration

Last run (will destroy the machines):

PUPPET_INSTALL_TYPE=agent ORDERING=manifest BEAKER_provision=no bundle exec rake integration

Complete example:

PUPPET_INSTALL_TYPE=agent ORDERING=manifest BEAKER_destroy=no BEAKER_provision=yes BEAKER_debug=yes BEAKER_set=default BEAKER_role=none bundle exec rake integration

Other commands:

• List nodes: bundle exec rake beaker_nodes

The following environment variables can be used to influence how beaker works:

• BEAKER_role: choose a role from spec/acceptance/support/roles/*.yml
• BEAKER_set: choose a nodeset from spec/acceptance/nodesets/*.yml defaults to default
• BEAKER_setfile: specify a nodeset using a full path
• BEAKER_provision: set to no to re-use existing VMs
• BEAKER_keyfile: specify alternate SSH key
• BEAKER_debug: set to any value to enable beaker debug logging
• BEAKER_color: set to no to disable color output
• BEAKER_destroy: set to no to keep the VMs after the test run. Set to onpass to keep the VMs around only after a test failure.
• ORDERING: How unrelated resources should be ordered when applying a catalog (https://docs.puppetlabs.com/puppet/latest/reference/configuration.html#ordering)
• PUPPET_INSTALL_TYPE: specify puppet type (one of: pe, foss, agent)
• PUPPET_INSTALL_VERSION: specify the version to install

Clean-up

cd dist/profile
bundle exec rake test_clean

Contribute

See CONTRIBUTING.md file.

License

Licensed under the Apache License, Version 2.0. See LICENSE file.

puppet's Issues

Add a cron job during maintenance window to update everything (repo, docker images, restart containers)

Install modules from GitHub

It's faster than Puppet Forge

Take advantage of https://github.com/puppetlabs/puppetlabs_spec_helper#parallel-fixture-downloads

The r10k:dependencies should check git modules too

Do not encourage piping curl into sh

For the bootstrap script

Replace profile::swap with a puppet approved module

https://forge.puppet.com/petems/swap_file

Replace `create_resources` with iterations

Use parallel testing

https://github.com/puppetlabs/puppetlabs_spec_helper

env:
  - FUTURE_PARSER=yes CI_NODE_TOTAL=2 CI_NODE_INDEX=1
  - FUTURE_PARSER=yes CI_NODE_TOTAL=2 CI_NODE_INDEX=2

aws sync does not download one byte changes

For example change 1.8.0 to 1.8.1 is synced TO S3 but not FROM

Remove "essential packages" from nodes

Leave only what is required on each

Explicitly reject the connection if backend servers are down

acl backend_dead nbsrv(backend_name) lt 1
tcp-request reject if backend_dead

    environment:
      EXTRA_FRONTEND_SETTINGS_8140: 'acl backend_dead nbsrv lt 1, tcp-request connection reject if backend_dead'

Terminate SSL at LB level

Switch to Puppet 5

Move RSpec tests into the root of the control repo

Add the VGS Library

https://github.com/vghn/vgs

Add documentation

https://github.com/blog/2233-publish-your-project-documentation-with-github-pages

Test each AMI before it is created

Reuse the acceptance testing for that role

Remove ec2tagfacts module

For now this doesn't work. It requires aws cli which might not be installed in the Docker container (errors if not present).

Create a profile for small changes

Use `$facts[]` instead of global variables

Benefits: More readable and maintainable code, by making facts visibly distinct from other variables. Eliminates possible confusion if you use a local variable whose name happens to match that of a common fact.

Drawbacks: Only works with Puppet 3.5 or later. Disabled by default in open source releases prior to Puppet 4.0.

Generate a CSR Attributes file before installing Puppet

In case the agent starts during installation, and generates the certificates.

Configure Samba to use at least SMBv2

https://www.cyberciti.biz/faq/how-to-configure-samba-to-use-smbv2-and-disable-smbv1-on-linux-or-unix/

Replace `validate_` functions with Puppet 4 data types

Pin down docker repository architecture

Restart service in profile::misc

trusty sudo restart systemd-logind
xenial systemctl restart systemd-logind.service

Improve lookup functions

Make a single options hash that includes 'name'

Remove docker profile until the module is updated

For now, it does not have the new docker repositories and package names.

Fix acceptance tests

WARNING: Shared example group 'profile::aws::cloudformation' has been previously defined at:
  /Users/vlad/Dropbox/Projects/puppet/dist/profile/spec/acceptance/support/aws/cloudformation_spec.rb:1
...and you are now defining it at:
  /Users/vlad/Dropbox/Projects/puppet/dist/profile/spec/acceptance/support/aws/cloudformation_spec.rb:1
The new definition will overwrite the original one.

Add hardening modules

https://github.com/dev-sec/hardening/tree/master/puppet

However, saz/ssh is a dependency and it needs to replace ghoneycutt/ssh

Add cronjob to clean RHEA instance

echo "Removing dangling images..."
docker images --no-trunc --quiet --filter dangling=true | xargs --no-run-if-empty docker rmi

Docker module reinstalls old linux kernel packages (after apt updates them)

Set manage_kernel to false (it defaults to true)

Clean-up hooks

Put all checks in functions and fix checks for exit code

Bootstrap script should check if apt repo is installed

if which puppet > /dev/null 2>&1 && apt-cache policy | grep --quiet apt.puppetlabs.com; then
  echo "Puppet is already installed."
  exit 0
fi

Remove MaxPermSize from docker-compose.yml

OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0

Fix deploy command

bin/deploy throws ERROR: No such service:

Update docker-compose

Use host local time inside containers

RuboCop

RuboCop should only be used locally, in the development group. It's probably not a good idea to run on the CI.

Add vladgh/common module

Do not hardcode puppet environment in the configuration when bootstrapping

Just use the ENVTYPE var for the puppet run commands.

PS: Ensure the environment directory exists though. Puppet will complain if it doesn't.

Use the containerized TravisCI environment

Use the new Puppet rolling repos

Use the new docker package

Fix gem dependecies

Your bundle requires gems that depend on each other, creating an infinite loop. Please remove gem 'stomp' and try again.

Simplify Puppet bootstrap script

Remove the apply functions.

Ensure a logical manifest order through out

By default, Puppet will use the order in which the resources were declared in their manifest files: https://docs.puppet.com/puppet/latest/reference/configuration.html#ordering

Use valid SSL certificates for the data agent

Allow generation if no certificate is specified

Add rubycritic and reek

https://github.com/whitesmith/rubycritic

https://github.com/troessner/reek

Replace VGS release scripts, with a simpler rake task

Adhere to recommended community standards

Upgrade git

There are always 2 packages that need upgrading because git is installed during the image build and puppet adds the latest git repo, but the git package defaults to ensure=latest.

The git class should have package_ensure => 'latest'