jclaveau/ansible-vagrant-modules

Vagrant commands and config management as modules for Ansible to let you control vagrant VMs from an ansible playbook.

ansible-vagrant-modules's Issues

Add support for docker provisionning

https://blog.scottlowe.org/2015/02/10/using-docker-with-vagrant/
https://www.vagrantup.com/docs/provisioning/docker

• Investigate if a single integration for all types provisioning is possible (based on functions calls)

Module error while calling status command on non-existing vm

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: KeyError: 'non_existing_vm'

• Fix this error
• Add integration tests

Integration test for config

create a tests.sh file to fit the README

Create python-vagrant wrapper to gather missing features

python-vagrant misses a lot of features / parameters and is not maintained. A wrapper to it seems to be the best way to gather those new features

TODO:

• Create the VagrantUpdated class
• Move python-vagrant parts to it

Remove config files if all vms are removed

If all vms are removed from the config remove vagrant-hosts.yml
If the Vagrantfile is the same as the one in the module, remove it.
Concerning the groups redefining its api first is required #36

Investigate vagrant box cache for integration test workflow

https://stackoverflow.com/questions/66261101/using-vagrant-on-github-actions-ideally-incl-virtualbox

Replace inconsistent _instX implementation by a pattern based system

vagrant:
  cmd: up
  vm_name: frank_%d
  count: 8

vagrant:
  cmd: halt
  vm_name: frank_\d+
  vm_name: frank_(0|3|8)

https://pytutorial.com/python-variable-in-string

Support a network entry

network:
 - - public_network
   - ip: x.x.x.x
     type: dhcp
     bridge: ...
 - - private_network
   - ip: x.x.x.x
     netmask: x.x.x.x
     :libvirt__network_name: my_network

Tell inspiring repos that this work is based on them and that their are credited

• forks of ansible-vagrant
  => majidaldo/ansible-vagrant#5
  => robparrott/ansible-vagrant#6
• ansible-skeleton https://github.com/bertvv/ansible-skeleton
  => bertvv/ansible-skeleton#20
• Vagrant roles amtega: https://github.com/amtega/ansible_role_vagrant_provisioner
  => need a tech monitoring before, their work is handles all asnyc cases
• Vagrant roles teampants:
  => teampants/ansible-vagrant#2

See: #17

Fix integration tests in a playbook && ensure no more false positive

Implement suspend

Refacto destroy

Same as jclaveau/ansible-vagrant-modules-legacy@08fa2bc for destroy

Refactor config

Implement the binding to parallel parameter

vagrant up and vagrant destroy provide a parallel parameter which is passed to the chosen provider.
As a workaround I chose to delegate this responsibility to Ansible for now but it would be a very nice feature.

Acceptance:

• Have a working parallel parameter for vagrant up.
• Have a working parallel parameter for vagrant destroy.

Todo

• Check if no other command has a parallel parameter
• Implement commands for all or several instances #31
• Extend python-vagrant commands to support it #32
• Remove this issue from the Important know issues of the readme

Remove dependency to Vagrant.json (at least for the sub commands, maybe for Up)

This dependency seems to be fat over-engineered code painful to maintain. This module meant to be a binding to Vagrant with as limited changes as the shell usage

• Remove all references to _instances
• Uncomment test on count in ssh_config testcase

Vagrantfile parser in python

• https://github.com/drewsonne/pyvagrantfile

Update de documentation

https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_documenting.html

• Add my modifications
• Move changelog at the bottom
• Add a usage preview e.g a playbook
• Update to Ansible 2.10 syntaxe
• Update status data structure if not done
• Change "codecov" as "coverage" in the tag
• Add atag for ansible-galaxy (once published?)

Playbook integration tests

delegate_to doesn't work when applied to hosts added with add_host in roles so in integration tests but it works in playbooks.

• Add integration-playbooks tests

Have badges by branch

Integration test for ssh command

Add license scan report and status

Legacy from jclaveau/ansible-vagrant-modules-legacy#7

Integration test for halt and fix it

Code coverage

Use codecov from github action

Up integration tests in refacto

Check if alternative code doesn't exist

• The common use of vagrant is by creating a vagrantfile manually https://docs.ansible.com/ansible/2.5/scenario_guides/guide_vagrant.html

• Best practices for modules https://docs.ansible.com/ansible/latest/dev_guide/developing_modules.html

Found here: https://galaxy.ansible.com/search?deprecated=false&keywords=vagrant&order_by=-relevance&page_size=100&page=1

• https://github.com/amtega/ansible_role_vagrant_provisioner
• https://github.com/teampants/ansible-vagrant

Integration test for destroy

Github workflow for sanity tests

Rename vm_name parameter as names and turn it to a list

Investigate workflow with ChoasToolkit

https://chaostoolkit.org/reference/usage/discover/
https://github.com/chaostoolkit/chaostoolkit

Replace all include_tasks by import_tasks in tests

Implement resume

Remove count argument to simplify code

• Remove count arg and related code
• Remove related tests and replace them with https://docs.ansible.com/ansible/latest/collections/ansible/builtin/sequence_lookup.html as example
• Investigate adding support of regexp patterns for other states than up/running

Investigate integration of vagrant-groups.yml (Ansible groups)

Simply adding a groups in ansible entry?

Common usage of ansible roles, ansible groups with vagrant and kube: https://devopssec.fr/article/creer-cluster-kubernetes-multi-noeud-vagrant-ansible

Investigate / implement? support of Docker as a provider

Vagrant supports Docker as a provider which seems to be really interesting for performances when no docker in docker is required.

Acceptation:

• Be able to choose and configure Docker as provider
• Have a CI environment for Docker
• Have the required commands to be able ton control Docker easily: Already done, vagrant ssh, halt and destroy would work

Todo:

• List Docker options as a provider and define a clean API to configure it
  => https://www.vagrantup.com/docs/providers/docker/configuration
  => No box required for docker, what about git_repo, image, build_dir?
• Investigate how much dedicated Vagrant commands to Docker are required to manage it
  => https://dev.to/mattdark/using-docker-as-provider-for-vagrant-10me
• Implement docker case in the Vagrantfile and test it
• Test Docker integration in CI workflows, add a dedicated badge

Integration tests in github's workflow

Integration tests of idempotency for vagrant up && state running

Fix ansible provisionning and test it

Implement tests for failing add_host and delegate_to in roles / ansible-test

• Add failing add_host / delegate_to integration test
• Add test with ssh connection for delegate_to case ansible/ansible#36396 (comment)
• Open a bug on Ansible repo

Can this bug be the source of issues? ansible/ansible#16403

Docs:

• https://docs.ansible.com/ansible/latest/user_guide/connection_details.html

Integration tests with up, halt states instead of commands

Use Vagrant's names : running, poweroff... ?

Integration tests for clear

Polish vagrant.config

• githook to copy Vagrantfile content into y python wrapper
• Move VagrantConfig class to module_utils
• copyright
• examples add / remove / read

Clean vagrant status output

        "status": {
            "frank": [
                [
                    [
                        "frank",
                        "running",
                        "virtualbox"
                    ]
                ],
                [
                    [
                        "frank_inst2",
                        "running",
                        "virtualbox"
                    ]
                ]
            ]
        }

Should be:

        "status": {
            "frank": [
                {
                    "name": "frank",
                    "state": "running",
                    "provider": "virtualbox",
                    "changed": true
                },
                {
                    "name": "frank_inst2",
                    "state": "running",
                    "provider": "virtualbox",
                    "changed": true
                }
            ]
        }

Integration test for status

Coverage for the Vagrantfile

https://github.com/simplecov-ruby/simplecov

Publish on ansible-galaxy

• galaxy.yml

• add examples to modules

• Tell this version is updated to previous contributors 70ca815

• Requirements https://zaiste.net/posts/automatically-install-ansible-galaxy-roles-requirements-yml/

• Respect guidelines https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_checklist.html#developing-modules-checklist

• Contributing to Ansible https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_general.html#contributing-back-to-ansible

Compare with alternative solutions

Acceptance:

• Find workflows with other tools providing the possibility to change the infra during the tests
• Define a workflow for this lib https://www.jeffgeerling.com/blog/2019/how-add-integration-tests-ansible-collection-ansible-test
• Workflow with docker https://jon.sprig.gs/blog/post/2145
  => ansible/molecule#2060
  => William-Yeh/docker-ansible#4
• Know if molecule can change (not recreate but halt, resume, disconnect, add vms between playbook runs)

Sources

• Terraform / Terratest
• Kitchen https://blog.octo.com/vos-tests-dinfrastructure-de-bout-en-bout-avec-kitchen/
• https://medium.com/@chaks/testing-ansible-role-with-molecule-docker-testinfra-goss-part-2-b5138eff59ab
• Molecule https://www.ansible.com/infrastructure-testing-with-molecule
  https://blog.florianlopes.io/testing-ansible-roles-and-playbooks-with-molecule/
  https://medium.com/swlh/testing-ansible-roles-for-multiple-hosts-or-clusters-with-molecule-e4e67a2d0d83
  Good workflow https://www.ansible.com/blog/developing-and-testing-ansible-roles-with-molecule-and-podman-part-2
• https://github.com/chrismeyersfsu/provision_docker
  https://www.ansible.com/blog/five-questions-testing-ansible-playbooks-roles
• Other backends than Vagrant https://github.com/ansible/test-playbooks/tree/master/cloud_modules
• Differences between provisioning, orchestration, configuration and deployment http://codefol.io/posts/deployment-versus-provisioning-versus-orchestration/
• Vagrant workflow using Ansible provisioning https://devopssec.fr/article/creer-cluster-kubernetes-multi-noeud-vagrant-ansible
  => Cons: Provisioning playbook is run once a node so gathering facts accross multiple node or using run_once has no sense here
• Vagrant cheetsheat https://gist.github.com/wpscholar/a49594e2e2b918f4d0c4

Do not call status twice when not required

Replace

        if status_before['state'] != 'saved':
            self.vg.suspend(
                vm_name=name
            )

        status_after = self.raw_statuses(name, must_be_present=True)[name]
        if status_before['state'] != status_after['state']:
            changed = True

by

        if status_before['state'] != 'saved':
            self.vg.suspend(
                vm_name=name
            )
            status_after = self.raw_statuses(name, must_be_present=True)[name]
            if status_before['state'] != status_after['state']:
                changed = True
        else:
            status_after = status_before

Add a `provider` option to vagrant.config

Acceptation

• Be able to choose your provider
• Be able to configure it
• Tests
• Badge by provider

Todo

• List providers et and available options / urls
  => virtualbox, docker, libvirt
• Choose which options should be made common / a way to let the liberty to set the others for the user
  => cpus, memory, box / image?, synced folders
  => provider, provider_options
• Implement and test Virtualbox
• Implement and test Libvirt
• Implement and test docker? #45
• Allow other providers with provider_options_inline
• Check if vagrant validate is enough to test configuration from yaml or if it just checks the syntax
• Patch the workflows for CI:
  • Add Ubuntu / Libvirt && rename Integration Tests for MacOS / Virtualbox
  • Run dedicated tests for each platform: failing libvirt on MacOS and failing virtualbox on Ubuntu
• Update the docs
  • Table for badges
  • Working MacOS/Virtualbox provider
  • Failing MacOS/Libvirt provider
  • Working MacOS/Docker provider
  • Examples for each coverage
• Clean usunsed vars in CI
• Fix coverage in CI

Investigate integration with amtega roles as they handle concurrency / lock ...

https://github.com/amtega/ansible_role_vagrant_provisioner

Support all commands with empty "name" or "names"

Currently I chose to made the "name" parameter required. Matching vagrant commands behavior would required to support it which requires a clean code for manipulating many machines:
TODO:

• List commands where it could have sens (Vagrant doesn't support "vagrant port" with no name specified for example)
• Define outputs having the same philosophy as Ansible loop results
• Factorize calls in the Vagrant adapter
• Make the name parameter optional when possible
• Update the tests
• Update the docs

NOT TO DO: These features depend on the current one

• Async subcalls to make vagrant use faster
• Implement names parameter