nickjj/ansigenome

A tool to help you gather information and manage your Ansible roles.

ansigenome's Issues

Provide a list of licenses to choose from

Picking licenses and providing URLs for them is hard.

Show an user a list of known popular licenses - MIT, BSD, Apache 2.0, GPLv2, GPLv3, other (to specify name and URL). If a user selects one of the known licenses, configure it with an URL to http://tldrlegal.com/.

Breaks with inline vault-encrypted variables

When running ansigenome export on my project I get:

 contains 1 or more syntax errors:
could not determine a constructor for the tag '!vault'
  in "<string>", line 15, column 22:
    foo: !vault |

ansigenome needs to tell PyYAML how to load them, like https://github.com/ansible/ansible/blob/e241e15899d864fdad67e198c0bce3457622a01c/lib/ansible/parsing/yaml/objects.py#L73

location / config for custom meta templates

It's mentioned in the readme that ansigenome supports custom templates for ./meta/main.yml. However I have failed to find a configuration option to plug-in a custom meta files path.

There used to be a pluggable template till v0.2.0
https://pypi.org/project/ansigenome/0.2.0/

Python 2.6 support

Is there support for Python2.6 planned? I want to use ansigenome on RHEL6 and Python 2.6 is the default there.

Currently, it crashes with the following exception, as Python 2.6 doesn't support dict comprehensions (I guess).

Traceback (most recent call last):
  File "/usr/bin/ansigenome", line 5, in <module>
    pkg_resources.run_script('ansigenome==0.5.1', 'ansigenome')
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 461, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 1194, in run_script
    execfile(script_filename, namespace, namespace)
  File "/usr/lib/python2.6/site-packages/ansigenome-0.5.1-py2.6.egg/EGG-INFO/scripts/ansigenome", line 8, in <module>
    import ansigenome.constants as c
  File "/usr/lib/python2.6/site-packages/ansigenome-0.5.1-py2.6.egg/ansigenome/constants.py", line 4, in <module>
    import utils
  File "/usr/lib/python2.6/site-packages/ansigenome-0.5.1-py2.6.egg/ansigenome/utils.py", line 128
    return {v[key]: v for v in items}.values()
                        ^
SyntaxError: invalid syntax

License type out of range when first time setup is ran

Pick a license type by choosing a number:

  1. MIT
  2. GPLv2
  3. GPLv3
  5. LGPL
  6. Apache-2.0
  7. BSDv2
  8. BSDv3
  9. Other

 []: 9

Traceback (most recent call last):
  File "/usr/local/bin/ansigenome", line 6, in <module>
    exec(compile(open(__file__).read(), __file__, 'exec'))
  File "/Users/kgarland/Code/ansigenome/bin/ansigenome", line 326, in <module>
    main()
  File "/Users/kgarland/Code/ansigenome/bin/ansigenome", line 309, in main
    (config, config_path) = load_config()
  File "/Users/kgarland/Code/ansigenome/bin/ansigenome", line 248, in load_config
    Config([], {}, {})
  File "/Users/kgarland/Code/ansigenome/ansigenome/config.py", line 28, in __init__
    out_config = self.ask_questions()
  File "/Users/kgarland/Code/ansigenome/ansigenome/config.py", line 77, in ask_questions
    out_config["license_type"] = c.LICENSE_TYPES[answer][0]
IndexError: list index out of range

Add ability to ignore role's subfolders

Hi,

I'm currently using Molecule. It creates a molecule sub-folder in each role, in which we can find some playbooks unrelated to the functioning of the role. Ansigenome scans this directory and finds some variables that are listed as "internal variables".

Would it be possible to add the ability to ignore some role's sub-folders to Ansigenome? It would allow me to ignore the molecule sub-folder and prevent useless variables to be listed in the generated README.

Regards

Ansible vs non-ansible machine(s) graph

How hard would it be to bend this fantastic tool into a dynamic machine graph (via gather_facts)?

My usecase would be to have ansigenome assess the "ansible coverage" over a bunch of machines. That is, how many machines have been "ansiblized" using roles vs those that were installed manually (legacy manual installations).

Like regular developers do with test coverage.

question: had anyone automated ansigenome?

Hi, we would like ansignenome to automatically be called when we commit for example.
Are there any automation already done? Thank you!

86a62d40adac956784510ec11fe8ee197a9f05c8 broke `role.galaxy_name`

I already mentioned this in #47 (comment)
86a62d4 broke role.galaxy_name and with this also the links based on it.

I did not look more closely what the patch actually does.

Here is a mini working example using this template:

{{ role.galaxy_name | replace('-', '--')}}
{{ role.name | replace('-', '--')}}

And this role as an example: https://github.com/debops-contrib/ansible-apparmor

With 86a62d4, this generates the following:

debops--contrib.debops--contrib.apparmor
debops--contrib.apparmor

And with eb85483 (one commit before), it generates this:

debops--contrib.apparmor
apparmor

cc: @jimbocoder

PS: Might come in handy, I wrote a little wrapper script for ansigenome gendoc to handle multiple environments: https://github.com/ypid/ypid-ansible-common/tree/master/template_READMEs

Link role-dependencies to github repo

Example: https://github.com/debops/ansible-apt#role-dependencies

debops.etc_services could link to https://github.com/debops/ansible-etc_services

ansigenome not installing properly

Hi Nick,

Thanks for putting together ansigenome, which I found at https://github.com/nickjj/ansigenome. It seems to be what I’m looking for, i.e. a tool to graphically show me what’s in my playbooks. However, I’m having issues getting it to install, and am wondering if you have thoughts on what’s wrong. Note that setuptools is already installed, so not sure why it’s giving that message.

On MacOS 10.11.6, python 3.6.5:

$ pip3 install ansigenome
Collecting ansigenome
Using cached https://files.pythonhosted.org/packages/68/82/f41ce20b98bd91e64ea49aee3660516c4d40f0a09ffa2c2932d4552c44c8/ansigenome-0.6.0.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "/private/var/folders/bh/7zd_g3hd4b7bkv2hf2_cnmwsf0zxn9/T/pip-install-9h5i5f1r/ansigenome/setup.py", line 8
print "Ansigenome needs setuptools in order to build. " +
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Ansigenome needs setuptools in order to build. " + )?

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/bh/7zd_g3hd4b7bkv2hf2_cnmwsf0zxn9/T/pip-install-9h5i5f1r/ansigenome/

On RHEL 6.7, python 3.4.2:

$ dzdo pip3.4 install ansigenome
Downloading/unpacking ansigenome
Downloading ansigenome-0.6.0.tar.gz (863kB): 863kB downloaded
Running setup.py (path:/tmp/pip_build_root/ansigenome/setup.py) egg_info for package ansigenome
Traceback (most recent call last):
File "", line 17, in
File "/tmp/pip_build_root/ansigenome/setup.py", line 8
print "Ansigenome needs setuptools in order to build. " +
^
SyntaxError: Missing parentheses in call to 'print'
Complete output from command python setup.py egg_info:
Traceback (most recent call last):

File "", line 17, in

File "/tmp/pip_build_root/ansigenome/setup.py", line 8

print "Ansigenome needs setuptools in order to build. " + \

                                                      ^

SyntaxError: Missing parentheses in call to 'print'

---

Cleaning up...

On MacOS, installing from source:

$ sudo python3 setup.py develop
Password:
File "setup.py", line 8
print "Ansigenome needs setuptools in order to build. " +
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Ansigenome needs setuptools in order to build. " + )?

$ pip3 install setuptools
Requirement already satisfied: setuptools in /usr/local/lib/python3.6/site-packages (39.0.1)

TypeError exception when using export command

I just installed ansigenome with pip install. Don't know if use it correctly, but it crash when I try to export a graph.

xxxx@xxxx:~/ansible/roles$ ansigenome export -o graph.png
Traceback (most recent call last):
File "/usr/local/bin/ansigenome", line 326, in
main()
File "/usr/local/bin/ansigenome", line 322, in main
fn(args, options, reloaded_config, parser)
File "/usr/local/bin/ansigenome", line 177, in execute_export
Scan(args, options, config, export=True)
File "/usr/local/lib/python2.7/dist-packages/ansigenome/scan.py", line 83, in init
self.scan_roles()
File "/usr/local/lib/python2.7/dist-packages/ansigenome/scan.py", line 117, in scan_roles
self.report["roles"][key] = self.report_role(key)
File "/usr/local/lib/python2.7/dist-packages/ansigenome/scan.py", line 190, in report_role
"meta": self.gather_meta(),
File "/usr/local/lib/python2.7/dist-packages/ansigenome/scan.py", line 213, in gather_meta
dep_list.append(dependency["role"])
TypeError: string indices must be integers, not str

Support Ansible custom filters

As this tool is tightly related to Ansible and folks using Ansigenome are probably very familiar with Ansible’s custom Jinja filters I think it would come in handy to support them. From what I saw when I last checked, Ansible has all the filters defined in one Python module. I played with that for a different project but Ansible has quite some dependencies. So not too easy unfortunately.

parser fails due to "Too many levels of symbolic links"

I attempted to run this on my repository to get a feel for what our graph looks like and it failed with the error message:

Traceback (most recent call last):
  File "./ansigenome", line 326, in <module>
    main()
  File "./ansigenome", line 322, in main
    fn(args, options, reloaded_config, parser)
  File "./ansigenome", line 153, in execute_scan
    Scan(args, options, config)
  File "/home/kwoodson/git/ansigenome/ansigenome/scan.py", line 83, in __init__
    self.scan_roles()
  File "/home/kwoodson/git/ansigenome/ansigenome/scan.py", line 117, in scan_roles
    self.report["roles"][key] = self.report_role(key)
  File "/home/kwoodson/git/ansigenome/ansigenome/scan.py", line 185, in report_role
    "total_lines": self.gather_lines(),
  File "/home/kwoodson/git/ansigenome/ansigenome/scan.py", line 315, in gather_lines
    with open(full_path, "r") as f:
IOError: [Errno 40] Too many levels of symbolic links: '/home/kwoodson/git/openshift-ansible/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks'

It looks like it started doing a recursive decent and was broken by the symlinks.

Possible fixes:

• Keep a hash of the paths that have been covered as to not descend more than once.
• Add flag --no-symlinks

If I can find some time I'll look at submitting a patch.

Python 3 incompatibility

Please fix/handle Python 3 incompatibility.
When I tried to install, I got this:

halis@eleanor ~ (master) $ pip install ansigenome
Collecting ansigenome
  Using cached ansigenome-0.5.6.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 20, in <module>
      File "/tmp/pip-build-xng2b7ao/ansigenome/setup.py", line 8
        print "Ansigenome needs setuptools in order to build. " + \
                                                              ^
    SyntaxError: Missing parentheses in call to 'print'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-xng2b7ao/ansigenome

Some version info:

halis@eleanor ~ (master) $ pip -V
pip 7.1.2 from /usr/lib64/python3.4/site-packages (python 3.4)

halis@eleanor ~ (master) $ python
Python 3.4.3 (default, Nov 11 2015, 11:23:26) 
[GCC 4.9.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

ansigenome 0.6.0 ignores the ansigenome.yml

Hi, very useful tool,
we finally managed to have a PoC of ansigenome working on my local mac, with ansigenome 0.5.6 from ypid. ANd it works ok with the meta/ansigenome.yml a custom template in md format.

The same role and the same ansigenome.conf used on a Linux host with ansigenome 0.6.0 installed by pip ignore anything in the ansigenome.yml.

I can provide logs if instructed how to.
Thanks!

ansigenome config fails with non ASCII characters

I've installed ansigenome in virtual env running Python 2.7.3. After running ansigenome config I got following traceback:

Traceback (most recent call last):
  File "/home/scibi/.virtualenvs/ansible_xxxxx/bin/ansigenome", line 326, in <module>
    main()
  File "/home/scibi/.virtualenvs/ansible_xxxxx/bin/ansigenome", line 309, in main
    (config, config_path) = load_config()
  File "/home/scibi/.virtualenvs/ansible_xxxxx/bin/ansigenome", line 248, in load_config
    Config([], {}, {})
  File "/home/scibi/.virtualenvs/ansible_xxxxx/local/lib/python2.7/site-packages/ansigenome/config.py", line 29, in __init__
    utils.write_config(self.config_path, out_config)
  File "/home/scibi/.virtualenvs/ansible_xxxxx/local/lib/python2.7/site-packages/ansigenome/utils.py", line 398, in write_config
    string_to_file(path, config_as_string)
  File "/home/scibi/.virtualenvs/ansible_xxxxx/local/lib/python2.7/site-packages/ansigenome/utils.py", line 50, in string_to_file
    file.write(input)
  File "/home/scibi/.virtualenvs/ansible_xxxxx/lib/python2.7/codecs.py", line 691, in write
    return self.writer.write(data)
  File "/home/scibi/.virtualenvs/ansible_xxxxx/lib/python2.7/codecs.py", line 351, in write
    data, consumed = self.encode(object, self.errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 81: ordinal not in range(128)

The cause of that was letter 'Ś' in my name. Replacing it with 'S' "solved" the issue.

Support all-contained-in-role-repo mode.

It would be nice to include all the required configuration and templates into the role itself so that everyone can regenerate the readme.

Related to: weareinteractive/gem-ansible-role#2

variables graph

Hi,
would it be possible to track variables visually in a similar way?
we have 3 places from where a variable can be changed: role, group_vars and host_vars
Thanks!

allow include templates from url

Hi,
this very nice tool we are adopting for some time ad try to fully automate the generation based on commits (not only precommit) (gitlab runner, jenkins job) , now gets more attention, in this phase we were wondering if the templates can be included from an online repo?
Thanks!

can we have multiple authors?

now that I have somehow got it working, thanks ppl!
can we have multiple authors?

cuts the first letters from beginning of the name in gendoc

we have roles that start with xx-role-name
when we use gendoc the name of the role in the readme is only -role-name

template not found error

Hi, we use the latest ansigenome 0.6.0 at this point and we are getting into the habit of using our own templates.
The template is there in place, but the ansigenome gendoc cannot find it.

Traceback (most recent call last):
File "/usr/bin/ansigenome", line 326, in
main()
File "/usr/bin/ansigenome", line 322, in main
fn(args, options, reloaded_config, parser)
File "/usr/bin/ansigenome", line 161, in execute_gendoc
Scan(args, options, config, gendoc=True)
File "/usr/lib/python2.7/site-packages/ansigenome/scan.py", line 83, in init
self.scan_roles()
File "/usr/lib/python2.7/site-packages/ansigenome/scan.py", line 125, in scan_roles
self.write_readme(key)
File "/usr/lib/python2.7/site-packages/ansigenome/scan.py", line 435, in write_readme
j2_out = self.readme_template.render(self.readme_template_vars)
File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "", line 1, in top-level template code
File "/usr/lib/python2.7/site-packages/jinja2/loaders.py", line 286, in get_source
raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: redacted_path/README.md.j2

Thank you!

Warning: <stdin>: syntax error in line 1 near 'No'

If I use the command line to export roles dependencies into a png file, the bellow error is displayed:

$ ansigenome export ./git_repos/devops/roles/ -o ./git_repos/devops/roles/role_schema.png
Warning: <stdin>: syntax error in line 1 near 'No'

But it works if I go to role directory and execute the command like as bellow:

$ cd ./git_repos/devops/roles/ && ansigenome export -o role_schema.png

check template for defaults

I noticed when running this it did not report any defaults for my role however there are a number of defaults outside of the standard tasks folder

Ansible 2.0 not supported

When trying to use Ansigenome with Ansible 2.0 i get this error:
ERROR! 'ansigenome_info' is not a valid attribute for a RoleMetadata

I checked the MetaData class in ansible 2.0 and it seems to be more strict than earlier.
https://github.com/ansible/ansible/blob/devel/lib/ansible/playbook/role/metadata.py

Do you have any idea how this could be solved? I volunteer to help fix it, but I'm unsure how to proceed.

The reqs command could read in versions from a VERSION file

Use case:

You have 80 roles and want to keep track of the versions. Versioning the requirements file by hand is tedious.

Solution:

Each role has a VERSION file which is a git tag. When you run the reqs command with -v|--version it will read in that version and write it out to the requirements file.

Roles that are unversioned by either an empty VERSION file or no VERSION file will be written out as master for the version.

Scan does not detect markdown readmes

Currently, running ansigenome scan still marks roles with markdown readmes (named README.md) as missing readmes, even if these readmes are generated by ansigenome itself.

Store `ansigenome_info` dict in `meta/ansigenome.yml` or in `meta/main.yml`?

@drybjed "Move[d] [all] ansigenome metadata to a separate file " some time ago. Reading thought the Readme, one gets the idea to put everything into meta/main.yml. As I know @drybjed by now, there is usually a good reason to do something a certain way. What is the best-practice here and why?

Roles don't connect to one another

Hi,

The picture will likely speak for itself:

The roles are not connected to one another and instead of having a graph where everything starts from the comb/ role and then connects to one another, ansigenome seems to miss that the roles should indeed all be connected.

Could you point me into the direction to fix it, please?

Exported png doesn't seem valid

Are there any requirements for what ansigenome expects when does a scan? I ask because I ran the following:

ansigenome scan directory-here/
ansigenome export -o /tmp/out.png

where directory-here contains my yaml playbooks and other include files that those yaml files refer to.

However, when I open /tmp/out.png, it just shows what seem to be a series of unconnected lines, laid out sequentially in a row. When I zoom in, it doesn't show me much; I can't make out anything for each line. I expected to see a graph of relationships, much like is shown in the README file on this github site.

I thought the reason I'm not seeing anything may have been because I have yaml files that are essentially just dictionary definitions or other files that are referred to, i.e. included, in the actual playbooks that have roles and hosts defined, i.e. that have a roles: and a hosts: section in the .yml file. So I copied to a subdirectory, and ran a scan on that subdirectory, but that doesn't even find any roles.

What do I need to do to see a graph of relationships like shown in the README?

Ansigenome missed one instance of a internal variable.

Hi

I just noticed that Ansigenome missed packages__whitelist_by_vars_combined which is used in the same set_fact tasks as packages__blacklist_by_vars_combined (is which is listed as expected). Without looking into the code I would expected that Ansigenome only finds the first occurrence of variables in set_fact tasks?

ypid/ansible-packages@b609f76

https://github.com/ypid/ansible-packages/blob/b609f7683f2adf04890d550b772f6f5ac19542ed/tasks/include_item_vars.yml#L7-L13

Make it possible to use $HOME in options_readme_template

Hi

Great project. Is it possible to specify the path to the readme template as follows, where ~ would be expanded as $HOME?

options_readme_template: ~/.ansigenome/ypid.README.rst.j2

Currently, ansigenome says:

The following template could not be found:
~/.ansigenome/README.rst.j2

or

The following template could not be found:
$HOME/.ansigenome/README.rst.j2

Feature Request - create "files"-folder on ansigenome init

Would be nice to also create the "files"-folder when using ´ansigenome init´.
ansible-galaxy init also creates this directory.

errors when handling missing meta/main.yml

roles without meta/main.yml files seem to produce spurious dependencies. I think this is due to a mistake in scan.py around line 202: