lae/ansible-role-netbox

Cross-platform Ansible role for deploying NetBox, a DCIM/IPAM tool, in a production environment.

ansible-role-netbox's Issues

Netbox 2.7 Remote File Systems

Netbox 2.7 introduced the ability to utilize Django storages for the file system.
https://django-storages.readthedocs.io/en/stable/
https://netbox.readthedocs.io/en/stable/configuration/optional-settings/#storage_backend

I think the best way to handle this for the role is to keep the critical files local, and only really move the scripts, media, and reports off to the remote storage provider.

Additionally due to the number of variables in utilizing these providers I think we should probably keep the roles implementation of this as simple as possible by allowing the user to declare their json in a separate file, and having the role inject the end-users json straight into the config.py.

I wrote some code demonstrating my concept today, but due to not being able to utilize all the possible providers I am not sure how we would even go about testing all the possibilities.

Thoughts?

SECRET_KEY - autogeneration broken

From reading the source, intended behaviour is, if SECRET_KEY is not in netbox_config parameter, it will be generated. But

      - include_role:
          name: "netbox"
        vars:
          netbox_stable: true
          netbox_database_host: localhost
          netbox_database_user: "netbox"
          netbox_database_password: "netbox"
          netbox_superuser_password: "netbox"
          netbox_socket: "0.0.0.0:9000"
          netbox_load_initial_data: false
          netbox_application_log: "file:{{ netbox_shared_path }}/application.log"
          netbox_requests_log: "file:{{ netbox_shared_path }}/requests.log"
          netbox_scripts: []
          netbox_config:
             ALLOWED_HOSTS:
               - 0.0.0.0
             MEDIA_ROOT: "{{ netbox_shared_path }}/media"
             REPORTS_ROOT: "{{ netbox_shared_path }}/reports"
             SCRIPTS_ROOT: "{{ netbox_shared_path }}/scripts"
          #   SECRET_KEY: "" # auto-generated

Is ailing with:

    qemu: TASK [_netbox : Run database migrations for NetBox] ****************************
    qemu: fatal: [localhost]: FAILED! => {"changed": false, "cmd": "./manage.py migrate --noinput", "msg": "\n:stderr: Traceback (most recent call last):\n  File \"./manage.py\", line 10, in <module>\n    execute_from_command_line(sys.argv)\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/core/management/__init__.py\", line 381, in execute_from_command_line\n    utility.execute()\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/core/management/__init__.py\", line 375, in execute\n    self.fetch_command(subcommand).run_from_argv(self.argv)\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/core/management/base.py\", line 323, in run_from_argv\n    self.execute(*args, **cmd_options)\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/core/management/base.py\", line 361, in execute\n    self.check()\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/core/management/base.py\", line 390, in check\n    include_deployment_checks=include_deployment_checks,\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/core/management/commands/migrate.py\", line 64, in _run_checks\n    issues = run_checks(tags=[Tags.database])\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/core/checks/registry.py\", line 72, in run_checks\n    new_errors = check(app_configs=app_configs)\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/core/checks/database.py\", line 9, in check_database_backends\n    for conn in connections.all():\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/db/utils.py\", line 216, in all\n    return [self[alias] for alias in self]\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/db/utils.py\", line 213, in __iter__\n    return iter(self.databases)\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/utils/functional.py\", line 80, in __get__\n    res = instance.__dict__[self.name] = self.func(instance)\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/db/utils.py\", line 147, in databases\n    self._databases = settings.DATABASES\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/conf/__init__.py\", line 79, in __getattr__\n    self._setup(name)\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/conf/__init__.py\", line 66, in _setup\n    self._wrapped = Settings(settings_module)\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/conf/__init__.py\", line 157, in __init__\n    mod = importlib.import_module(self.SETTINGS_MODULE)\n  File \"/usr/lib/python3.7/importlib/__init__.py\", line 127, in import_module\n    return _bootstrap._gcd_import(name[level:], package, level)\n  File \"<frozen importlib._bootstrap>\", line 1006, in _gcd_import\n  File \"<frozen importlib._bootstrap>\", line 983, in _find_and_load\n  File \"<frozen importlib._bootstrap>\", line 967, in _find_and_load_unlocked\n  File \"<frozen importlib._bootstrap>\", line 677, in _load_unlocked\n  File \"<frozen importlib._bootstrap_external>\", line 728, in exec_module\n  File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n  File \"/srv/netbox/releases/netbox-2.6.7/netbox/netbox/settings.py\", line 46, in <module>\n    \"Required parameter {} is missing from configuration.py.\".format(parameter)\ndjango.core.exceptions.ImproperlyConfigured: Required parameter SECRET_KEY is missing from configuration.py.\n", "path": "/srv/netbox/current/venv-py3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "state": "absent", "syspath": ["/tmp/ansible_django_manage_payload_79_dlb_l/ansible_django_manage_payload.zip", "/usr/lib/python37.zip", "/usr/lib/python3.7", "/usr/lib/python3.7/lib-dynload", "/usr/local/lib/python3.7/dist-packages", "/usr/lib/python3/dist-packages"]}

Fancied the feature, but have no time to cure myself. Should be light one though.

Permission error with version v1.0.0rc1

Hello,

I installed netbox with the version 0.9.x of this playbook.
I would like to switch to new version (v1.0.0rc1) of this playbook (to install new pip dependencies) without upgrading netbox.

When I execute the playbook again, I have this error:

failed: [xxxxxx] (item=dns) => changed=false 
  ansible_loop_var: item
  attempts: 2
  cmd:
  - /opt/netbox/current/venv-py3/bin/pip3
  - install
  - dns
  item: dns
  msg: |-
    stdout: Collecting dns
  
    :stderr: The directory '/opt/netbox/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
    The directory '/opt/netbox/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
      Could not find a version that satisfies the requirement dns (from versions: )
    No matching distribution found for dns

However:

root@netbox:~# ll /opt/netbox/.cache
ls: cannot access '/opt/netbox/.cache': No such file or directory

Thanks.

Move upload directories to shared directory

{{ netbox_current_path }}/netbox/media/image-attachments seems to be used for image uploads. This should be a symlink to a directory in {{ netbox_shared_path }}, probably the media directory entirely.

Currently, on upgrades, that directory'll unexpectedly be empty should users have uploaded images.

I don't think there are any other upload directories, but there could be - so skim through the NetBox source code for them.

uwsgi.ini config customization

Hi,

we are experiencing uWSGI errors when using apache proxy in front of it, like below:
[WARNING] unable to add HTTP_CONNECTION=Keep-Alive to uwsgi packet, consider increasing buffer size
And 502 error in apache proxy.
It happens usually when posting via apache proxy, actions like import vlan list or create new user (in case of ldap auth configured in netbox).
uWSGI mention in it's docs that it's default buffer size is a bit small:

By default uWSGI allocates a very small buffer (4096 bytes) for the headers of each request. If you start receiving “invalid request block size” in your logs, it could mean you need a bigger buffer. Increase it (up to 65535) with the buffer-size option.

We would like a possibility to pass the buffer size setting as variable when running your netbox role, like below config line:
buffer-size=65535

Centos7/pip3/uWSGI (2.0.18)/netbox-2.5.12

Thank you

Drop Python 2 support

I don't think there are any users explicitly using this role to deploy NetBox with Python 2 and NetBox will be dropping support for it later this year (netbox-community/netbox#2000).

This isn't too difficult to do (if anyone wants to contribute) and would help a lot by cutting build times in half.

NetBox 2.6 support

2.6 looks like it requires Redis setup and configuration, and maybe some other things. I started a branch at https://github.com/lae/ansible-role-netbox/tree/feature/netbox-2.6 but if anyone wants to contribute, please feel free.

Permissions on netbox_home directory

I am not sure if this is a rhel/centos specific issue or if this is happening to everyone, but I noticed while messing around with using nginx as a reverse proxy with wsgi sockets that when netbox_home gets created in the deployment play, since there is no explicit mode setting being passed ansible sets the directory permissions to 0700.

Obviously this annoying with dealing with a second process that needs to read items below that directory like httpd. I've just been changing the permissions on that directory after executing the role in my playbook to 0711, I don't know if it makes sense to roll a change like that into the larger role or not depending on how others are doing things?

Tests failing on Ubuntu Bionic

Tests have been failing since build #688 and it appears to be Ubuntu 18/Bionic only. Deployment does not error out, but the service does not appear to be online.

In 688 it appears to have failed for both stable and git versions - in build #689 a week later, only stable seems to be failing, so maybe there was a commit upstream that fixed it but still isn't in stable? (cursory search for Ubuntu issues on the NetBox issue tracker turned up nothing)

Validate configuration files

I tried to implement this earlier, but was running into an Ansible bug related to validating templates while setting file permissions it looks like.

Example test case: https://travis-ci.org/lae/ansible-role-netbox/builds/242510642

Opened a bug in Ansible about this ansible/ansible#25667

Would probably be worthwhile to come up with a fix that doesn't require the latest Ansible release, though, should that bug get fixed.

SSL connection to Redis

Add option to enable SSL for Redis. This could be achieved by adding line netbox_redis_ssl_enabled: false on defaults/main.yml and modifying the redis section in templates/configuration.py.j2

REDIS = {
    'HOST': '{{ netbox_redis_host }}',
    'PORT': '{{ netbox_redis_port }}',
    'PASSWORD': '{{ netbox_redis_password }}',
    'DATABASE': '{{ netbox_redis_database }}',
    'CACHE_DATABASE': '{{ netbox_redis_cache_database }}',
    'DEFAULT_TIMEOUT': '{{ netbox_redis_default_timeout }}',
    'SSL': '{{ netbox_redis_ssl_enabled }}',
}

I can create PR for this.

Dump database if migrations are required

It would be extremely useful to check if any migrations need to be applied before doing them, and if migrations are required then have the option to perform a dump of the database data.

This can be easily accomplished with
./manage.py showmigrations | grep '\[ \]'

An empty output is returned if no migrations need to be applied. If migrations are needed, then this task could run:

- name: Backup database prior to Netbox upgrade
  postgresql_db:
    name: "{{ netbox_database }}"
    state: dump
    target: "netbox_{{ netbox_database }}_backup.tar.gz"
    target_opts: "-n public"
    login_host: "{{ netbox_database_host }}"
    login_password: "{{ netbox_database_password }}"
    login_user: "{{ netbox_database_user }}"
    port: "{{ netbox_database_port }}"
  run_once: True

Main concerns are that the server needs to have pg_dump installed (part of the postgresql package that this package installs anyway) that matches the major release of the database server. That's why this set of tasks needs to happen after system packages are installed but before the migration step is actually run - landing us right in the middle of this role's workflow.

Django Migration issue on First run

I am running into an issue when the Django migrate task runs. This is for a brand new installation.

Centos 7.4
Python 3.6
Ansible 2.7.5

The gist of the issue has been added here.
https://gist.github.com/hkdanalyser/1851b2b694fd3792d14cf2a900d47314

The config for the ansible role is here
https://gist.github.com/hkdanalyser/700876920c7905854e39bd527b961afc

Any insight would be appreciated.

Thanks !

Provide interface for additional database parameters

The configuration template needs tweaking in some way. Django (and therefore Netbox) has a number of database specific parameters available that aren't currently able to be passed in easily.

The original requirement that brought this idea up was that CONN_MAX_AGE is not set by default in Netbox, but in many cases ought to be. I don't think there's a way to get that parameter passed in currently. Perhaps some sort of for-loop in the Jinja template for any additional keys would suffice. Happy to submit a PR.

Add support for Netbox metrics configurations

When using the built-in metrics for Netbox (based on django-prometheus), there are a couple new configuration items that need to be added across the project.

The systemd service file would need this parameter set (to auto-create the metrics storage folder in /run on boot):

[Service]
{% if netbox_metrics_enabled %}
RuntimeDirectory={{ netbox_metrics_dir }}
{% endif %}

uwsgi.ini - This whole line can be set by the user already using netbox_uwsgi_options

[uwsgi]
env=prometheus_multiproc_dir=/run/{{ netbox_metrics_dir }}

So depending on your preference it may really just be a case of adding the new template snippet to netbox.service.j2

Upgrade of psycopg2 dependency fails

I've been away on leave and when I returned last week I bumped netbox to the latest version, low and behold its failed to deploy psycopg2 python package, where before its been fine.

It appears that pg_config can't be found when pip goes to install psycopg2. Looking back at my previous successful deploys it was using v0.7.6 of this role. changes seem to indicate that psycopg2 was added in 0.7.7

Ive been trying to track down source of this issue and see that pg_config is installed on the target machine and available from command shell, yet doesn't appear to be available to pip3?

TASK [lae.netbox : Install psycopg2/selinux via pip on Red Hat-based distros] ***
[DEPRECATION WARNING]: Invoking "pip" only once while using a loop via 
squash_actions is deprecated. Instead of using a loop to supply multiple items 
and specifying `name: "{{ item }}"`, please use `name: ['psycopg2', 'selinux']`
 and remove the loop. This feature will be removed in version 2.11. Deprecation
 warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
failed: [stg001] (item=['psycopg2', 'selinux']) => {"changed": false, "cmd": ["/usr/local/bin/pip3", "install", "-U", "psycopg2", "selinux"], "item": ["psycopg2", "selinux"], "msg": "stdout: Collecting psycopg2\n  Using cached https://files.pythonhosted.org/packages/23/7e/93c325482c328619870b6cd09370f6dbe1148283daca65115cd63642e60f/psycopg2-2.8.2.tar.gz\n    Complete output from command python setup.py egg_info:\n    running egg_info\n    creating pip-egg-info/psycopg2.egg-info\n    writing pip-egg-info/psycopg2.egg-info/PKG-INFO\n    writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt\n    writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt\n    writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'\n    \n    Error: pg_config executable not found.\n    \n    pg_config is required to build psycopg2 from source.  Please add the directory\n    containing pg_config to the $PATH or specify the full executable path with the\n    option:\n    \n        python setup.py build_ext --pg-config /path/to/pg_config build ...\n    \n    or with the pg_config option in 'setup.cfg'.\n    \n    If you prefer to avoid building psycopg2 from source, please install the PyPI\n    'psycopg2-binary' package instead.\n    \n    For further information please check the 'doc/src/install.rst' file (also at\n    <http://initd.org/psycopg/docs/install.html>).\n    \n    \n    ----------------------------------------\n\n:stderr: Command \"python setup.py egg_info\" failed with error code 1 in /tmp/pip-install-gwcflx76/psycopg2/\n

strange errorcode when migrating netbox database

gist link

Add an option to specify custom python3 requirements for virtualenv

Hello,

Currently I am using netbox reports which are using the DNS python library.
So I need to install this python library directly in the netbox virtualenv.
I saw this variable netbox_pip_constraints which is in a template but there is not task to install this template.
When we will agree with this feature, I am able to implement it if you want.

Regards.

Stable builds failing due to DRF 3.7.0 upgrade

Django REST Framework 3.7.0 removed some code that NetBox 2.1.5 and below depend on. Enforcing <3.7.0 during deployments should fix this.

See: netbox-community/netbox#1563

Does anyone use Redis Sentinel?

Background: netbox-community/netbox#3984

This'll require some mucking about with the Redis connection role variables again, but I'm not sure if there's value in implementing support for this. Does anyone use Sentinel and have a need for being able to configure NetBox/Django accordingly?

CentOS 8 CI tests missing

#90 was merged since I deployed it fine locally, but there should be a deploy test for the weekly CI test.

Right now the test suite doesn't support it yet, so this'll have to wait until it does.

Template standard netbox configuration

Configuration files should be stored in netbox_shared_path and symlinked into the appropriate deployment directory (stable/tarball release or git directory).

File to be templated

Only the relevant fields should be configurable. Some should use already defined role variables.

http://netbox.readthedocs.io/en/stable/installation/netbox/#configuration
http://netbox.readthedocs.io/en/stable/configuration/mandatory-settings/
http://netbox.readthedocs.io/en/stable/configuration/optional-settings/

Ubuntu 20.04 Focal Fossa support missing

I tried to install NetBox on a machine that was upgraded to the latest Ubuntu (LTS) release, and ran into the problem that no OS-specific file such as vars/ubuntu-20.yml was found. So I made one, starting from the ubuntu-18 one. The only modification I made was replacing python3.6 with python3.8, as Focal comes with Python 3.8. With this new file, I was able to provision a(n apparently) working NetBox installation on my Focal box.

I'm happy to submit a PR. Unfortunately I was unable so far to get Molecule to run the tests on my (Mac OS) development machine, even though I think I have installed all prerequisites.

Local testing environment documentation

Need to include instructions for setting up a local dev environment for testing the role locally under Contributing in the README, or maybe just start a separating doc for contributors.

#57 (comment)

Python 3.5 is the minimum supported for Django 2.1

Release notes: https://docs.djangoproject.com/en/2.1/releases/2.1/

NetBox 2.5 depends on Django 2.1 now, which breaks builds for CentOS/RHEL 7 (see #40) and likely also Debian 8.

I think we should still support CentOS 7, but maybe drop support for Debian 8 (since we already have 9).

Since the latest in CentOS 3.4, it looks like it might be necessary to use a third-party repo. Is https://ius.io/ the recommended one?

SVG not showing

Hi everyone,
I've just upgraded my Netbox to 2.8.4 version via lae role, and now seems that SVG images are not working anymore...

Could you please suggest me something?

v0.9.3 doesn't work

v0.9.3 doesn't have the check for if netbox >= 2.7.11 so migration fails.

Deploy netbox via git

Deployment and configuration should follow same pattern as #1, not much should differ, except in the cases where configuration files might change. For release we'll focus on current develop.

• netbox_git_path should store the git repository - unlike the stable scenario (where there would be multiple folders for different releases).
• netbox_current_path should be symlinked to netbox_git_path.
• Changes to git repository should trigger a restart/reload.

This should support users who want to use their own forks as well (hence the need for the netbox_git_uri variable, which you can already see in defaults).

http://netbox.readthedocs.io/en/stable/installation/netbox/#option-b-clone-the-git-repository

Add support for database OPTIONS dictionary

Similar to the discussion in #64 (and realizing I put my own foot in my mouth because here we are) there are some use cases where it's up to the user's environment. In my particular case, I need to set the sslmode parameter to something other than the default.

I think the simplest option would be to introduce a new var like:

netbox_database_options that would be a dictionary used similar to netbox_uwsgi_options and netbox_config are already handled today.

For cases like isolation levels we could add a note that the real numerical value must be used instead of the constant psycopg2.extensions.ISOLATION_LEVEL_SERIALIZABLE as specified in the example. It seems like the other possible options for PostgreSQL are either strings or numbers so the existing logic used for netbox_config should work fine here as well.

A quick pass at the template logic

DATABASE = {
    'NAME': '{{ netbox_database }}',
    'USER': '{{ netbox_database_user }}',
{% if netbox_database_host is defined %}
    'PASSWORD': '{{ netbox_database_password }}',
    'HOST':  '{{ netbox_database_host }}',
    'PORT': '{{ netbox_database_port }}',
{% else %}
    {% if netbox_database_password is defined %}
    'PASSWORD': '{{ netbox_database_password }}',
    {% endif %}
    'HOST': '{{ netbox_database_socket }}',
{% endif %}
    'CONN_MAX_AGE': {{ netbox_database_conn_age }},
{% if netbox_database_options is defined %}\
    'OPTIONS': {
        {% for setting, value in netbox_database_options.items() %}
        {% if value is string or value is number %}
        "{{ setting }}": {{ value | to_nice_json }},
        {% else %}
        "{{ setting }}": json.loads(r'''{{ value | to_json }}'''),
        {% endif %}
        {% endfor %}
    }
{% endif %}
}

Prune old deployment directories

For every release, this role creates a new directory to deploy NetBox into, as well as a virtualenv for that particular release. This allows us to do naive downgrades (it's fine if there aren't any database migrations) to an older version when necessary, but after some time leads to a lot of old release directories:

:~$ ls -l /srv/netbox/releases/
total 48
drwxr-xr-x 8 netbox netbox 4096 Jun 23 01:48 git
drwxr-xr-x 7 netbox netbox 4096 Aug 29 01:18 git-deploy
drwxr-xr-x 7 netbox netbox 4096 Aug 29 01:17 git-repo
drwxr-xr-x 6 netbox netbox 4096 Jul 10 22:36 git-static
drwxr-xr-x 7 netbox netbox 4096 Jul 17 15:34 netbox-2.0.10
drwxr-xr-x 7 netbox netbox 4096 Jun 19 20:43 netbox-2.0.7
drwxr-xr-x 7 netbox netbox 4096 Jul 11 18:39 netbox-2.0.9
drwxr-xr-x 7 netbox netbox 4096 Jul 25 17:12 netbox-2.1.0
drwxr-xr-x 7 netbox netbox 4096 Aug  2 21:26 netbox-2.1.1
drwxr-xr-x 7 netbox netbox 4096 Aug 29 00:04 netbox-2.1.3
drwxr-xr-x 7 netbox netbox 4096 Aug 31 20:43 netbox-2.1.4
drwxr-xr-x 7 netbox netbox 4096 Oct  3 00:44 netbox-2.1.5

This role should probably have a role variable to specify when to prune old releases, e.g. prune all releases except the last 5. There should be a task to check for prunable releases and remove them entirely.

RHEL 8 / Centos 8 Support

Any chance we can get this role updated for RHEL 8 / Centos 8 ?
Busy looking at migrating an old NetBox Centos 6 install to Centos 8

Problem with molecule test

Error when run molecule test

lae.netbox❯ molecule test
---
dependency:
  name: galaxy
driver:
  name: vagrant
  provider:
    name: virtualbox
lint:
  enabled: false
  name: yamllint
platforms:
  - box: generic/debian9
    name: debian-9
provisioner:
  lint:
    enabled: false
    name: ansible-lint
  log: true
  name: ansible
scenario:
  name: default
verifier:
  lint:
    name: flake8
  name: testinfra

ERROR: Failed to pre-validate.

{'lint': ['must be of string type']}

Forcing the role to use Python 3 breaks deployment on CentOS/RHEL 7

As shown in ansible/ansible#53929, by forcing Ansible to use Python 3 as the interpreter causes the role to break at any template task.

Is there a reason using Python 3 for Ansible is enforced?

Add HTTPS and letsencrypt compatibility.

Add example playbook with Redis/webhooks enabled

Readme currently suggests a Redis role, but doesn't provide some quick pointers to get off the ground with that role. An example playbook should be included in examples/ demonstrating how to use it, briefly.

Support LDAP authentication

• Provide a flag to toggle LDAP authentication.
• Perform checks that appropriate variables are defined
• Install correct apt/yum dependencies (use files in vars/)
• template out ldap_config.py
• ldap_config.py should be absent if LDAP authentication is disabled

http://netbox.readthedocs.io/en/stable/installation/ldap/

Install/configure gunicorn and service

Should be installed within the virtualenv for the active deployment. Template out the configuration, store in netbox_shared_path.

Preferably, to be less intrusive, this should also be configured as a service using systemd unit files/sysvinit scripts (check using Ansible facts) rather than supervisord.

Once this is done, a test should be added to .travis.yml to check that the gunicorn server is running and returns an HTTP 200. See lae.proxmox for an example.

http://netbox.readthedocs.io/en/stable/installation/web-server/#gunicorn-installation

Add netbox_reports variable for uploading of reports

There's currently functionality to upload scripts to the SCRIPTS_ROOT directory. I'd like to be able to do the same thing with reports.

From the readme:

---

netbox_scripts: []

Scripts to upload for use within NetBox. This should be a list of dictionaries with a src attribute, specifying the local path to the script, and a name attribute, specifying the script/module name. For example:

netbox_scripts:
  - src: netbox_scripts/migrate_application.py
    name: migrate_application

---

Method for skipping EPEL install step

This task should be made optional via a variable or tag. In many enterprise RHEL environments, installing your own repository can be a big no-no, or an EPEL clone for the environment may already exist that isn't going to match the public EPEL package in yum.

For instance:
netbox_install_epel = false

- name: Install EPEL repository
  yum:
    name: epel-release
    state: installed
  when: netbox_install_epel

Add extra playbook examples

• one with load balancer and multiple NetBox workers

Run database backups when release changes

It should be possible to create somewhat of a safeguard before running database migrations due to a NetBox upgrade. Basically:

• check that the current symlink has changed, indicating a different NetBox release is being deployed (typically updates)
• Run a pgdump (or something with psycopg2 to keep footprint small, since I don't think we install a postgres client) to dump the NetBox database and store it for safekeeping

Race condition for git checkouts in CI

Sometimes the idempotency checks fail on Travis CI because new commits are added upstream while tests are running. Maybe a preparatory task can be added to the beginning of CI tests to identify the latest commit upstream and pin the git version to that commit for the entire test?

Deploy netbox via stable tarball releases

When the netbox_stable scenario is set to True/yes, the role should download and extract the release, defined by netbox_stable_release, from digitalocean/netbox to netbox_releases_path (e.g. {{ netbox_releases_path }}/{{ netbox_stable_release }}). This should be (as everything) an idempotent task - i.e. if a release already is downloaded it shouldn't be downloaded and extracted again.

• netbox_current_path should be symlinked to the active release.
• If the release changes, the netbox service should be restarted/reloaded.
• Deployment should use a virtualenv per release. Install dependencies using requirements.txt from the release tarball.
  • Be careful when creating the virtualenv - make sure to specify path to Python 3 or Python 2 depending on the role variable.

http://netbox.readthedocs.io/en/stable/installation/netbox/#option-a-download-a-release

Issue when Postgresql is a remote server

I try to install netbox with a remote postgresql database server on ubuntu 18.04.
My Playbook Looks like this:
`

• hosts: all
  become: yes
  roles:

  • davidwittman.redis
  • lae.netbox
    vars:
    netbox_stable: true
    netbox_stable_version: 2.7.9
    netbox_database_host:
    netbox_database_port: 5432
    netbox_database_conn_age: 300

  netbox_config:
  ALLOWED_HOSTS:
  - 127.0.0.1
  - localhost
  # note that the array is nested above
  MEDIA_ROOT: '{{ netbox_shared_path }}/media'
  REPORTS_ROOT: '{{ netbox_shared_path }}/reports'
  SCRIPTS_ROOT: '{{ netbox_shared_path }}/scripts'
  redis_bind: 127.0.0.1
  `

At the task Run database migrations for NetBox i receive the following error:
:stderr: Traceback (most recent call last): File "./manage.py", line 10, in <module> execute_from_command_line(sys.argv) File "/srv/netbox/current/venv-py3/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line utility.execute() File "/srv/netbox/current/venv-py3/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/srv/netbox/current/venv-py3/lib/python3.6/site-packages/django/core/management/base.py", line 323, in run_from_argv self.execute(*args, **cmd_options) File "/srv/netbox/current/venv-py3/lib/python3.6/site-packages/django/core/management/base.py", line 361, in execute self.check() File "/srv/netbox/current/venv-py3/lib/python3.6/site-packages/django/core/management/base.py", line 390, in check include_deployment_checks=include_deployment_checks, File "/srv/netbox/current/venv-py3/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 64, in _run_checks issues = run_checks(tags=[Tags.database]) File "/srv/netbox/current/venv-py3/lib/python3.6/site-packages/django/core/checks/registry.py", line 72, in run_checks new_errors = check(app_configs=app_configs) File "/srv/netbox/current/venv-py3/lib/python3.6/site-packages/django/core/checks/database.py", line 9, in check_database_backends for conn in connections.all(): File "/srv/netbox/current/venv-py3/lib/python3.6/site-packages/django/db/utils.py", line 216, in all return [self[alias] for alias in self] File "/srv/netbox/current/venv-py3/lib/python3.6/site-packages/django/db/utils.py", line 213, in __iter__ return iter(self.databases) File "/srv/netbox/current/venv-py3/lib/python3.6/site-packages/django/utils/functional.py", line 80, in __get__ res = instance.__dict__[self.name] = self.func(instance) File "/srv/netbox/current/venv-py3/lib/python3.6/site-packages/django/db/utils.py", line 147, in databases self._databases = settings.DATABASES File "/srv/netbox/current/venv-py3/lib/python3.6/site-packages/django/conf/__init__.py", line 79, in __getattr__ self._setup(name) File "/srv/netbox/current/venv-py3/lib/python3.6/site-packages/django/conf/__init__.py", line 66, in _setup self._wrapped = Settings(settings_module) File "/srv/netbox/current/venv-py3/lib/python3.6/site-packages/django/conf/__init__.py", line 157, in __init__ mod = importlib.import_module(self.SETTINGS_MODULE) File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 994, in _gcd_import File "<frozen importlib._bootstrap>", line 971, in _find_and_load File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 665, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 678, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/srv/netbox/releases/netbox-2.7.9/netbox/netbox/settings.py", line 163, in <module> "REDIS section in configuration.py is missing webhooks subsection." django.core.exceptions.ImproperlyConfigured: REDIS section in configuration.py is missing webhooks subsection.

If i do the installation on a single host or a vagrant box (All on one host postgres, redis and netbox) everything works fine.

Can you please help me?

Allow downgrades when necessary

• need to keep track of what version was previously deployed (easy for stable)
• backup database on upgrade

Would probably only make sense to support this for stable releases.

builds broken on ansible 2.5.0

Build link: https://travis-ci.org/lae/ansible-role-netbox/builds/357621896

Error:

TASK [ansible-role-netbox : Create NetBox virtualenv and install needed Python dependencies] ***
task path: /home/travis/build/lae/ansible-role-netbox/tasks/deploy_netbox.yml:26
Friday 23 March 2018  22:48:48 +0000 (0:00:00.115)       0:00:21.057 ********** 
fatal: [test03.lxc]: FAILED! => {
    "changed": false, 
    "cmd": "/usr/local/bin/virtualenv --no-site-packages -p/usr/bin/python3.4 /srv/netbox/current/venv-py3"
}
MSG:
stdout: Using base prefix '/usr'
Running virtualenv with interpreter /usr/bin/python3.4
:stderr: Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/virtualenv.py", line 2349, in <module>
    main()
  File "/usr/local/lib/python3.4/dist-packages/virtualenv.py", line 712, in main
    symlink=options.symlink)
  File "/usr/local/lib/python3.4/dist-packages/virtualenv.py", line 927, in create_environment
    site_packages=site_packages, clear=clear, symlink=symlink))
  File "/usr/local/lib/python3.4/dist-packages/virtualenv.py", line 1118, in install_python
    mkdir(lib_dir)
  File "/usr/local/lib/python3.4/dist-packages/virtualenv.py", line 324, in mkdir
    os.makedirs(path)
  File "/usr/lib/python3.4/os.py", line 227, in makedirs
    makedirs(head, mode, exist_ok)
  File "/usr/lib/python3.4/os.py", line 227, in makedirs
    makedirs(head, mode, exist_ok)
  File "/usr/lib/python3.4/os.py", line 237, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/srv/netbox/current/venv-py3'

Also add an extra build for ansible-devel (guess I forgot) because this should have been caught way earlier.

Deprecated REDIS configuration format

The 'webhooks' REDIS configuration section has been renamed to 'tasks'.

Currently, netbox complains that we should update our configuration like so:
UserWarning: The 'webhooks' REDIS configuration section has been renamed to 'tasks'. Please update your configuration as support for the old name will be removed in a future release."

Wrong requirement version for django-rq

Hello,

I tried to upgrade my netbox from version 2.7.6 to 2.7.7 and the database migration failed with this message:

ImportError: cannot import name 'UnpickleError' from 'rq.exceptions'

After searching I found this https://groups.google.com/forum/#!searchin/netbox-discuss/UnpickleError%7Csort:date/netbox-discuss/UEbwcPZLOqU/IRm0vPnZAgAJ.

The version in requirements.txt is not correct. Not really a bug of this playbook but ...
Is-it possible to add a task to replace the version of django-rq to 2.3.2 to fix this issue ?

I can do the MR if needed.

Thanks.

Add option to enable webhooks

Webhooks are not enabled by default with netbox install so additional packages and configuration is required
(as per https://netbox.readthedocs.io/en/latest/installation/2-netbox/#webhooks-configuration)

install redis
install django-rq via pip

addition configuration required

WEBHOOKS_ENABLED = True
REDIS = {
    'HOST': 'localhost',
    'PORT': 6379,
    'PASSWORD': '',
    'DATABASE': 0,
    'DEFAULT_TIMEOUT': 300,
}

Update configuration.py template for prometheus_client change

The prometheus_client library had a change back in February that is breaking with the metrics configuration for uwsgi.

prometheus/client_python#514

This line will need to be changed in one of two ways:

Rename the argument:

# Old
_pidFunc=uwsgi.worker_id)
# New
process_identifier=uwsgi.worker_id)

or just remove the kwarg altogether as the function only has one possible argument.

# Old
_pidFunc=uwsgi.worker_id)
# New
uwsgi.worker_id)