device42/ansible_device42

Scripts to generate static inventory file or generate dynamic inventory for ansible from Device42 data

Device42 is a Continuous Discovery software for your IT Infrastructure. It helps you automatically maintain an up-to-date inventory of your physical, virtual, and cloud servers and containers, network components, software/services/applications, and their inter-relationships and inter-dependencies.

This Ansible Collection contains the inventory and lookup plugins for Device42.

1. Inventory (device42.d42.d42) - can be used to dynamically generate Ansible inventory from Device42 devices.
2. Lookup (device42.d42.d42)- This lookup plugin allows a playbook to fetch passwords or ip addresses for hosts from Device42.
3. Lookup (device42.d42.d42_prompt) - This lookup plugin allows a playbook to fetch passwords or ip addresses for hosts from Device42 with prompt.

In the contrib directory you will find the legacy inventory scripts. Please favor the plugin over the legacy scripts:

1. contrib\inventory\d42_ansible_inventory_hostfile.py can be used to create a static inventory file for ansible. You can group hosts by tags, customers, building or service level from Device42 data.
2. contrib\inventory\d42_ansible_dynamic_inventory.py can be used to dynamically by ansible to get hosts from Device42 based on certain filters.

---

Requirements

• ansible 2.9+
• python 3.6.x+
• Device42 16.12.00+
• requests (you can install it with pip install requests or apt-get install python-requests)
• Ansible must have an available connection to your Device42 instance in order to collect devices for inventory

Installation Methods

Galaxy

ansible-galaxy collection install device42.d42

Automation Hub

To consume content from hub as part of your automation workflows the content can also be accessed via CLI. For this an offline token is required which can be obtained via the web UI at automation hub, and needs to be added to the configuration file as follows:

[galaxy]
server_list = automation_hub, galaxy

[galaxy_server.automation_hub]
url=https://cloud.redhat.com/api/automation-hub/
auth_url=https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token
token=AABBccddeeff112233gghh...

[galaxy_server.galaxy]
url=https://galaxy.ansible.com/

Once configured the collection can be installed by running the following command

$ ansible-galaxy collection install device42.d42

Ansible Tower:

• Create a collections requirement file in the SCM

touch requirements.yml

• add the following to your file and save it

collections:
- name: device42.d42
version: 1.1.2
source: https://galaxy.ansible.com/

* Install the Ansible collection by running command
```bash
ansible-galaxy collection install -r requirements.yml -p <job tmp location>

Starting with Ansible Tower 3.6, the project folder will be copied for each job run. This allows playbooks to make local changes to the source tree for convenience, such as creating temporary files, without the possibility of interference with other jobs.

• Set the following environmental variables (optional)

D42_USER = 'device42 user'
D42_PWD = 'device42 password'
D42_URL = 'https:// device42 server IP address'
D42_SKIP_SSL_CHECK = False
D42_CLEAN_DEVICE_NAME = True

For more information on installing collections please follow documentation here https://docs.ansible.com/ansible/latest/user_guide/collections_using.html

---

Inventory Plugin

Configuration

Define an inventory file (*.d42.yml)

View documentation using ansible-doc -t inventory device42.d42.d42

Environmental variables not defined:

plugin: device42.d42.d42
url: https://10.10.10.10
username: admin
password: password
ssl_check: False
debug: False
clean_device_name: True
keyed_groups:
    - key: d42_service_level
      prefix: ''
      separator: ''

Environmental variables defined:

plugin: device42.d42.d42
keyed_groups:
    - key: d42_service_level
      prefix: ''
      separator: ''

See Ansible documentation for more constructed examples.

How to run

from the directory of your newly created file run the following command.

ansible-inventory -i *.d42.yml --graph

---

Lookup Plugin

Configuration

If using environmental variables skip this step, if not create a d42.py or d42_prompt.py in ansible/lib/ansible/plugins/lookup/ with the following information

D42_USER = 'device42 user'
D42_PWD = 'device42 password'
D42_URL = 'https:// device42 server IP address'
D42_SKIP_SSL_CHECK = False
D42_DEBUG = False
D42_CLEAN_DEVICE_NAME = True

How to run

To get password call: lookup('d42', 'device_name', 'password', 'username')

device_name and username need to be filled in

To run any doql request: lookup('d42', 'SELECT ... FROM ...', 'doql', 'list')

doql query need to be filled in + we need to set data type of returned result ( 'string', 'list', 'list_dicts' )

• string - return single string without column headers
• list - return list of string ( we split DOQL result line by line without column headers )
• list_dicts - return list of dicts with column headers

All above works the same for the prompt version, we just add 3 more arguments in the yaml file, please check reference in promt example.

The following was tested in a playbook using the included example template example_playbook.yaml playbooks can be run by using the following command

ansible-playbook *.yaml -f 10

Legacy Inventory Usage

---

Requirements

• ansible 2.9+

• python 3.6.x+ or python 2.7.x

• Device42 16.12.00+

• requests (you can install it with pip install requests or apt-get install python-requests)

• Ansible must have an available connection to your Device42 instance in order to collect devices for inventory

  • rename conf.sample.ini to conf.ini
  • in conf add D42 URL/credentials ( also instead of conf file, possible to use environment variables )

# ====== Device42 upload settings =========
[DEFAULT]
D42_USER = admin
D42_PWD = adm!nd42
D42_URL = https://10.10.10.10
D42_SKIP_SSL_CHECK = True
D42_DEBUG = FALSE

# ====== Ansible settings =========
[DOQL]
GROUP_BY_QUERY = select name, service_level from view_device_v1
GROUP_BY_FIELD = service_level
GROUP_BY_REFERENCE_FIELD = name
SPLIT_GROUP_BY_COMMA = False

Navigate to the root folder of the script .../ansible_device42

Run the python -m contrib.inventory.d42_ansible_inventory_hostfile and enjoy!

Also you may use automatic version with Ansible commands ex.

ping :

ansible all -i d42_ansible_dynamic_inventory.py -m ping

copy ssh file :

ansible all -i d42_ansible_dynamic_inventory.py -m authorized_key -a "user=root key='ssh-rsa AAAA...XXX == root@hostname'"

modify file :

ansible all -i d42_ansible_dynamic_inventory.py -m lineinfile -a "dest=/etc/group regexp='^(users:x:100:)(.*)' line='\1ldapusername,\2' state=present backrefs=yes"

and much more! Please read Ansible docs.

If you have any questions - feel free to reach out to us at support at device42.com

ansible_device42's Issues

Device42 Superuser privileges required by user

In order to use these python scripts, the user connecting to Device42 requires Superuser privileges which seems related to them using Device42's query interface (DOQL).

I think this information is worth noting in the README, or perhaps this is a bug with Device42, where a READ-ONLY user should be able to query with the DOQL API?

exception imp.load_source('conf', 'conf')

i tryed to start the script with my credentials in conf file, but i run directly in the exception in the lib.py after loading the conf file. Permissions are set

multiple tags, but one group

Hi all,
GROUP_BY_QUERY = 'SELECT vd.name, vd.tags, vi.ip_address FROM view_device_v1 vd join view_ipaddress_v1 vi on vd.device_pk = vi.device_fk WHERE vd.tags=\'all,role_percona_xtradb,dc_de_fra_bcknd_pdymisc,role_apache\''
GROUP_BY_FIELD = 'tags'
GROUP_BY_REFERENCE_FIELD = 'ip_address'

That is my content of the conf file.
Expected output:

[all]
Specific_instance1
 
[role_percona_xtradb]
Specific_instance1
 
[dc_de_fra_bcknd_pdymisc]
Specific_instance1
 
[role_apache]
Specific_instance1

But i got this output

[all,role_percona_xtradb,dc_de_fra_bcknd_pdymisc,role_apache]
Specific_instance1

As far as i know can't ansible work with that format.
Is there a workaround around this issue?

When is this scirpt support by Python3?

Hi. I have a question.
When is this scirpt support by Python3?

Authenticate via API Key

Any plans to support authentication via API Key vs using a password?

Getting custom fields: Response Status Code 500

D42 version: 16.12.00.1590694668

Command used: ansible-inventory -i vnp.d42.yml --graph

Results in empty inventory. Seems like the action Getting custom fields fails:

Response Status: 200
Getting MAC Addresses
Response Status: 200
Getting HDD Details
Response Status: 200
Getting Device External Links
Response Status: 200
Getting Device Entitlements
Response Status: 200
Getting Custom Fields
Response Status: 500
'str' object has no attribute 'get'
@all:
  |--@ungrouped:

After some digging, the used API Request doensn't work:

Is there a way to fix this?

Thanks!

error 500

Hello,

i'm trying to use the Ansible inventory but I'm getting a 500 error.

url: https://device42-t.contoso.local/
username: <username>
password: <password>
ssl_check: False
debug: True
clean_device_name: True
keyed_groups:
    - key: d42_service_level
      prefix: ''
      separator: ''

No config file found; using defaults
Using inventory plugin 'ansible_collections.device42.d42.plugins.inventory.d42' to process inventory source '/inventory.d42.yml'
Getting Devices
/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py:1103: InsecureRequestWarning: Unverified HTTPS request is being made to host 'device42-t.contoso.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
  warnings.warn(
an error was encountered on query API call to Device42
Response Status: 500
Getting IPs
an error was encountered on query API call to Device42
Response Status: 500
Getting MAC Addresses
an error was encountered on query API call to Device42
Response Status: 500
Getting HDD Details
an error was encountered on query API call to Device42
Response Status: 500
Getting Device External Links
an error was encountered on query API call to Device42
Response Status: 500
Getting Device Entitlements
an error was encountered on query API call to Device42
Response Status: 500
Getting Custom Fields
an error was encountered on query API call to Device42
Response Status: 500
@all:
  |--@ungrouped:

UnicodeEncodeError in get_list_from_csv

Thought this was due to not running python 2.7 as I was in 2.6 by default.
Installed 2.7 and running it gets the same Unicode error due to this decode call

#/opt/python-2.7.7/bin/python d42_ansible_inventory_hostfile.py
Traceback (most recent call last):
  File "d42_ansible_inventory_hostfile.py", line 9, in <module>
    groups = ansible.get_grouping(Device42(conf).doql())
  File "/home/tmcnulty/git/ansible_device42/lib.py", line 80, in doql
    return self.get_list_from_csv(self.fetcher(url, self.query))
  File "/home/tmcnulty/git/ansible_device42/lib.py", line 84, in get_list_from_csv
    f = StringIO(text.decode("utf-8"))
  File "/opt/python-2.7.7/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 234153-234154: ordinal not in range(128)

Dynamic Inventory - unrecognized arguments

I receive the following error when using the dynamic inventory script:

[WARNING]:  * Failed to parse /Users/my-user/git/ansible-home/inventory/ansible_device42/d42_ansible_dynamic_inventory.py with script plugin:
Inventory script (/Users/my-user/git/ansible-home/inventory/ansible_device42/d42_ansible_dynamic_inventory.py) had an execution error: usage:
d42_ansible_dynamic_inventory.py [-h] d42_ansible_dynamic_inventory.py: error: unrecognized arguments: --list

According the Ansible Script Inventory Plugin, the source must accept --list as an argument.

Looks like this was working and broken by commit 156c48f. Reverting this change solves the problem.

Dynamic Inventory doesn't seem to work due to missing SPLIT_GROUP_BY_COMMA element.

Hi folks. I've been having some trouble with the dynamic inventory always returning empty data, or errors. Further investigation seems to indicate a couple of problems with the lib.py.

• SPLIT_GROUP_BY_COMMA is never added to the conf dict, so in Ansible.get_grouping an exception is raised for the missing element.

Also, there seems to be a redundant parameter to d42_ansible_dynamic_inventory.py called --list that as far as I can tell does nothing.

static inventory doesn't work with GROUP_BY = 'tag'

I followed the instructions to run the script and got an inventory file that looked like:

[group1]
hosts

[group2]
hosts

[None]
host1
host2
host3
...

The code is just printing the name of the nested dict 'hosts' instead of the values. I fixed that problem, and then fixed the next problem (hosts in the 'None' group don't have the nested dict), so this script now works as designed.