redhatinsights/topological_inventory-ansible_tower

topological_inventory-ansible_tower

License

This project is available as open source under the terms of the Apache License 2.0.

topological_inventory-ansible_tower's Issues

Redis-backed on-premise collecting

Proposed structure of redis key

Refresh_state_uuid => {
  :started_at => <ts>,
  :sweep_scope => <array>,
  :total_parts => <int>,
  :messages => [
    {:msg_id => {:entity_type => <string>, :started_at => <ts>, :finished_at => <ts>},
    {:msg_id => {:entity_type => <string>, :started_at => <ts>, :finished_at => <ts>},
    ....
  ],
  :all_requests_registered => T/F
}

Can be updated by Redis WATCH query

Redesign ordering process to avoid 30 minutes timeout

Current 30 minutes timeout for completing an Ansible Tower job order is not practical for GA. Tower jobs can take hours to days to complete.

PRs are ordered by dependency

!Important Core changes will affect persister/api/sync in production. Needed to change dependency on master branch in all environments first

• Prerequisity RedHatInsights/topological_inventory-core#190
  • core v1.0.0
  • Affected repos must be updated in QA(Production?) first!

---

• #85
• RedHatInsights/topological_inventory-core#187 [ requires RubyGems release - v1.1.0 ]
• RedHatInsights/topological_inventory-api#295
• RedHatInsights/topological_inventory-api-client-ruby#31 [ requires RubyGems release - v3.0.1 ]
• RedHatInsights/topological_inventory-providers-common#16 [ requires RubyGems release - v1.0.0 ]
• #84
• RedHatInsights/topological_inventory-core#188 [ requires RubyGems release - v1.1.1 ]
• RedHatInsights/topological_inventory-core#191
• RedHatInsights/topological_inventory-core#193 [ requires RubyGems release - v1.1.2 ]
• RedHatInsights/topological_inventory-persister#56
• RedHatInsights/topological_inventory-api#299

---

TPINVTRY-904

Collecting data through insights-upload (topological_inventory-sync)

ISSUE TYPE

• Feature Idea

Following discussion with @Ladas and ansible/awx#5931

---

SUMMARY

Full refresh through AnsibleTower collector <-> Tower API makes big traffic to Tower.
And through receptor it'll be quite slow.
So there is an idea to collect data through insights-upload service.

DETAILS

• This will be applicable for new versions of Tower, so collector's full refresh will be needed for some time.
• Targeted refresh has to be implemented for operations worker (job templates ordering)
• List of collected entities:
  • credentials (sensitive data not needed)
  • credential_types
  • inventories
  • job_templates/workflow_job_templates
  • surveys for job_templates/workflow_job_templates
  • workflow_job_template_nodes
  • credentials for workflow_job_template_nodes
  • jobs/workflow_jobs
  • workflow_job_nodes
  • credentials for workflow_job_nodes
  • https://github.com/RedHatInsights/topological_inventory-ansible_tower/blob/master/lib/topological_inventory/ansible_tower/collector/service_catalog.rb

Fix hakiri.io link in readme.md

we need to change
https://hakiri.io/github/ManageIQ/topological_inventory-ansible_tower/master
->
https://hakiri.io/github/RedHatInsights/topological_inventory-ansible_tower/master
but the link is not working

Order Operation takes too long

With the Thread pool functionality collector can maintain up to 10 sources and run refresh of max. 2 at the same time.

It's ok for refresh, but Order operations depends also on refresh (TODO: targeted refresh needed).
So between job is completed in tower and URL is collected to db there can be many minutes.

Solution: Decrease Sources per collector to 2 (for Ansible Tower) until targeted refresh will be ready.