skx/puppet-summary

The Puppet Summary is a web interface providing reporting features for Puppet, it replaces the Puppet Dashboard project

Steve Kemp

• I've been programming for over half my life, I'm comfortable creating, developing, maintaining, and improving software written in C, C++, Emacs Lisp, Perl, Ruby, Java, Shell, TCL, etc.
  • Most of my new projects are written in Golang.
  • But also I've been writing code in Z80 assembly language, to run under CP/M and the humble ZX Spectrum 48k.
• My interests primarily revolve around compilers, interpreters, domain-specific languages, and virtual machines.
  • Examples of scripting languages include a simple BASIC, a simple FORTH, a simple Lisp, and a simple TCL.
  • DSLs are great tools for automation, etc.
• Location: Helsinki, Finland.
• Occupation: Sysadmin / Devops / Cloud-person.

puppet-summary's Issues

Show relative times in our HTML-view

We used to do this, via a javascript helper, but we ran into problems because the puppet-parsing in yaml_parser.go throws away the timezone.

So we'll need to revisit this issue from the parser-upwards.

See #24 for reference.

Our metrics should be complete

Right now our metrics reflect reality, which means they show nodes in the states which are present.

For example I see this:

   p-s@master:~$ /srv/puppet-summary/puppet-summary metrics -nop
   puppet.attic_default_skx_uk0_bigv_io.runtime 2.322936554
   puppet.backup_vlan299_skx_uk0_bigv_io.runtime 2.0161260719546745
   ...
   puppet.web1_vlan299_skx_uk0_bigv_io.runtime 2.5634991849999995
   puppet.web2_vlan299_skx_uk0_bigv_io.runtime 1.971556927
   puppet.web3_vlan299_skx_uk0_bigv_io.runtime 2.1677419789999997
   ...
   puppet.state.unchanged 22

Here we see that the run-time figures are 100% complete, there is a figure for every node. But the state-metrics are incomplete:

• We only report upon states that are actually present.

We should always output values for each possible state (change, failed, unchanged), setting them to zero if not present.

Improve golint warnings.

We've still got a bunch of go lint warnings/errors:

• https://goreportcard.com/report/github.com/skx/puppet-summary#golint

Fix them all up, and then make a new release.

We should show orphaned nodes.

The puppet-dashboard has a notion of "unresponsive" nodes, nodes which haven't seen an update in the past while.

This project prunes reports after a few days, so such nodes will gradually fall off the display over time, but we should still show them.

Suggest we change getIndexNodes to only consider reports which are <24hours old, and then add a faux "orphaned" state to cover those which are outside that window.

Our command-line handling is horrid.

Because the flags are global they must be set before the sub-command.

e.g. "puppet-summary -port 3333 serve"

We should allow them to be after , as expected. For example:

   puppet-summary metrics -host localhost -port 2003 -prefix "puppet.one_puppet" -nop

Use TABs on the front-page.

Tested this with production data, 325 hosts. Everything was nice and responsive with 15 reports for each host, but the front-page made it hard to see what had changed/failed.

Use tabs like the original-project to show:

• all
• changed
• failed
• unchanged

Timezone not displayed correctly

I have an issue with the display of the report time. See the Screenshot attached. I think this is a timezone issue. The time in the report detail is correct (Screenshot 1), but at the summary site there are always displayed as "in 2 hours".

$ ls -l /etc/localtime
lrwxrwxrwx 1 root root 33 Apr 12 10:38 /etc/localtime -> /usr/share/zoneinfo/Europe/Berlin

Berlin is 2 hours off. So I think this is where those 2 hours come from.

I also see in the report

  logs:
    - !ruby/object:Puppet::Util::Log
      level: !ruby/sym notice
      tags:
        - notice
      message: "Finished catalog run in 7.91 seconds"
      source: Puppet
      time: 2017-08-16 12:53:58.519986 +02:00

This could also be an configuration error. But I can not think about any setting to fix this.

Raise alerts when state of run is `failed`.

Obviously I want to use mauvealert for that, but I should use a plugin or os.Execute with templated parameters to make it flexible.

e.g something like this:

 raise = mauvealert -i puppet-${fqnd} -s "Failed to run puppet on ${fqdn}" --detail="<p>The followign errors were found : ${failed_resources}"

Doesn't work with all reports? (maybe puppet 5?)

I'm unable to upload my reports to puppet-summary.

curl --data-binary @./reports/oz.example.com/201708071637.yaml 

http://192.168.1.79:3001/upload
Failed to parse 'time' from YAML

My report:

--- !ruby/object:Puppet::Transaction::Report
host: oz.example.com
time: '2017-08-07T16:37:42.659245699+00:00'
configuration_version: 1502123857
transaction_uuid: 99d2fd52-b3ac-44b3-b6c6-1eb7ba7d6564
report_format: 7
puppet_version: 5.0.1
status: changed
noop: false
noop_pending: false
environment: production
logs:
- level: notice
  message: created
  source: "/Stage[main]/Base/File[/usr/local/bin/puppet]/ensure"
  tags:
  - notice
  - file
  - class
  - base
  - consul
  time: '2017-08-07T16:37:43.088088347+00:00'
  file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
  line: 47
- level: notice
  message: created
  source: "/Stage[main]/Base/File[/usr/local/bin/facter]/ensure"
  tags:
  - notice
  - file
  - class
  - base
  - consul
  time: '2017-08-07T16:37:43.092097197+00:00'
  file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
  line: 52
- level: notice
  message: created
  source: "/Stage[main]/Base/File[/usr/local/bin/hiera]/ensure"
  tags:
  - notice
  - file
  - class
  - base
  - consul
  time: '2017-08-07T16:37:43.096002661+00:00'
  file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
  line: 57
- level: notice
  message: Applied catalog in 0.72 seconds
  source: Puppet
  tags:
  - notice
  time: '2017-08-07T16:37:43.418346261+00:00'
  file:
  line:
metrics:
  resources:
    name: resources
    label: Resources
    values:
    - - total
      - Total
      - 46
    - - skipped
      - Skipped
      - 0
    - - failed
      - Failed
      - 0
    - - failed_to_restart
      - Failed to restart
      - 0
    - - restarted
      - Restarted
      - 0
    - - changed
      - Changed
      - 3
    - - out_of_sync
      - Out of sync
      - 3
    - - scheduled
      - Scheduled
      - 0
    - - corrective_change
      - Corrective change
      - 0
  time:
    name: time
    label: Time
    values:
    - - package
      - Package
      - 0.002444112
    - - file
      - File
      - 0.164708044
    - - exec
      - Exec
      - 0.017048105
    - - user
      - User
      - 0.009217334
    - - group
      - Group
      - 0.007927178
    - - service
      - Service
      - 0.079661994
    - - schedule
      - Schedule
      - 0.002001613
    - - filebucket
      - Filebucket
      - 0.000354167
    - - config_retrieval
      - Config retrieval
      - 5.247163277
    - - total
      - Total
      - 5.530525824000001
  changes:
    name: changes
    label: Changes
    values:
    - - total
      - Total
      - 3
  events:
    name: events
    label: Events
    values:
    - - total
      - Total
      - 3
    - - failure
      - Failure
      - 0
    - - success
      - Success
      - 3
resource_statuses:
  Package[git]:
    title: git
    file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
    line: 6
    resource: Package[git]
    resource_type: Package
    containment_path:
    - Stage[main]
    - Base
    - Package[git]
    evaluation_time: 0.000800416
    tags:
    - package
    - git
    - class
    - base
    - consul
    time: '2017-08-07T16:37:42.936530651+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Package[rsync]:
    title: rsync
    file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
    line: 6
    resource: Package[rsync]
    resource_type: Package
    containment_path:
    - Stage[main]
    - Base
    - Package[rsync]
    evaluation_time: 0.000537864
    tags:
    - package
    - rsync
    - class
    - base
    - consul
    time: '2017-08-07T16:37:42.938222264+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Package[vim]:
    title: vim
    file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
    line: 6
    resource: Package[vim]
    resource_type: Package
    containment_path:
    - Stage[main]
    - Base
    - Package[vim]
    evaluation_time: 0.000581041
    tags:
    - package
    - vim
    - class
    - base
    - consul
    time: '2017-08-07T16:37:42.939855595+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Package[bash-completion]:
    title: bash-completion
    file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
    line: 6
    resource: Package[bash-completion]
    resource_type: Package
    containment_path:
    - Stage[main]
    - Base
    - Package[bash-completion]
    evaluation_time: 0.000524791
    tags:
    - package
    - bash-completion
    - class
    - base
    - consul
    time: '2017-08-07T16:37:42.941326479+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  File[/usr/local/bin/hardware_watchdog.sh]:
    title: "/usr/local/bin/hardware_watchdog.sh"
    file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
    line: 10
    resource: File[/usr/local/bin/hardware_watchdog.sh]
    resource_type: File
    containment_path:
    - Stage[main]
    - Base
    - File[/usr/local/bin/hardware_watchdog.sh]
    evaluation_time: 0.112167584
    tags:
    - file
    - class
    - base
    - consul
    time: '2017-08-07T16:37:42.942659290+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Exec[run-watchdog]:
    title: run-watchdog
    file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
    line: 18
    resource: Exec[run-watchdog]
    resource_type: Exec
    containment_path:
    - Stage[main]
    - Base
    - Exec[run-watchdog]
    evaluation_time: 0.014980556
    tags:
    - exec
    - run-watchdog
    - class
    - base
    - consul
    time: '2017-08-07T16:37:43.055932341+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  File[/root/.bashrc]:
    title: "/root/.bashrc"
    file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
    line: 31
    resource: File[/root/.bashrc]
    resource_type: File
    containment_path:
    - Stage[main]
    - Base
    - File[/root/.bashrc]
    evaluation_time: 0.004680151
    tags:
    - file
    - class
    - base
    - consul
    time: '2017-08-07T16:37:43.072266906+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  User[pi]:
    title: pi
    file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
    line: 39
    resource: User[pi]
    resource_type: User
    containment_path:
    - Stage[main]
    - Base
    - User[pi]
    evaluation_time: 0.001155207
    tags:
    - user
    - pi
    - class
    - base
    - consul
    time: '2017-08-07T16:37:43.079244242+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Group[pi]:
    title: pi
    file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
    line: 43
    resource: Group[pi]
    resource_type: Group
    containment_path:
    - Stage[main]
    - Base
    - Group[pi]
    evaluation_time: 0.001764998
    tags:
    - group
    - pi
    - class
    - base
    - consul
    time: '2017-08-07T16:37:43.082316739+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  File[/usr/local/bin/puppet]:
    title: "/usr/local/bin/puppet"
    file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
    line: 47
    resource: File[/usr/local/bin/puppet]
    resource_type: File
    containment_path:
    - Stage[main]
    - Base
    - File[/usr/local/bin/puppet]
    evaluation_time: 0.00367687
    tags:
    - file
    - class
    - base
    - consul
    time: '2017-08-07T16:37:43.085102205+00:00'
    failed: false
    changed: true
    out_of_sync: true
    skipped: false
    change_count: 1
    out_of_sync_count: 1
    events:
    - audited: false
      property: ensure
      previous_value: absent
      desired_value: link
      historical_value:
      message: created
      name: link_created
      status: success
      time: '2017-08-07T16:37:43.086543505+00:00'
      redacted:
      corrective_change: false
    corrective_change: false
  File[/usr/local/bin/facter]:
    title: "/usr/local/bin/facter"
    file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
    line: 52
    resource: File[/usr/local/bin/facter]
    resource_type: File
    containment_path:
    - Stage[main]
    - Base
    - File[/usr/local/bin/facter]
    evaluation_time: 0.002724216
    tags:
    - file
    - class
    - base
    - consul
    time: '2017-08-07T16:37:43.090154959+00:00'
    failed: false
    changed: true
    out_of_sync: true
    skipped: false
    change_count: 1
    out_of_sync_count: 1
    events:
    - audited: false
      property: ensure
      previous_value: absent
      desired_value: link
      historical_value:
      message: created
      name: link_created
      status: success
      time: '2017-08-07T16:37:43.091388135+00:00'
      redacted:
      corrective_change: false
    corrective_change: false
  File[/usr/local/bin/hiera]:
    title: "/usr/local/bin/hiera"
    file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
    line: 57
    resource: File[/usr/local/bin/hiera]
    resource_type: File
    containment_path:
    - Stage[main]
    - Base
    - File[/usr/local/bin/hiera]
    evaluation_time: 0.002664528
    tags:
    - file
    - class
    - base
    - consul
    time: '2017-08-07T16:37:43.093989330+00:00'
    failed: false
    changed: true
    out_of_sync: true
    skipped: false
    change_count: 1
    out_of_sync_count: 1
    events:
    - audited: false
      property: ensure
      previous_value: absent
      desired_value: link
      historical_value:
      message: created
      name: link_created
      status: success
      time: '2017-08-07T16:37:43.095146777+00:00'
      redacted:
      corrective_change: false
    corrective_change: false
  File[/usr/local/src]:
    title: "/usr/local/src"
    file: "/etc/puppetlabs/code/environments/production/modules/staging/manifests/init.pp"
    line: 26
    resource: File[/usr/local/src]
    resource_type: File
    containment_path:
    - Stage[main]
    - Staging
    - File[/usr/local/src]
    evaluation_time: 0.001984321
    tags:
    - file
    - class
    - staging
    - consul
    time: '2017-08-07T16:37:43.102629164+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  User[consul]:
    title: consul
    file: "/etc/puppetlabs/code/environments/production/modules/consul/manifests/init.pp"
    line: 27
    resource: User[consul]
    resource_type: User
    containment_path:
    - Stage[main]
    - Consul
    - User[consul]
    evaluation_time: 0.002749945
    tags:
    - user
    - consul
    - class
    time: '2017-08-07T16:37:43.108784626+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Group[consul]:
    title: consul
    file: "/etc/puppetlabs/code/environments/production/modules/consul/manifests/init.pp"
    line: 32
    resource: Group[consul]
    resource_type: Group
    containment_path:
    - Stage[main]
    - Consul
    - Group[consul]
    evaluation_time: 0.002215622
    tags:
    - group
    - consul
    - class
    time: '2017-08-07T16:37:43.113201288+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  File[/etc/consul]:
    title: "/etc/consul"
    file: "/etc/puppetlabs/code/environments/production/modules/consul/manifests/init.pp"
    line: 37
    resource: File[/etc/consul]
    resource_type: File
    containment_path:
    - Stage[main]
    - Consul
    - File[/etc/consul]
    evaluation_time: 0.001856091
    tags:
    - file
    - class
    - consul
    time: '2017-08-07T16:37:43.116743680+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  File[/etc/consul/default.json]:
    title: "/etc/consul/default.json"
    file: "/etc/puppetlabs/code/environments/production/modules/consul/manifests/init.pp"
    line: 43
    resource: File[/etc/consul/default.json]
    resource_type: File
    containment_path:
    - Stage[main]
    - Consul
    - File[/etc/consul/default.json]
    evaluation_time: 0.002538383
    tags:
    - file
    - class
    - consul
    time: '2017-08-07T16:37:43.119816905+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  File[/var/lib/consul]:
    title: "/var/lib/consul"
    file: "/etc/puppetlabs/code/environments/production/modules/consul/manifests/init.pp"
    line: 52
    resource: File[/var/lib/consul]
    resource_type: File
    containment_path:
    - Stage[main]
    - Consul
    - File[/var/lib/consul]
    evaluation_time: 0.001898227
    tags:
    - file
    - class
    - consul
    time: '2017-08-07T16:37:43.123354089+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Group[stahnma]:
    title: stahnma
    file: "/etc/puppetlabs/code/environments/production/modules/users/manifests/init.pp"
    line: 13
    resource: Group[stahnma]
    resource_type: Group
    containment_path:
    - Stage[main]
    - Users
    - Group[stahnma]
    evaluation_time: 0.002232654
    tags:
    - group
    - stahnma
    - class
    - users
    time: '2017-08-07T16:37:43.129288249+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  User[stahnma]:
    title: stahnma
    file: "/etc/puppetlabs/code/environments/production/modules/users/manifests/init.pp"
    line: 3
    resource: User[stahnma]
    resource_type: User
    containment_path:
    - Stage[main]
    - Users
    - User[stahnma]
    evaluation_time: 0.003342809
    tags:
    - user
    - stahnma
    - class
    - users
    time: '2017-08-07T16:37:43.133463036+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  File[/home/stahnma/.ssh]:
    title: "/home/stahnma/.ssh"
    file: "/etc/puppetlabs/code/environments/production/modules/users/manifests/init.pp"
    line: 25
    resource: File[/home/stahnma/.ssh]
    resource_type: File
    containment_path:
    - Stage[main]
    - Users
    - File[/home/stahnma/.ssh]
    evaluation_time: 0.001933227
    tags:
    - file
    - class
    - users
    time: '2017-08-07T16:37:43.138039177+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  File[/home/stahnma/.ssh/authorized_keys]:
    title: "/home/stahnma/.ssh/authorized_keys"
    file: "/etc/puppetlabs/code/environments/production/modules/users/manifests/init.pp"
    line: 18
    resource: File[/home/stahnma/.ssh/authorized_keys]
    resource_type: File
    containment_path:
    - Stage[main]
    - Users
    - File[/home/stahnma/.ssh/authorized_keys]
    evaluation_time: 0.003842756
    tags:
    - file
    - class
    - users
    time: '2017-08-07T16:37:43.141122351+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  File[/etc/sudoers.d/stahnma]:
    title: "/etc/sudoers.d/stahnma"
    file: "/etc/puppetlabs/code/environments/production/modules/users/manifests/init.pp"
    line: 32
    resource: File[/etc/sudoers.d/stahnma]
    resource_type: File
    containment_path:
    - Stage[main]
    - Users
    - File[/etc/sudoers.d/stahnma]
    evaluation_time: 0.00362911
    tags:
    - file
    - class
    - users
    time: '2017-08-07T16:37:43.145885783+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Group[nomad]:
    title: nomad
    file: "/etc/puppetlabs/code/environments/production/modules/nomad/manifests/init.pp"
    line: 9
    resource: Group[nomad]
    resource_type: Group
    containment_path:
    - Stage[main]
    - Nomad
    - Group[nomad]
    evaluation_time: 0.001713904
    tags:
    - group
    - nomad
    - class
    time: '2017-08-07T16:37:43.153044369+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  User[nomad]:
    title: nomad
    file: "/etc/puppetlabs/code/environments/production/modules/nomad/manifests/init.pp"
    line: 4
    resource: User[nomad]
    resource_type: User
    containment_path:
    - Stage[main]
    - Nomad
    - User[nomad]
    evaluation_time: 0.001969373
    tags:
    - user
    - nomad
    - class
    time: '2017-08-07T16:37:43.156455771+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  File[/etc/nomad]:
    title: "/etc/nomad"
    file: "/etc/puppetlabs/code/environments/production/modules/nomad/manifests/init.pp"
    line: 14
    resource: File[/etc/nomad]
    resource_type: File
    containment_path:
    - Stage[main]
    - Nomad
    - File[/etc/nomad]
    evaluation_time: 0.001682498
    tags:
    - file
    - class
    - nomad
    time: '2017-08-07T16:37:43.159556497+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  File[/var/lib/nomad]:
    title: "/var/lib/nomad"
    file: "/etc/puppetlabs/code/environments/production/modules/nomad/manifests/init.pp"
    line: 20
    resource: File[/var/lib/nomad]
    resource_type: File
    containment_path:
    - Stage[main]
    - Nomad
    - File[/var/lib/nomad]
    evaluation_time: 0.001578175
    tags:
    - file
    - class
    - nomad
    time: '2017-08-07T16:37:43.162324671+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  File[/etc/systemd/system/nomad.service]:
    title: "/etc/systemd/system/nomad.service"
    file: "/etc/puppetlabs/code/environments/production/modules/nomad/manifests/init.pp"
    line: 26
    resource: File[/etc/systemd/system/nomad.service]
    resource_type: File
    containment_path:
    - Stage[main]
    - Nomad
    - File[/etc/systemd/system/nomad.service]
    evaluation_time: 0.003686871
    tags:
    - file
    - class
    - nomad
    time: '2017-08-07T16:37:43.164733574+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  File[/etc/nomad/nomad.hcl]:
    title: "/etc/nomad/nomad.hcl"
    file: "/etc/puppetlabs/code/environments/production/modules/nomad/manifests/init.pp"
    line: 42
    resource: File[/etc/nomad/nomad.hcl]
    resource_type: File
    containment_path:
    - Stage[main]
    - Nomad
    - File[/etc/nomad/nomad.hcl]
    evaluation_time: 0.002221664
    tags:
    - file
    - class
    - nomad
    time: '2017-08-07T16:37:43.169457892+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Service[nomad]:
    title: nomad
    file: "/etc/puppetlabs/code/environments/production/modules/nomad/manifests/init.pp"
    line: 36
    resource: Service[nomad]
    resource_type: Service
    containment_path:
    - Stage[main]
    - Nomad
    - Service[nomad]
    evaluation_time: 0.039530841
    tags:
    - service
    - nomad
    - class
    time: '2017-08-07T16:37:43.173070440+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  File[/usr/local/src/consul]:
    title: "/usr/local/src/consul"
    file: "/etc/puppetlabs/code/environments/production/modules/staging/manifests/file.pp"
    line: 42
    resource: File[/usr/local/src/consul]
    resource_type: File
    containment_path:
    - Stage[main]
    - Consul
    - Staging::Deploy[consul.zip]
    - Staging::File[consul.zip]
    - File[/usr/local/src/consul]
    evaluation_time: 0.000808124
    tags:
    - file
    - staging::file
    - staging
    - consul.zip
    - staging::deploy
    - deploy
    - class
    - consul
    time: '2017-08-07T16:37:43.216649142+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Exec[/usr/local/src/consul/consul.zip]:
    title: "/usr/local/src/consul/consul.zip"
    file: "/etc/puppetlabs/code/environments/production/modules/staging/manifests/file.pp"
    line: 134
    resource: Exec[/usr/local/src/consul/consul.zip]
    resource_type: Exec
    containment_path:
    - Stage[main]
    - Consul
    - Staging::Deploy[consul.zip]
    - Staging::File[consul.zip]
    - Exec[/usr/local/src/consul/consul.zip]
    evaluation_time: 0.000880051
    tags:
    - exec
    - staging::file
    - staging
    - file
    - consul.zip
    - staging::deploy
    - deploy
    - class
    - consul
    time: '2017-08-07T16:37:43.218738150+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  File[/usr/local/src/consul/consul.zip]:
    title: "/usr/local/src/consul/consul.zip"
    file: "/etc/puppetlabs/code/environments/production/modules/staging/manifests/file.pp"
    line: 157
    resource: File[/usr/local/src/consul/consul.zip]
    resource_type: File
    containment_path:
    - Stage[main]
    - Consul
    - Staging::Deploy[consul.zip]
    - Staging::File[consul.zip]
    - File[/usr/local/src/consul/consul.zip]
    evaluation_time: 0.000956353
    tags:
    - file
    - staging::file
    - staging
    - consul.zip
    - staging::deploy
    - deploy
    - class
    - consul
    time: '2017-08-07T16:37:43.221180282+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Exec[extract consul.zip]:
    title: extract consul.zip
    file: "/etc/puppetlabs/code/environments/production/modules/staging/manifests/extract.pp"
    line: 123
    resource: Exec[extract consul.zip]
    resource_type: Exec
    containment_path:
    - Stage[main]
    - Consul
    - Staging::Deploy[consul.zip]
    - Staging::Extract[consul.zip]
    - Exec[extract consul.zip]
    evaluation_time: 0.000562291
    tags:
    - exec
    - staging::extract
    - staging
    - extract
    - consul.zip
    - staging::deploy
    - deploy
    - class
    - consul
    time: '2017-08-07T16:37:43.225559913+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  File[consul]:
    title: consul
    file: "/etc/puppetlabs/code/environments/production/modules/consul/manifests/init.pp"
    line: 19
    resource: File[consul]
    resource_type: File
    containment_path:
    - Stage[main]
    - Consul
    - File[consul]
    evaluation_time: 0.002502549
    tags:
    - file
    - consul
    - class
    time: '2017-08-07T16:37:43.228964336+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  File[/etc/systemd/system/consul.service]:
    title: "/etc/systemd/system/consul.service"
    file: "/etc/puppetlabs/code/environments/production/modules/consul/manifests/init.pp"
    line: 58
    resource: File[/etc/systemd/system/consul.service]
    resource_type: File
    containment_path:
    - Stage[main]
    - Consul
    - File[/etc/systemd/system/consul.service]
    evaluation_time: 0.003899579
    tags:
    - file
    - class
    - consul
    time: '2017-08-07T16:37:43.232687926+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Exec[systemd-reload]:
    title: systemd-reload
    file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
    line: 26
    resource: Exec[systemd-reload]
    resource_type: Exec
    containment_path:
    - Stage[main]
    - Base
    - Exec[systemd-reload]
    evaluation_time: 0.000625207
    tags:
    - exec
    - systemd-reload
    - class
    - base
    - consul
    time: '2017-08-07T16:37:43.237782764+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  File[/usr/local/bin/leader]:
    title: "/usr/local/bin/leader"
    file: "/etc/puppetlabs/code/environments/production/modules/consul/manifests/init.pp"
    line: 68
    resource: File[/usr/local/bin/leader]
    resource_type: File
    containment_path:
    - Stage[main]
    - Consul
    - File[/usr/local/bin/leader]
    evaluation_time: 0.003776767
    tags:
    - file
    - class
    - consul
    time: '2017-08-07T16:37:43.240921927+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Service[consul]:
    title: consul
    file: "/etc/puppetlabs/code/environments/production/modules/consul/manifests/init.pp"
    line: 75
    resource: Service[consul]
    resource_type: Service
    containment_path:
    - Stage[main]
    - Consul
    - Service[consul]
    evaluation_time: 0.040131153
    tags:
    - service
    - consul
    - class
    time: '2017-08-07T16:37:43.246037286+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Schedule[puppet]:
    title: puppet
    file:
    line:
    resource: Schedule[puppet]
    resource_type: Schedule
    containment_path:
    - Schedule[puppet]
    evaluation_time: 0.000395417
    tags:
    - schedule
    - puppet
    time: '2017-08-07T16:37:43.290429684+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Schedule[hourly]:
    title: hourly
    file:
    line:
    resource: Schedule[hourly]
    resource_type: Schedule
    containment_path:
    - Schedule[hourly]
    evaluation_time: 0.000409947
    tags:
    - schedule
    - hourly
    time: '2017-08-07T16:37:43.291429423+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Schedule[daily]:
    title: daily
    file:
    line:
    resource: Schedule[daily]
    resource_type: Schedule
    containment_path:
    - Schedule[daily]
    evaluation_time: 0.000275572
    tags:
    - schedule
    - daily
    time: '2017-08-07T16:37:43.292587078+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Schedule[weekly]:
    title: weekly
    file:
    line:
    resource: Schedule[weekly]
    resource_type: Schedule
    containment_path:
    - Schedule[weekly]
    evaluation_time: 0.000353021
    tags:
    - schedule
    - weekly
    time: '2017-08-07T16:37:43.293321035+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Schedule[monthly]:
    title: monthly
    file:
    line:
    resource: Schedule[monthly]
    resource_type: Schedule
    containment_path:
    - Schedule[monthly]
    evaluation_time: 0.000330677
    tags:
    - schedule
    - monthly
    time: '2017-08-07T16:37:43.294179420+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Schedule[never]:
    title: never
    file:
    line:
    resource: Schedule[never]
    resource_type: Schedule
    containment_path:
    - Schedule[never]
    evaluation_time: 0.000236979
    tags:
    - schedule
    - never
    time: '2017-08-07T16:37:43.295016867+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
  Filebucket[puppet]:
    title: puppet
    file:
    line:
    resource: Filebucket[puppet]
    resource_type: Filebucket
    containment_path:
    - Filebucket[puppet]
    evaluation_time: 0.000354167
    tags:
    - filebucket
    - puppet
    time: '2017-08-07T16:37:43.295908480+00:00'
    failed: false
    changed: false
    out_of_sync: false
    skipped: false
    change_count: 0
    out_of_sync_count: 0
    events: []
    corrective_change: false
corrective_change: false
catalog_uuid: cc7d64ff-f1bc-4317-89a2-2285a95790fa
cached_catalog_status: not_used

Our yaml-parsing of times is bogus

We should do better - as per the logs/changed/skipped/failed status.

favicon?

When I have lots of tabs open, it's hard to tell which one is the puppet-summary tab. Perhaps it needs a favicon so it can be noticeable in my seas of browser tabs.

Feature requests welcome!

For my own personal needs I think this project is pretty much done.

• There might be bugs, which will be fixed.
• There might be improvements to the code-quality, test-cases, and coverage, but from an end-user perspective these are internal details which don't matter.

Right now I can view my nodes, I can see their states, and I can look at what changed/failed. That's pretty much all I wanted. Compared to the original puppet-dashboard there are two notable omissions:

• We don't show "orphaned nodes".
  • i.e. Nodes that used to report in the past, but no longer do so.
  • In fact we explicitly prune reports with the expectation that nodes which no longer report will silently fall off the display.
• We don't show unchanged resources.
  • i.e. We show "changed", "failed", and "skipped" resources in the report-view, but not those that were OK.
  • I think that provides a less cluttered display, and is a good thing. But I can see an argument for including the details.

If I'm missing anything I'd be happy to hear though.

Introduce API support

At the moment we have a simple API-handler, which allows fetching nodes in a particular state:

• /api/state/$state

This is nice, but it isn't as useful as it could be. For example it doesn't allow discovery of all known-nodes, or similar. Ideally we should allow programmatic access to all our state/data.

I was going to write "/api/summary" to replicate the front-page when I had a thought - why not add Accept-header parsing on the existing HTTP-handlers? If we do that we allow this:

• curl -H "Accept: application/xml" https://master.steve.org.uk/report/7
• curl -H "Accept: application/json" https://master.steve.org.uk/node/www.steve.org.uk
• curl -H "Accept: application/json" https://master.steve.org.uk/

I suggest explicitly allowing both:

• application/json
• application/xml

Front-page graph should be more useful.

At the moment the front-page shows a donut-chart of the current state of each node. It would be more useful to show a stacked graph of the previous N days.

(i.e. Just like puppet-dashboard.)

• Get the dates we have coverage for:
  • SELECT DISTINCT(strftime('%d/%m/%Y', DATE(executed_at, 'unixepoch'))) FROM reports;
• Get the data for each day:
  • SELECT distinct state, COUNT(state) AS CountOf FROM reports WHERE strftime('%d/%m/%Y', DATE(executed_at, 'unixepoch'))= "06/08/2017" GROUP by state;

Pruning doesn't work if reports are stored in a different location.

If the defaults are left alone this works:

   puppet-summary serve [-port nn -host xx.yy.zz.ii]
   puppet-summary prune [-days ..]

However the user might choose to store the reports, on-disk, at a different path:

   puppet-summary serve -prefix=/var/log/puppet-summary

In that case the prune command will remove the references to the file(s) in the database, but not remove the orphaned reports from disk.

• prune sub-command needs to accept -prefix too

Perhaps we should just use a config-file and be done with it?

Our API end-point should allow you to choose data-types.

There is an API end-point which allows you to retrieve nodes in a given state:

• GET /api/state/${state}

This end-point only returns JSON, as you can see here:

   $ curl https://master.steve.org.uk/api/state/changed
   ["ssh.steve.org.uk"]

This should be updated to allow JSON, XML, and plain-text via either the Accept header, or via the ?accept=XXX parameter.

No way to view logs of what's happening

In trying to play with this, I can't see what's happening if a server returns a 500 level error message. I can't find a way to see logs to debug from the server.

Can't click on all links in radiator-view

Recently (read: last night) I added support to the radiator view for clickable links, but unfortunately this doesn't work as well as it could.

Currently all my machines are in the unchanged state, so the radiator view has the coloured-backdrop which obscures the link:

• https://master.steve.org.uk/radiator/

Perhaps the best solution is to make the click happen in the td-cell, rather than on the text. Just like we do for the other table-views?

Feature Request: Certificate request manual approvals

I realise this may be going outside the scope/intention of the project as it's not specifically reporting, but I think having the ability to approve pending/manual certificate signing requests similar to how PE provides this feature would be useful.

Specially, in situations where this dashboard is being used to provide an interface to operations teams, having this manual approval would provide the ability to give the task signing responsibility to an operator through dashboard access removing their need to login to shell etc. to approve new servers/certificates.

Basically page structure is 'unsigned certificates' equivalent to
puppet cert list

With the ability to sign/reject the cert request which is equivalent to the action
puppet cert sign <cert name>

Pruning does not appear to be working

When I run the following command:
puppet-summary prune -days 7 -prefix /mnt/puppetSummary/reports
The command returns immediately with no pruning actually occurring.

Metrics submission seems to be somewhat broken right now.

puppet-summary automatically submits metrics about the go-environment, if the appropriate environmental variables are set. This is done via (my) golang-metrics library.

The system-metrics are working just fine, but it seems that the submission of node-state is failing at the moment. You should be able to run this, via cron, with */15, */5, or similar frequency:

       ./puppet-summary metrics -host metrics.example.com

Unfortunately this has started to fail "recently":

        Error getting node states: database is locked

If you stop the server then it all works as expected, but of course that is not a solution at all.

go-bindata is depreciated

go-bindata was taken over by another user, as the original author deleted their (github) account.

Need to look at alternatives. Might be a fun project to write a replacement of our own.

Add screenshots, remove sample link.

The public-instance I have, linked in the project, is going to go away soon.

To make things neater I should add screenshots to the repository, and remove the references to it.

Drop canvasjs

The unregistered text is annoying to me.

Look at chartjs instead - I have a proof of concept for #11 already using their stacked graph example.

Performance issues in initial display

We're running the latest revision of puppet-summary on RHEL7.6, front-ended with Apache and experiencing really poor performance from the web front-end. Page load times are on the order of 90+ seconds for 499 hosts and we're pruning the database to 14 days of retention. Are there any tips for gaining more performance out of this application?

Add security-headers

We should add suitable security-headers to our output, to prevent frame-embedding, click-jacking, and XSS.

We should handle puppet line/file better.

Right now we build up lists of resources for various states:

• changed
• failed

We just append the puppet line/file to the string, resulting in things like this:

     update sysadmin utils(/etc/puppet/code/environments/production/modules/sysadmin_util/manifests/init.pp:31)

Rather than using an array of strings we should have a structure "PuppetResource", which would allow neater output. (i.e. Just show the resource, but click to reveal the file/line).

Restrict submissions?

It might be worth adding an IP-ACL to deny uploading of reports from unknown sources.

For example my own installation is public, so somebody could submit YAML files to the /upload end-point.

That's a potential DoS if they did that non-stop and ate all my disk.

In a real/production use firewalls would take care of this, as the reports wouldn't be exported, but for small installs like this personal one it might be worth handling:

   $ puppet-summary serve -acl 127.0.0.0/8

Comma-separated IP ACL to cover both IPv4 and IPv6, I guess.

Don't think it's worth having a view-ACL, as I expect this would be handled via the reverse-proxy I suggest users use.

Add static-check analysis to the test-script,.

Some code-purity issues flagged:

cmd_metrics.go:64:22: should omit comparison to bool constant, can be simplified to !nop (S1002)
cmd_metrics_test.go:71:12: do not pass a nil Context, even if a function permits it; pass context.TODO if you are unsure about which Context to use (SA1012)
cmd_serve.go:76:19: error strings should not be capitalized (ST1005)
cmd_serve.go:89:19: error strings should not be capitalized (ST1005)
cmd_serve.go:256:21: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:275:21: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:316:19: error strings should not be capitalized (ST1005)
cmd_serve.go:358:29: the argument is already a string, there's no need to use fmt.Sprintf (S1025)
cmd_serve.go:377:45: the argument is already a string, there's no need to use fmt.Sprintf (S1025)
cmd_serve.go:385:19: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:417:19: error strings should not be capitalized (ST1005)
cmd_serve.go:432:19: error strings should not be capitalized (ST1005)
cmd_serve.go:477:20: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:496:20: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:541:19: error strings should not be capitalized (ST1005)
cmd_serve.go:551:19: error strings should not be capitalized (ST1005)
cmd_serve.go:619:21: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:659:21: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:705:19: error strings should not be capitalized (ST1005)
cmd_serve.go:784:21: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:815:21: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:854:20: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:890:20: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:994:21: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:1013:21: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_version_test.go:20:12: do not pass a nil Context, even if a function permits it; pass context.TODO if you are unsure about which Context to use (SA1012)
cmd_version_test.go:37:12: do not pass a nil Context, even if a function permits it; pass context.TODO if you are unsure about which Context to use (SA1012)
db.go:247:24: error strings should not be capitalized (ST1005)
db.go:462:4: should merge variable declaration with assignment on next line (S1021)
db.go:588:26: error strings should not be capitalized (ST1005)
db.go:633:27: error strings should not be capitalized (ST1005)
static.go:124:24: error strings should not be capitalized (ST1005)
yaml_parser.go:118:20: error strings should not be capitalized (ST1005)
yaml_parser.go:126:20: error strings should not be capitalized (ST1005)
yaml_parser.go:144:20: error strings should not be capitalized (ST1005)
yaml_parser.go:173:20: error strings should not be capitalized (ST1005)
yaml_parser.go:280:20: error strings should not be capitalized (ST1005)
yaml_parser.go:317:20: error strings should not be capitalized (ST1005)
yaml_parser.go:408:23: error strings should not be capitalized (ST1005)
yaml_parser_test.go:98:3: should merge variable declaration with assignment on next line (S1021)
yaml_parser_test.go:165:3: should merge variable declaration with assignment on next line (S1021)

Feature request, sorting

Would like to be able to sort the columns. We could see nodes that have recently run easier.

Thanks!

Add option to delete reports if status is unchanged

I'm testing your tool for the last 24 hours and as I already said, it's really nice but the problem that is's consuming a lot of space.

In my case, after a bit more then 24h of it running, it's already using 14GB of space for reports (puppet is running every 30min on nodes) with 10k+ reports generated daily. Most of the reports are "unchanged", which I think could easily be remove since they don't contain any useful data (at least for me).

A solution could be an option to delete reports, when status is unchanged. When puppet uploads the reports, it pulls out data it needs (status, timestamp, execution time,...) for sqlite db and then remove it. In case status is failed/changed, it would save it.
This also means there would be no link to reports if status is unchanged in node details.

What do you think?

Working with musl?

This is more of a question than an issue, but have you tried getting this to work with either musl or a from scratch docker build? I get odd error like "file not found" when working with anything not glibc backed. It could be because of libraries in play here (probably is), but I'm not sure. Basically, I wanted a very tiny container to run this from. I can make it work with something like Ubuntu, but that's 128MB container. I was hoping for something < 5.

I'll keep fiddling with it, but figured I put a note here to see if anybody else had success here.

Node Search?

I'm looking for a replacement for puppet dashboard and was playing with the live example of puppet-summary. Is there any way to search for nodes using puppet-summary?

Do not depend on external sites

Hello,

Currently several URLs under https://maxcdn.bootstrapcdn.com/, https://code.jquery.com/ and https://cdnjs.cloudflare.com/ are hardcoded in the templates. Would it be possible to expose these URLs are configuration options rather than depending on third parties?

Test coverage is appalling.

After a bit more work this evening:

  deagol ~/go/src/github.com/skx/puppet-summary $ go test -coverprofile fmt 
  Error: The report ID must be numeric
  Error: SetupDB not called
  Error: SetupDB not called
  Error: SetupDB not called
  Error: Must be called via HTTP-POST
  PASS
  coverage: 13.4% of statements
  ok  	github.com/skx/puppet-summary	0.134s

13%. Terrible. Need to mock SQLite. Or create a fixed database in a temp-file, and add some records to it. At least that would let us get more work tests written..

Feature requests / questions welcome!

For my own personal needs I think this project is pretty much complete.

• There might be bugs, which will be fixed.
• There might be improvements to the code-quality, test-cases, and coverage, but from an end-user perspective these are internal details which don't matter.

Right now I can view my nodes, I can see their states, and I can look at what changed/failed. That's pretty much all I wanted. Compared to the original puppet-dashboard there is only one real omission:

• We don't show unchanged resources.
  • i.e. We show "changed", "failed", and "skipped" resources in the report-view, but not those that were OK.
  • I think that provides a less cluttered display, and is a good thing. But I can see an argument for including the details.

The notion of classes/groups is missing, as is the ability to operate as a node-classifier, but that is by design. The only real issue I have here is that with a few hundred managed-nodes the front-page gets terribly long and hard to view.

Anyway if the documentation is lacking/incomplete, or you have feature-requests/questions, please do report a new issue :)

Unify our DB code

At the moment this is a bit horrid, so at the very least create db.go and only connect to the DB on startup.

Add a radiator-view.

Add a simple radiator view. We should have the data via getIndexNodes() - as used in the stats.

Only missing data will be the percentages, then some horrid CSS-formatting.

Newlines in logs

Hello,

first, thank you for this great tool. I'm just testing it out now as a replacement for our puppet-dashboard and it's looking great:)

One thing I noticed is that it doesn't take new lines into account when showing logs. This would be useful when looking at diffs of files.

Current output:

--- /etc/my.cnf.d/server.cnf	2017-12-29 13:29:49.617471149 +0100 +++ /tmp/puppet-file20180305-43916-149xpky	2018-03-05 09:16:57.740678088 +0100 @@ -49,6 +49,7 @@ read-only = ON relay_log = /var/lib/mysql/data/relay-bin relay_log_index = /var/lib/mysql/data/relay-bin.index +replicate-wild-ignore-table = otrs.% report_host = foo.bar.org server_id = 13 slow_query_log = 1

I think it would be really nice to have some structure, so it would be easier to see changes.

Do we need an API?

I wrote a couple of endpoints:

• /api/state/XXX
  • Return a JSON array of hosts in the give state (failed, changed, unchanged).
• /api/nodes/(pattern)
  • Return all known node hostnames.
  • Optionally all that match the supplied pattern (as a regular-expression)

Not sure if it is useful enough to add on its own?

When I do this kind of thing at work I want to export data based on the local system. (i.e. "Nodes with package bash, or nodes with package foo version bar.) That's hard to do without adding a specific module to collect that data - such that it ends up in the YAML report(s).

Use colours to indicate changes

As per the puppet-dashboard, using ticks/crosses for "changed" and "failed"

Feature Request, tab for environments

In the report there is "environment: env_name"

Would it be possible to have the main page create tabs based on the env_name? You could click on an env_name to show all the nodes in that environment. Or just a drop down to separate nodes in different environments.

Thanks!

Option for setting days for orphan

I manage a lab full of puppet managed desktops. They dual-boot between Linux and Windows, and often get shutdown over weekends. Only the Linux side is puppet managed (Windows side is managed by App-V). It is not uncommon for these nodes to spend 3 or more days without reporting in to the puppet master.

Could we please get an option to set the number of days before a node is declared an orphan?

Probably not worth showing run-time.

The graph shows it nicely, no need to include it in the table.

Also ridiculous precision.

Pruning/auto-pruning not working due to locked database

When running
puppet-summary prune
with any command-line flag, the command returns
Error pruning: database is locked
and pruning does not occur. The -verbose flag did not provide more information.
-auto-prune is also affected, judging from the ever-increasing space occupied by reports. (Edit: Actually, it doesn't seem to be affected; my bad.)
I was able to reproduce the issue on multiple clean Vagrant boxes running Debian Stretch.
To reproduce:

• Install puppet-summary, using the binary release, go get installation or building it manually from source.
• Start puppet-summary directly with puppet-summary serve
• (Import some data)
• Execute puppet-summary prune with any option/flag combination

I was able to resolve my issue by modifying SetupDB code & rebuilding:

@@ -84,7 +84,7 @@ func SetupDB(path string) error {
        //
        // Return if the database already exists.
        //
-       db, err = sql.Open("sqlite3", path)
+  db, err = sql.Open("sqlite3", "file:"+path+"?_journal_mode=WAL")
        if err != nil {
                return err
        }

I was "inspired" to try this after reading mattn/go-sqlite3#607 (among many other related issues), but I'm not particularly suggesting my fix -- I'll gladly admit that I'm out of my depth.

We should move to "split commands".

As per sub-command we should have at least:

• p-s serve [-port xx | -host xx]
• p-s prune [-days 30]
• p-s help [prune | serve | help

Add screenshots

Graph timestamp in node view

Hello,

I was debuging a high runtime today and noticed I can't get a timestamp from the graph or table below. My only option was to count dots and locate the run in table below.

I think this feature could be useful for such cases if it's not too much trouble to implement.

Issue: Node count doesn't increase when new nodes come online.

I've been able to successfully install and run the package but I can't get my node page summary to show new nodes once they've been added to the puppet server. I've tried restarting the puppet-summary binary, restarting the puppetserver and impacting my forehead with the keyboard, none of which seem to work. Thanks in advance for any help.

Unable to cross compile

I love the xcc capabilities of Go usually. I run several daemons on ARM, however, I can't XCC this project, I think due to https://github.com/mattn/go-sqlite3. I don't know how tied to this SQLite binding you are, but I believe there are others that do work during a cross compile.

(I realize this a fringe request and feel free to close it if don't really care to spend time on it. I've been there as a maintainer)