skx / puppet-summary Goto Github PK
View Code? Open in Web Editor NEWThe Puppet Summary is a web interface providing reporting features for Puppet, it replaces the Puppet Dashboard project
License: GNU General Public License v2.0
The Puppet Summary is a web interface providing reporting features for Puppet, it replaces the Puppet Dashboard project
License: GNU General Public License v2.0
It might be worth adding an IP-ACL to deny uploading of reports from unknown sources.
For example my own installation is public, so somebody could submit YAML files to the /upload
end-point.
That's a potential DoS if they did that non-stop and ate all my disk.
In a real/production use firewalls would take care of this, as the reports wouldn't be exported, but for small installs like this personal one it might be worth handling:
$ puppet-summary serve -acl 127.0.0.0/8
Comma-separated IP ACL to cover both IPv4 and IPv6, I guess.
Don't think it's worth having a view-ACL, as I expect this would be handled via the reverse-proxy I suggest users use.
We've still got a bunch of go lint
warnings/errors:
Fix them all up, and then make a new release.
When I run the following command:
puppet-summary prune -days 7 -prefix /mnt/puppetSummary/reports
The command returns immediately with no pruning actually occurring.
Some code-purity issues flagged:
cmd_metrics.go:64:22: should omit comparison to bool constant, can be simplified to !nop (S1002)
cmd_metrics_test.go:71:12: do not pass a nil Context, even if a function permits it; pass context.TODO if you are unsure about which Context to use (SA1012)
cmd_serve.go:76:19: error strings should not be capitalized (ST1005)
cmd_serve.go:89:19: error strings should not be capitalized (ST1005)
cmd_serve.go:256:21: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:275:21: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:316:19: error strings should not be capitalized (ST1005)
cmd_serve.go:358:29: the argument is already a string, there's no need to use fmt.Sprintf (S1025)
cmd_serve.go:377:45: the argument is already a string, there's no need to use fmt.Sprintf (S1025)
cmd_serve.go:385:19: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:417:19: error strings should not be capitalized (ST1005)
cmd_serve.go:432:19: error strings should not be capitalized (ST1005)
cmd_serve.go:477:20: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:496:20: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:541:19: error strings should not be capitalized (ST1005)
cmd_serve.go:551:19: error strings should not be capitalized (ST1005)
cmd_serve.go:619:21: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:659:21: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:705:19: error strings should not be capitalized (ST1005)
cmd_serve.go:784:21: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:815:21: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:854:20: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:890:20: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:994:21: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_serve.go:1013:21: printf-style function with dynamic format string and no further arguments should use print-style function instead (SA1006)
cmd_version_test.go:20:12: do not pass a nil Context, even if a function permits it; pass context.TODO if you are unsure about which Context to use (SA1012)
cmd_version_test.go:37:12: do not pass a nil Context, even if a function permits it; pass context.TODO if you are unsure about which Context to use (SA1012)
db.go:247:24: error strings should not be capitalized (ST1005)
db.go:462:4: should merge variable declaration with assignment on next line (S1021)
db.go:588:26: error strings should not be capitalized (ST1005)
db.go:633:27: error strings should not be capitalized (ST1005)
static.go:124:24: error strings should not be capitalized (ST1005)
yaml_parser.go:118:20: error strings should not be capitalized (ST1005)
yaml_parser.go:126:20: error strings should not be capitalized (ST1005)
yaml_parser.go:144:20: error strings should not be capitalized (ST1005)
yaml_parser.go:173:20: error strings should not be capitalized (ST1005)
yaml_parser.go:280:20: error strings should not be capitalized (ST1005)
yaml_parser.go:317:20: error strings should not be capitalized (ST1005)
yaml_parser.go:408:23: error strings should not be capitalized (ST1005)
yaml_parser_test.go:98:3: should merge variable declaration with assignment on next line (S1021)
yaml_parser_test.go:165:3: should merge variable declaration with assignment on next line (S1021)
I wrote a couple of endpoints:
/api/state/XXX
failed
, changed
, unchanged
)./api/nodes/(pattern)
Not sure if it is useful enough to add on its own?
When I do this kind of thing at work I want to export data based on the local system. (i.e. "Nodes with package bash
, or nodes with package foo
version bar
.) That's hard to do without adding a specific module to collect that data - such that it ends up in the YAML report(s).
I love the xcc capabilities of Go usually. I run several daemons on ARM, however, I can't XCC this project, I think due to https://github.com/mattn/go-sqlite3. I don't know how tied to this SQLite binding you are, but I believe there are others that do work during a cross compile.
(I realize this a fringe request and feel free to close it if don't really care to spend time on it. I've been there as a maintainer)
As per the puppet-dashboard, using ticks/crosses for "changed" and "failed"
I have an issue with the display of the report time. See the Screenshot attached. I think this is a timezone issue. The time in the report detail is correct (Screenshot 1), but at the summary site there are always displayed as "in 2 hours".
$ ls -l /etc/localtime
lrwxrwxrwx 1 root root 33 Apr 12 10:38 /etc/localtime -> /usr/share/zoneinfo/Europe/Berlin
Berlin is 2 hours off. So I think this is where those 2 hours come from.
I also see in the report
logs:
- !ruby/object:Puppet::Util::Log
level: !ruby/sym notice
tags:
- notice
message: "Finished catalog run in 7.91 seconds"
source: Puppet
time: 2017-08-16 12:53:58.519986 +02:00
This could also be an configuration error. But I can not think about any setting to fix this.
We should add suitable security-headers to our output, to prevent frame-embedding, click-jacking, and XSS.
When I have lots of tabs open, it's hard to tell which one is the puppet-summary tab. Perhaps it needs a favicon so it can be noticeable in my seas of browser tabs.
After a bit more work this evening:
deagol ~/go/src/github.com/skx/puppet-summary $ go test -coverprofile fmt
Error: The report ID must be numeric
Error: SetupDB not called
Error: SetupDB not called
Error: SetupDB not called
Error: Must be called via HTTP-POST
PASS
coverage: 13.4% of statements
ok github.com/skx/puppet-summary 0.134s
13%. Terrible. Need to mock SQLite. Or create a fixed database in a temp-file, and add some records to it. At least that would let us get more work tests written..
The graph shows it nicely, no need to include it in the table.
Also ridiculous precision.
Hello,
I was debuging a high runtime today and noticed I can't get a timestamp from the graph or table below. My only option was to count dots and locate the run in table below.
I think this feature could be useful for such cases if it's not too much trouble to implement.
Add a simple radiator view. We should have the data via getIndexNodes()
- as used in the stats.
Only missing data will be the percentages, then some horrid CSS-formatting.
I'm unable to upload my reports to puppet-summary.
curl --data-binary @./reports/oz.example.com/201708071637.yaml
http://192.168.1.79:3001/upload
Failed to parse 'time' from YAML
--- !ruby/object:Puppet::Transaction::Report
host: oz.example.com
time: '2017-08-07T16:37:42.659245699+00:00'
configuration_version: 1502123857
transaction_uuid: 99d2fd52-b3ac-44b3-b6c6-1eb7ba7d6564
report_format: 7
puppet_version: 5.0.1
status: changed
noop: false
noop_pending: false
environment: production
logs:
- level: notice
message: created
source: "/Stage[main]/Base/File[/usr/local/bin/puppet]/ensure"
tags:
- notice
- file
- class
- base
- consul
time: '2017-08-07T16:37:43.088088347+00:00'
file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
line: 47
- level: notice
message: created
source: "/Stage[main]/Base/File[/usr/local/bin/facter]/ensure"
tags:
- notice
- file
- class
- base
- consul
time: '2017-08-07T16:37:43.092097197+00:00'
file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
line: 52
- level: notice
message: created
source: "/Stage[main]/Base/File[/usr/local/bin/hiera]/ensure"
tags:
- notice
- file
- class
- base
- consul
time: '2017-08-07T16:37:43.096002661+00:00'
file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
line: 57
- level: notice
message: Applied catalog in 0.72 seconds
source: Puppet
tags:
- notice
time: '2017-08-07T16:37:43.418346261+00:00'
file:
line:
metrics:
resources:
name: resources
label: Resources
values:
- - total
- Total
- 46
- - skipped
- Skipped
- 0
- - failed
- Failed
- 0
- - failed_to_restart
- Failed to restart
- 0
- - restarted
- Restarted
- 0
- - changed
- Changed
- 3
- - out_of_sync
- Out of sync
- 3
- - scheduled
- Scheduled
- 0
- - corrective_change
- Corrective change
- 0
time:
name: time
label: Time
values:
- - package
- Package
- 0.002444112
- - file
- File
- 0.164708044
- - exec
- Exec
- 0.017048105
- - user
- User
- 0.009217334
- - group
- Group
- 0.007927178
- - service
- Service
- 0.079661994
- - schedule
- Schedule
- 0.002001613
- - filebucket
- Filebucket
- 0.000354167
- - config_retrieval
- Config retrieval
- 5.247163277
- - total
- Total
- 5.530525824000001
changes:
name: changes
label: Changes
values:
- - total
- Total
- 3
events:
name: events
label: Events
values:
- - total
- Total
- 3
- - failure
- Failure
- 0
- - success
- Success
- 3
resource_statuses:
Package[git]:
title: git
file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
line: 6
resource: Package[git]
resource_type: Package
containment_path:
- Stage[main]
- Base
- Package[git]
evaluation_time: 0.000800416
tags:
- package
- git
- class
- base
- consul
time: '2017-08-07T16:37:42.936530651+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Package[rsync]:
title: rsync
file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
line: 6
resource: Package[rsync]
resource_type: Package
containment_path:
- Stage[main]
- Base
- Package[rsync]
evaluation_time: 0.000537864
tags:
- package
- rsync
- class
- base
- consul
time: '2017-08-07T16:37:42.938222264+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Package[vim]:
title: vim
file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
line: 6
resource: Package[vim]
resource_type: Package
containment_path:
- Stage[main]
- Base
- Package[vim]
evaluation_time: 0.000581041
tags:
- package
- vim
- class
- base
- consul
time: '2017-08-07T16:37:42.939855595+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Package[bash-completion]:
title: bash-completion
file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
line: 6
resource: Package[bash-completion]
resource_type: Package
containment_path:
- Stage[main]
- Base
- Package[bash-completion]
evaluation_time: 0.000524791
tags:
- package
- bash-completion
- class
- base
- consul
time: '2017-08-07T16:37:42.941326479+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
File[/usr/local/bin/hardware_watchdog.sh]:
title: "/usr/local/bin/hardware_watchdog.sh"
file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
line: 10
resource: File[/usr/local/bin/hardware_watchdog.sh]
resource_type: File
containment_path:
- Stage[main]
- Base
- File[/usr/local/bin/hardware_watchdog.sh]
evaluation_time: 0.112167584
tags:
- file
- class
- base
- consul
time: '2017-08-07T16:37:42.942659290+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Exec[run-watchdog]:
title: run-watchdog
file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
line: 18
resource: Exec[run-watchdog]
resource_type: Exec
containment_path:
- Stage[main]
- Base
- Exec[run-watchdog]
evaluation_time: 0.014980556
tags:
- exec
- run-watchdog
- class
- base
- consul
time: '2017-08-07T16:37:43.055932341+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
File[/root/.bashrc]:
title: "/root/.bashrc"
file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
line: 31
resource: File[/root/.bashrc]
resource_type: File
containment_path:
- Stage[main]
- Base
- File[/root/.bashrc]
evaluation_time: 0.004680151
tags:
- file
- class
- base
- consul
time: '2017-08-07T16:37:43.072266906+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
User[pi]:
title: pi
file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
line: 39
resource: User[pi]
resource_type: User
containment_path:
- Stage[main]
- Base
- User[pi]
evaluation_time: 0.001155207
tags:
- user
- pi
- class
- base
- consul
time: '2017-08-07T16:37:43.079244242+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Group[pi]:
title: pi
file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
line: 43
resource: Group[pi]
resource_type: Group
containment_path:
- Stage[main]
- Base
- Group[pi]
evaluation_time: 0.001764998
tags:
- group
- pi
- class
- base
- consul
time: '2017-08-07T16:37:43.082316739+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
File[/usr/local/bin/puppet]:
title: "/usr/local/bin/puppet"
file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
line: 47
resource: File[/usr/local/bin/puppet]
resource_type: File
containment_path:
- Stage[main]
- Base
- File[/usr/local/bin/puppet]
evaluation_time: 0.00367687
tags:
- file
- class
- base
- consul
time: '2017-08-07T16:37:43.085102205+00:00'
failed: false
changed: true
out_of_sync: true
skipped: false
change_count: 1
out_of_sync_count: 1
events:
- audited: false
property: ensure
previous_value: absent
desired_value: link
historical_value:
message: created
name: link_created
status: success
time: '2017-08-07T16:37:43.086543505+00:00'
redacted:
corrective_change: false
corrective_change: false
File[/usr/local/bin/facter]:
title: "/usr/local/bin/facter"
file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
line: 52
resource: File[/usr/local/bin/facter]
resource_type: File
containment_path:
- Stage[main]
- Base
- File[/usr/local/bin/facter]
evaluation_time: 0.002724216
tags:
- file
- class
- base
- consul
time: '2017-08-07T16:37:43.090154959+00:00'
failed: false
changed: true
out_of_sync: true
skipped: false
change_count: 1
out_of_sync_count: 1
events:
- audited: false
property: ensure
previous_value: absent
desired_value: link
historical_value:
message: created
name: link_created
status: success
time: '2017-08-07T16:37:43.091388135+00:00'
redacted:
corrective_change: false
corrective_change: false
File[/usr/local/bin/hiera]:
title: "/usr/local/bin/hiera"
file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
line: 57
resource: File[/usr/local/bin/hiera]
resource_type: File
containment_path:
- Stage[main]
- Base
- File[/usr/local/bin/hiera]
evaluation_time: 0.002664528
tags:
- file
- class
- base
- consul
time: '2017-08-07T16:37:43.093989330+00:00'
failed: false
changed: true
out_of_sync: true
skipped: false
change_count: 1
out_of_sync_count: 1
events:
- audited: false
property: ensure
previous_value: absent
desired_value: link
historical_value:
message: created
name: link_created
status: success
time: '2017-08-07T16:37:43.095146777+00:00'
redacted:
corrective_change: false
corrective_change: false
File[/usr/local/src]:
title: "/usr/local/src"
file: "/etc/puppetlabs/code/environments/production/modules/staging/manifests/init.pp"
line: 26
resource: File[/usr/local/src]
resource_type: File
containment_path:
- Stage[main]
- Staging
- File[/usr/local/src]
evaluation_time: 0.001984321
tags:
- file
- class
- staging
- consul
time: '2017-08-07T16:37:43.102629164+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
User[consul]:
title: consul
file: "/etc/puppetlabs/code/environments/production/modules/consul/manifests/init.pp"
line: 27
resource: User[consul]
resource_type: User
containment_path:
- Stage[main]
- Consul
- User[consul]
evaluation_time: 0.002749945
tags:
- user
- consul
- class
time: '2017-08-07T16:37:43.108784626+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Group[consul]:
title: consul
file: "/etc/puppetlabs/code/environments/production/modules/consul/manifests/init.pp"
line: 32
resource: Group[consul]
resource_type: Group
containment_path:
- Stage[main]
- Consul
- Group[consul]
evaluation_time: 0.002215622
tags:
- group
- consul
- class
time: '2017-08-07T16:37:43.113201288+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
File[/etc/consul]:
title: "/etc/consul"
file: "/etc/puppetlabs/code/environments/production/modules/consul/manifests/init.pp"
line: 37
resource: File[/etc/consul]
resource_type: File
containment_path:
- Stage[main]
- Consul
- File[/etc/consul]
evaluation_time: 0.001856091
tags:
- file
- class
- consul
time: '2017-08-07T16:37:43.116743680+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
File[/etc/consul/default.json]:
title: "/etc/consul/default.json"
file: "/etc/puppetlabs/code/environments/production/modules/consul/manifests/init.pp"
line: 43
resource: File[/etc/consul/default.json]
resource_type: File
containment_path:
- Stage[main]
- Consul
- File[/etc/consul/default.json]
evaluation_time: 0.002538383
tags:
- file
- class
- consul
time: '2017-08-07T16:37:43.119816905+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
File[/var/lib/consul]:
title: "/var/lib/consul"
file: "/etc/puppetlabs/code/environments/production/modules/consul/manifests/init.pp"
line: 52
resource: File[/var/lib/consul]
resource_type: File
containment_path:
- Stage[main]
- Consul
- File[/var/lib/consul]
evaluation_time: 0.001898227
tags:
- file
- class
- consul
time: '2017-08-07T16:37:43.123354089+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Group[stahnma]:
title: stahnma
file: "/etc/puppetlabs/code/environments/production/modules/users/manifests/init.pp"
line: 13
resource: Group[stahnma]
resource_type: Group
containment_path:
- Stage[main]
- Users
- Group[stahnma]
evaluation_time: 0.002232654
tags:
- group
- stahnma
- class
- users
time: '2017-08-07T16:37:43.129288249+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
User[stahnma]:
title: stahnma
file: "/etc/puppetlabs/code/environments/production/modules/users/manifests/init.pp"
line: 3
resource: User[stahnma]
resource_type: User
containment_path:
- Stage[main]
- Users
- User[stahnma]
evaluation_time: 0.003342809
tags:
- user
- stahnma
- class
- users
time: '2017-08-07T16:37:43.133463036+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
File[/home/stahnma/.ssh]:
title: "/home/stahnma/.ssh"
file: "/etc/puppetlabs/code/environments/production/modules/users/manifests/init.pp"
line: 25
resource: File[/home/stahnma/.ssh]
resource_type: File
containment_path:
- Stage[main]
- Users
- File[/home/stahnma/.ssh]
evaluation_time: 0.001933227
tags:
- file
- class
- users
time: '2017-08-07T16:37:43.138039177+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
File[/home/stahnma/.ssh/authorized_keys]:
title: "/home/stahnma/.ssh/authorized_keys"
file: "/etc/puppetlabs/code/environments/production/modules/users/manifests/init.pp"
line: 18
resource: File[/home/stahnma/.ssh/authorized_keys]
resource_type: File
containment_path:
- Stage[main]
- Users
- File[/home/stahnma/.ssh/authorized_keys]
evaluation_time: 0.003842756
tags:
- file
- class
- users
time: '2017-08-07T16:37:43.141122351+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
File[/etc/sudoers.d/stahnma]:
title: "/etc/sudoers.d/stahnma"
file: "/etc/puppetlabs/code/environments/production/modules/users/manifests/init.pp"
line: 32
resource: File[/etc/sudoers.d/stahnma]
resource_type: File
containment_path:
- Stage[main]
- Users
- File[/etc/sudoers.d/stahnma]
evaluation_time: 0.00362911
tags:
- file
- class
- users
time: '2017-08-07T16:37:43.145885783+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Group[nomad]:
title: nomad
file: "/etc/puppetlabs/code/environments/production/modules/nomad/manifests/init.pp"
line: 9
resource: Group[nomad]
resource_type: Group
containment_path:
- Stage[main]
- Nomad
- Group[nomad]
evaluation_time: 0.001713904
tags:
- group
- nomad
- class
time: '2017-08-07T16:37:43.153044369+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
User[nomad]:
title: nomad
file: "/etc/puppetlabs/code/environments/production/modules/nomad/manifests/init.pp"
line: 4
resource: User[nomad]
resource_type: User
containment_path:
- Stage[main]
- Nomad
- User[nomad]
evaluation_time: 0.001969373
tags:
- user
- nomad
- class
time: '2017-08-07T16:37:43.156455771+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
File[/etc/nomad]:
title: "/etc/nomad"
file: "/etc/puppetlabs/code/environments/production/modules/nomad/manifests/init.pp"
line: 14
resource: File[/etc/nomad]
resource_type: File
containment_path:
- Stage[main]
- Nomad
- File[/etc/nomad]
evaluation_time: 0.001682498
tags:
- file
- class
- nomad
time: '2017-08-07T16:37:43.159556497+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
File[/var/lib/nomad]:
title: "/var/lib/nomad"
file: "/etc/puppetlabs/code/environments/production/modules/nomad/manifests/init.pp"
line: 20
resource: File[/var/lib/nomad]
resource_type: File
containment_path:
- Stage[main]
- Nomad
- File[/var/lib/nomad]
evaluation_time: 0.001578175
tags:
- file
- class
- nomad
time: '2017-08-07T16:37:43.162324671+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
File[/etc/systemd/system/nomad.service]:
title: "/etc/systemd/system/nomad.service"
file: "/etc/puppetlabs/code/environments/production/modules/nomad/manifests/init.pp"
line: 26
resource: File[/etc/systemd/system/nomad.service]
resource_type: File
containment_path:
- Stage[main]
- Nomad
- File[/etc/systemd/system/nomad.service]
evaluation_time: 0.003686871
tags:
- file
- class
- nomad
time: '2017-08-07T16:37:43.164733574+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
File[/etc/nomad/nomad.hcl]:
title: "/etc/nomad/nomad.hcl"
file: "/etc/puppetlabs/code/environments/production/modules/nomad/manifests/init.pp"
line: 42
resource: File[/etc/nomad/nomad.hcl]
resource_type: File
containment_path:
- Stage[main]
- Nomad
- File[/etc/nomad/nomad.hcl]
evaluation_time: 0.002221664
tags:
- file
- class
- nomad
time: '2017-08-07T16:37:43.169457892+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Service[nomad]:
title: nomad
file: "/etc/puppetlabs/code/environments/production/modules/nomad/manifests/init.pp"
line: 36
resource: Service[nomad]
resource_type: Service
containment_path:
- Stage[main]
- Nomad
- Service[nomad]
evaluation_time: 0.039530841
tags:
- service
- nomad
- class
time: '2017-08-07T16:37:43.173070440+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
File[/usr/local/src/consul]:
title: "/usr/local/src/consul"
file: "/etc/puppetlabs/code/environments/production/modules/staging/manifests/file.pp"
line: 42
resource: File[/usr/local/src/consul]
resource_type: File
containment_path:
- Stage[main]
- Consul
- Staging::Deploy[consul.zip]
- Staging::File[consul.zip]
- File[/usr/local/src/consul]
evaluation_time: 0.000808124
tags:
- file
- staging::file
- staging
- consul.zip
- staging::deploy
- deploy
- class
- consul
time: '2017-08-07T16:37:43.216649142+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Exec[/usr/local/src/consul/consul.zip]:
title: "/usr/local/src/consul/consul.zip"
file: "/etc/puppetlabs/code/environments/production/modules/staging/manifests/file.pp"
line: 134
resource: Exec[/usr/local/src/consul/consul.zip]
resource_type: Exec
containment_path:
- Stage[main]
- Consul
- Staging::Deploy[consul.zip]
- Staging::File[consul.zip]
- Exec[/usr/local/src/consul/consul.zip]
evaluation_time: 0.000880051
tags:
- exec
- staging::file
- staging
- file
- consul.zip
- staging::deploy
- deploy
- class
- consul
time: '2017-08-07T16:37:43.218738150+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
File[/usr/local/src/consul/consul.zip]:
title: "/usr/local/src/consul/consul.zip"
file: "/etc/puppetlabs/code/environments/production/modules/staging/manifests/file.pp"
line: 157
resource: File[/usr/local/src/consul/consul.zip]
resource_type: File
containment_path:
- Stage[main]
- Consul
- Staging::Deploy[consul.zip]
- Staging::File[consul.zip]
- File[/usr/local/src/consul/consul.zip]
evaluation_time: 0.000956353
tags:
- file
- staging::file
- staging
- consul.zip
- staging::deploy
- deploy
- class
- consul
time: '2017-08-07T16:37:43.221180282+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Exec[extract consul.zip]:
title: extract consul.zip
file: "/etc/puppetlabs/code/environments/production/modules/staging/manifests/extract.pp"
line: 123
resource: Exec[extract consul.zip]
resource_type: Exec
containment_path:
- Stage[main]
- Consul
- Staging::Deploy[consul.zip]
- Staging::Extract[consul.zip]
- Exec[extract consul.zip]
evaluation_time: 0.000562291
tags:
- exec
- staging::extract
- staging
- extract
- consul.zip
- staging::deploy
- deploy
- class
- consul
time: '2017-08-07T16:37:43.225559913+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
File[consul]:
title: consul
file: "/etc/puppetlabs/code/environments/production/modules/consul/manifests/init.pp"
line: 19
resource: File[consul]
resource_type: File
containment_path:
- Stage[main]
- Consul
- File[consul]
evaluation_time: 0.002502549
tags:
- file
- consul
- class
time: '2017-08-07T16:37:43.228964336+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
File[/etc/systemd/system/consul.service]:
title: "/etc/systemd/system/consul.service"
file: "/etc/puppetlabs/code/environments/production/modules/consul/manifests/init.pp"
line: 58
resource: File[/etc/systemd/system/consul.service]
resource_type: File
containment_path:
- Stage[main]
- Consul
- File[/etc/systemd/system/consul.service]
evaluation_time: 0.003899579
tags:
- file
- class
- consul
time: '2017-08-07T16:37:43.232687926+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Exec[systemd-reload]:
title: systemd-reload
file: "/etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp"
line: 26
resource: Exec[systemd-reload]
resource_type: Exec
containment_path:
- Stage[main]
- Base
- Exec[systemd-reload]
evaluation_time: 0.000625207
tags:
- exec
- systemd-reload
- class
- base
- consul
time: '2017-08-07T16:37:43.237782764+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
File[/usr/local/bin/leader]:
title: "/usr/local/bin/leader"
file: "/etc/puppetlabs/code/environments/production/modules/consul/manifests/init.pp"
line: 68
resource: File[/usr/local/bin/leader]
resource_type: File
containment_path:
- Stage[main]
- Consul
- File[/usr/local/bin/leader]
evaluation_time: 0.003776767
tags:
- file
- class
- consul
time: '2017-08-07T16:37:43.240921927+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Service[consul]:
title: consul
file: "/etc/puppetlabs/code/environments/production/modules/consul/manifests/init.pp"
line: 75
resource: Service[consul]
resource_type: Service
containment_path:
- Stage[main]
- Consul
- Service[consul]
evaluation_time: 0.040131153
tags:
- service
- consul
- class
time: '2017-08-07T16:37:43.246037286+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Schedule[puppet]:
title: puppet
file:
line:
resource: Schedule[puppet]
resource_type: Schedule
containment_path:
- Schedule[puppet]
evaluation_time: 0.000395417
tags:
- schedule
- puppet
time: '2017-08-07T16:37:43.290429684+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Schedule[hourly]:
title: hourly
file:
line:
resource: Schedule[hourly]
resource_type: Schedule
containment_path:
- Schedule[hourly]
evaluation_time: 0.000409947
tags:
- schedule
- hourly
time: '2017-08-07T16:37:43.291429423+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Schedule[daily]:
title: daily
file:
line:
resource: Schedule[daily]
resource_type: Schedule
containment_path:
- Schedule[daily]
evaluation_time: 0.000275572
tags:
- schedule
- daily
time: '2017-08-07T16:37:43.292587078+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Schedule[weekly]:
title: weekly
file:
line:
resource: Schedule[weekly]
resource_type: Schedule
containment_path:
- Schedule[weekly]
evaluation_time: 0.000353021
tags:
- schedule
- weekly
time: '2017-08-07T16:37:43.293321035+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Schedule[monthly]:
title: monthly
file:
line:
resource: Schedule[monthly]
resource_type: Schedule
containment_path:
- Schedule[monthly]
evaluation_time: 0.000330677
tags:
- schedule
- monthly
time: '2017-08-07T16:37:43.294179420+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Schedule[never]:
title: never
file:
line:
resource: Schedule[never]
resource_type: Schedule
containment_path:
- Schedule[never]
evaluation_time: 0.000236979
tags:
- schedule
- never
time: '2017-08-07T16:37:43.295016867+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
Filebucket[puppet]:
title: puppet
file:
line:
resource: Filebucket[puppet]
resource_type: Filebucket
containment_path:
- Filebucket[puppet]
evaluation_time: 0.000354167
tags:
- filebucket
- puppet
time: '2017-08-07T16:37:43.295908480+00:00'
failed: false
changed: false
out_of_sync: false
skipped: false
change_count: 0
out_of_sync_count: 0
events: []
corrective_change: false
corrective_change: false
catalog_uuid: cc7d64ff-f1bc-4317-89a2-2285a95790fa
cached_catalog_status: not_used
Recently (read: last night) I added support to the radiator view for clickable links, but unfortunately this doesn't work as well as it could.
Currently all my machines are in the unchanged
state, so the radiator view has the coloured-backdrop which obscures the link:
Perhaps the best solution is to make the click happen in the td
-cell, rather than on the text. Just like we do for the other table-views?
This is more of a question than an issue, but have you tried getting this to work with either musl or a from scratch
docker build? I get odd error like "file not found" when working with anything not glibc backed. It could be because of libraries in play here (probably is), but I'm not sure. Basically, I wanted a very tiny container to run this from. I can make it work with something like Ubuntu, but that's 128MB container. I was hoping for something < 5.
I'll keep fiddling with it, but figured I put a note here to see if anybody else had success here.
I'm looking for a replacement for puppet dashboard and was playing with the live example of puppet-summary. Is there any way to search for nodes using puppet-summary?
In trying to play with this, I can't see what's happening if a server returns a 500 level error message. I can't find a way to see logs to debug from the server.
The puppet-dashboard has a notion of "unresponsive" nodes, nodes which haven't seen an update in the past while.
This project prunes reports after a few days, so such nodes will gradually fall off the display over time, but we should still show them.
Suggest we change getIndexNodes
to only consider reports which are <24hours old, and then add a faux "orphaned" state to cover those which are outside that window.
go-bindata
was taken over by another user, as the original author deleted their (github) account.
Need to look at alternatives. Might be a fun project to write a replacement of our own.
As per sub-command we should have at least:
p-s serve [-port xx | -host xx]
p-s prune [-days 30]
p-s help [prune | serve | help
The unregistered
text is annoying to me.
Look at chartjs instead - I have a proof of concept for #11 already using their stacked graph example.
At the moment the front-page shows a donut-chart of the current state of each node. It would be more useful to show a stacked graph of the previous N days.
(i.e. Just like puppet-dashboard.)
SELECT DISTINCT(strftime('%d/%m/%Y', DATE(executed_at, 'unixepoch'))) FROM reports;
SELECT distinct state, COUNT(state) AS CountOf FROM reports WHERE strftime('%d/%m/%Y', DATE(executed_at, 'unixepoch'))= "06/08/2017" GROUP by state;
I've been able to successfully install and run the package but I can't get my node page summary to show new nodes once they've been added to the puppet server. I've tried restarting the puppet-summary binary, restarting the puppetserver and impacting my forehead with the keyboard, none of which seem to work. Thanks in advance for any help.
I manage a lab full of puppet managed desktops. They dual-boot between Linux and Windows, and often get shutdown over weekends. Only the Linux side is puppet managed (Windows side is managed by App-V). It is not uncommon for these nodes to spend 3 or more days without reporting in to the puppet master.
Could we please get an option to set the number of days before a node is declared an orphan?
At the moment we have a simple API-handler, which allows fetching nodes in a particular state:
/api/state/$state
This is nice, but it isn't as useful as it could be. For example it doesn't allow discovery of all known-nodes, or similar. Ideally we should allow programmatic access to all our state/data.
I was going to write "/api/summary
" to replicate the front-page when I had a thought - why not add Accept-header parsing on the existing HTTP-handlers? If we do that we allow this:
curl -H "Accept: application/xml" https://master.steve.org.uk/report/7
curl -H "Accept: application/json" https://master.steve.org.uk/node/www.steve.org.uk
curl -H "Accept: application/json" https://master.steve.org.uk/
I suggest explicitly allowing both:
application/json
application/xml
Obviously I want to use mauvealert
for that, but I should use a plugin or os.Execute with templated parameters to make it flexible.
e.g something like this:
raise = mauvealert -i puppet-${fqnd} -s "Failed to run puppet on ${fqdn}" --detail="<p>The followign errors were found : ${failed_resources}"
Hello,
Currently several URLs under https://maxcdn.bootstrapcdn.com/, https://code.jquery.com/ and https://cdnjs.cloudflare.com/ are hardcoded in the templates. Would it be possible to expose these URLs are configuration options rather than depending on third parties?
Because the flags are global they must be set before the sub-command.
e.g. "puppet-summary -port 3333 serve"
We should allow them to be after , as expected. For example:
puppet-summary metrics -host localhost -port 2003 -prefix "puppet.one_puppet" -nop
For my own personal needs I think this project is pretty much done.
Right now I can view my nodes, I can see their states, and I can look at what changed/failed. That's pretty much all I wanted. Compared to the original puppet-dashboard there are two notable omissions:
If I'm missing anything I'd be happy to hear though.
If the defaults are left alone this works:
puppet-summary serve [-port nn -host xx.yy.zz.ii]
puppet-summary prune [-days ..]
However the user might choose to store the reports, on-disk, at a different path:
puppet-summary serve -prefix=/var/log/puppet-summary
In that case the prune
command will remove the references to the file(s) in the database, but not remove the orphaned reports from disk.
prune
sub-command needs to accept -prefix
tooPerhaps we should just use a config-file and be done with it?
When running
puppet-summary prune
with any command-line flag, the command returns
Error pruning: database is locked
and pruning does not occur. The -verbose
flag did not provide more information.
-auto-prune
is also affected, judging from the ever-increasing space occupied by reports. (Edit: Actually, it doesn't seem to be affected; my bad.)
I was able to reproduce the issue on multiple clean Vagrant boxes running Debian Stretch.
To reproduce:
puppet-summary serve
puppet-summary
prune with any option/flag combinationI was able to resolve my issue by modifying SetupDB code & rebuilding:
@@ -84,7 +84,7 @@ func SetupDB(path string) error {
//
// Return if the database already exists.
//
- db, err = sql.Open("sqlite3", path)
+ db, err = sql.Open("sqlite3", "file:"+path+"?_journal_mode=WAL")
if err != nil {
return err
}
I was "inspired" to try this after reading mattn/go-sqlite3#607 (among many other related issues), but I'm not particularly suggesting my fix -- I'll gladly admit that I'm out of my depth.
We should do better - as per the logs/changed/skipped/failed status.
We're running the latest revision of puppet-summary on RHEL7.6, front-ended with Apache and experiencing really poor performance from the web front-end. Page load times are on the order of 90+ seconds for 499 hosts and we're pruning the database to 14 days of retention. Are there any tips for gaining more performance out of this application?
I realise this may be going outside the scope/intention of the project as it's not specifically reporting, but I think having the ability to approve pending/manual certificate signing requests similar to how PE provides this feature would be useful.
Specially, in situations where this dashboard is being used to provide an interface to operations teams, having this manual approval would provide the ability to give the task signing responsibility to an operator through dashboard access removing their need to login to shell etc. to approve new servers/certificates.
Basically page structure is 'unsigned certificates' equivalent to
puppet cert list
With the ability to sign/reject the cert request which is equivalent to the action
puppet cert sign <cert name>
Would like to be able to sort the columns. We could see nodes that have recently run easier.
Thanks!
We used to do this, via a javascript helper, but we ran into problems because the puppet-parsing in yaml_parser.go
throws away the timezone.
So we'll need to revisit this issue from the parser-upwards.
See #24 for reference.
puppet-summary
automatically submits metrics about the go-environment, if the appropriate environmental variables are set. This is done via (my) golang-metrics library.
The system-metrics are working just fine, but it seems that the submission of node-state is failing at the moment. You should be able to run this, via cron, with */15
, */5
, or similar frequency:
./puppet-summary metrics -host metrics.example.com
Unfortunately this has started to fail "recently":
Error getting node states: database is locked
If you stop the server then it all works as expected, but of course that is not a solution at all.
The public-instance I have, linked in the project, is going to go away soon.
To make things neater I should add screenshots to the repository, and remove the references to it.
There is an API end-point which allows you to retrieve nodes in a given state:
GET /api/state/${state}
This end-point only returns JSON, as you can see here:
$ curl https://master.steve.org.uk/api/state/changed
["ssh.steve.org.uk"]
This should be updated to allow JSON, XML, and plain-text via either the Accept
header, or via the ?accept=XXX
parameter.
Tested this with production data, 325 hosts. Everything was nice and responsive with 15 reports for each host, but the front-page made it hard to see what had changed/failed.
Use tabs like the original-project to show:
I'm testing your tool for the last 24 hours and as I already said, it's really nice but the problem that is's consuming a lot of space.
In my case, after a bit more then 24h of it running, it's already using 14GB of space for reports (puppet is running every 30min on nodes) with 10k+ reports generated daily. Most of the reports are "unchanged", which I think could easily be remove since they don't contain any useful data (at least for me).
A solution could be an option to delete reports, when status is unchanged. When puppet uploads the reports, it pulls out data it needs (status, timestamp, execution time,...) for sqlite db and then remove it. In case status is failed/changed, it would save it.
This also means there would be no link to reports if status is unchanged in node details.
What do you think?
At the moment this is a bit horrid, so at the very least create db.go
and only connect to the DB on startup.
Right now we build up lists of resources for various states:
We just append the puppet line/file to the string, resulting in things like this:
update sysadmin utils(/etc/puppet/code/environments/production/modules/sysadmin_util/manifests/init.pp:31)
Rather than using an array of strings we should have a structure "PuppetResource", which would allow neater output. (i.e. Just show the resource, but click to reveal the file/line).
In the report there is "environment: env_name"
Would it be possible to have the main page create tabs based on the env_name? You could click on an env_name to show all the nodes in that environment. Or just a drop down to separate nodes in different environments.
Thanks!
Right now our metrics reflect reality, which means they show nodes in the states which are present.
For example I see this:
p-s@master:~$ /srv/puppet-summary/puppet-summary metrics -nop
puppet.attic_default_skx_uk0_bigv_io.runtime 2.322936554
puppet.backup_vlan299_skx_uk0_bigv_io.runtime 2.0161260719546745
...
puppet.web1_vlan299_skx_uk0_bigv_io.runtime 2.5634991849999995
puppet.web2_vlan299_skx_uk0_bigv_io.runtime 1.971556927
puppet.web3_vlan299_skx_uk0_bigv_io.runtime 2.1677419789999997
...
puppet.state.unchanged 22
Here we see that the run-time figures are 100% complete, there is a figure for every node. But the state-metrics are incomplete:
We should always output values for each possible state (change
, failed
, unchanged
), setting them to zero if not present.
Hello,
first, thank you for this great tool. I'm just testing it out now as a replacement for our puppet-dashboard and it's looking great:)
One thing I noticed is that it doesn't take new lines into account when showing logs. This would be useful when looking at diffs of files.
Current output:
--- /etc/my.cnf.d/server.cnf 2017-12-29 13:29:49.617471149 +0100 +++ /tmp/puppet-file20180305-43916-149xpky 2018-03-05 09:16:57.740678088 +0100 @@ -49,6 +49,7 @@ read-only = ON relay_log = /var/lib/mysql/data/relay-bin relay_log_index = /var/lib/mysql/data/relay-bin.index +replicate-wild-ignore-table = otrs.% report_host = foo.bar.org server_id = 13 slow_query_log = 1
I think it would be really nice to have some structure, so it would be easier to see changes.
For my own personal needs I think this project is pretty much complete.
Right now I can view my nodes, I can see their states, and I can look at what changed/failed. That's pretty much all I wanted. Compared to the original puppet-dashboard there is only one real omission:
The notion of classes/groups is missing, as is the ability to operate as a node-classifier, but that is by design. The only real issue I have here is that with a few hundred managed-nodes the front-page gets terribly long and hard to view.
Anyway if the documentation is lacking/incomplete, or you have feature-requests/questions, please do report a new issue :)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.