puppetlabs / puppet Goto Github PK
View Code? Open in Web Editor NEWServer automation framework and application
Home Page: https://puppet.com/open-source/#osp
License: Apache License 2.0
Server automation framework and application
Home Page: https://puppet.com/open-source/#osp
License: Apache License 2.0
I will just link to the JIRA ticket as this is now a (almost) 10 years old feature request which I'm just mirroring over to the new Github issue tracking process.
with puppet8 creating users with passwords fails:
With puppet8 creating users with passwords succeeds:
# puppet apply -e 'user { "foo01": password => Sensitive("bah") }'
/usr/share/ruby/vendor_ruby/augeas.rb:48: warning: undefining the allocator of T_DATA class Augeas
Notice: Compiled catalog for swlaptop01.example.com in environment production in 0.01 seconds
Warning: /User[foo01]: Unable to mark 'password' as sensitive: the property itself was not assigned a value.
Or the same with:
# puppet apply -e 'define u(Sensitive[String[1]] $pw) { user { $title: password => $pw } } u { "foo01": pw => Sensitive("bah") }'
And if you don't sensitive the password, you get this:
root@swlaptop01 ~ # puppet apply -e 'user { "foo01": password => "bah", ensure => present }'
/usr/share/ruby/vendor_ruby/augeas.rb:48: warning: undefining the allocator of T_DATA class Augeas
Notice: Compiled catalog for swlaptop01.example.com in environment production in 0.01 seconds
Notice: /Stage[main]/Main/User[foo01]/ensure: created
Notice: Applied catalog in 0.60 seconds
root@swlaptop01 ~ # grep foo /etc/shadow
foo01:!:19825:0:99999:7:::
Notice the empty password.
When downloading external resources via file in a master-less environment, puppet throws warnings as it is missing agent ssl files.
root@node:~ # /opt/puppetlabs/bin/puppet apply init.pp
Notice: Compiled catalog for node in environment production in 0.01 seconds
Warning: Private key for 'node' does not exist
Warning: Client certificate for 'node' does not exist
Notice: /Stage[main]/Main/File[/tmp/file]/ensure: defined content as '{mtime}2023-09-25 06:36:57 UTC'
Notice: Applied catalog in 0.10 seconds
No warning being thrown when downloading external resources over https.
root@node:~ # /opt/puppetlabs/bin/puppet apply init.pp
Notice: Compiled catalog for node in environment production in 0.01 seconds
Notice: /Stage[main]/Main/File[/tmp/file]/ensure: defined content as '{mtime}2023-09-25 06:36:57 UTC'
Notice: Applied catalog in 0.10 seconds
wget https://apt.puppetlabs.com/puppet8-release-bookworm.deb
dpkg -i puppet8-release-bookworm.deb
apt-get update
apt-get install puppet-agent
init.pp
with following content:file { '/tmp/file':
source => 'https://artifacts.elastic.co/GPG-KEY-elasticsearch',
}
/opt/puppetlabs/bin/puppet apply init.pp
We found this issue when we tried to download apt key (via apt::keyring) over https from external source. There is no issue when downloading over insecure http.
Ruby 3.3.0 was released Dec 25, 2023, but puppet cannot be loaded with the default set of gems, because racc
was moved from a default to bundled gem. More gems will be moved in 3.4, so warnings are generated for those.
It should be possible to load puppet on Ruby 3.3 without warnings
PS C:\Users\Administrator\puppet> bundle exec puppet --version
C:/Users/Administrator/puppet/lib/puppet/settings.rb:4: warning: getoptlong was loaded from the standard library, but will no longer be part of the default gems since Ruby 3.4.0. Add getoptlong to your Gemfile or gemspec.
C:/tools/ruby33/lib/ruby/gems/3.3.0/gems/fast_gettext-2.3.0/lib/fast_gettext/vendor/mofile.rb:171: warning: prime was loaded from the standard library, but is not part of the default gems since Ruby 3.1.0. Add prime to your Gemfile or gemspec. Also contact author of fast_gettext-2.3.0 to add prime into its gemspec.
C:/Users/Administrator/puppet/lib/puppet/pops/types/p_binary_type.rb:3: warning: base64 was loaded from the standard library, but will no longer be part of the default gems since Ruby 3.4.0. Add base64 to your Gemfile or gemspec.
cannot load such file -- racc/parser.rb
PS C:\Users\Administrator\puppet> git grep racc/parser
lib/puppet/pops/parser/eparser.rb:require 'racc/parser.rb'
PS C:\Users\Administrator\puppet> ruby --version
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x64-mingw-ucrt]
PS C:\Users\Administrator\puppet> bundle list
Gems included by the bundle:
* CFPropertyList (2.3.6)
* abbrev (0.1.2)
* addressable (2.8.6)
* apt_stage_artifacts (0.11.0)
* artifactory (3.0.15)
* concurrent-ruby (1.2.3)
* crack (0.4.5)
* csv (3.2.8)
* declarative (0.0.20)
* deep_merge (1.2.2)
* diff-lcs (1.5.0)
* digest-crc (0.6.5)
* docopt (0.6.1)
* facter (4.5.1)
* faraday (2.9.0)
* faraday-net_http (3.1.0)
* fast_gettext (2.3.0)
* ffi (1.15.5)
* google-apis-core (0.11.3)
* google-apis-iamcredentials_v1 (0.17.0)
* google-apis-storage_v1 (0.31.0)
* google-cloud-core (1.6.1)
* google-cloud-env (2.1.0)
* google-cloud-errors (1.3.1)
* google-cloud-storage (1.47.0)
* googleauth (1.9.1)
* hashdiff (1.1.0)
* hiera-eyaml (3.4.0)
* highline (3.0.0)
* hocon (1.4.0)
* httpclient (2.8.3)
* json-schema (2.8.1)
* jwt (2.7.1)
* locale (2.1.3)
* mini_mime (1.1.5)
* minitar (0.9)
* msgpack (1.7.2)
* multi_json (1.15.0)
* net-http (0.4.1)
* optimist (3.1.0)
* os (1.1.4)
* packaging (0.113.0)
* public_suffix (5.0.4)
* puppet (8.5.0)
* puppet-resource_api (1.9.0)
* puppetserver-ca (2.6.0)
* rake (13.1.0)
* release-metrics (1.1.0)
* representable (3.2.0)
* retriable (3.1.2)
* rexml (3.2.6)
* rspec (3.12.0)
* rspec-core (3.12.2)
* rspec-expectations (3.12.3)
* rspec-its (1.3.0)
* rspec-mocks (3.12.6)
* rspec-support (3.12.1)
* scanf (1.0.0)
* semantic_puppet (1.1.0)
* signet (0.18.0)
* thor (1.3.0)
* trailblazer-option (0.1.2)
* uber (0.1.0)
* uri (0.13.0)
* vcr (6.2.0)
* webmock (3.19.1)
* webrick (1.8.1)
* yard (0.9.34)
PS C:\Users\Administrator\puppet> bundle config list
Settings are listed in order of priority. The top value will be used.
On *nix, add the development
group:
❯ bundle config set with development
❯ bundle install
Fetching gem metadata from https://rubygems.org/.......
Fetching racc 1.5.2
Installing racc 1.5.2 with native extensions
Bundle complete! 33 Gemfile dependencies, 98 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
❯ bundle exec puppet --version
/home/josh/work/puppet/lib/puppet/settings.rb:4: warning: getoptlong was loaded from the standard library, but will no longer be part of the default gems since Ruby 3.4.0. Add getoptlong to your Gemfile or gemspec.
/home/josh/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fast_gettext-2.3.0/lib/fast_gettext/vendor/mofile.rb:171: warning: prime was loaded from the standard library, but is not part of the default gems since Ruby 3.1.0. Add prime to your Gemfile or gemspec. Also contact author of fast_gettext-2.3.0 to add prime into its gemspec.
/home/josh/work/puppet/lib/puppet/util/feature.rb:118: warning: syslog was loaded from the standard library, but will no longer be part of the default gems since Ruby 3.4.0. Add syslog to your Gemfile or gemspec.
/home/josh/work/puppet/lib/puppet/pops/types/p_binary_type.rb:3: warning: base64 was loaded from the standard library, but will no longer be part of the default gems since Ruby 3.4.0. Add base64 to your Gemfile or gemspec.
8.5.0
Note adding the development
group does not resolve the issue on Windows, because the racc
gem is confined to the ruby
platform:
gem "racc", "1.5.2", require: false, platforms: [:ruby]
This is mostly likely due to https://github.com/ruby/ruby/blob/a49643340eb6b8d65299912b640887ea18495909/NEWS.md#stdlib-compatibility-issues
You need to add racc to your Gemfile if you use racc under bundler environment.
The default install of puppet-agent on Debian 12 lacks augeas support.
See bottom of report for my workaround.
When running puppet agent -t
on a Debian 12 machine, I get the following error:
root@baz:~# puppet agent -t
Info: Using environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Error: Failed to apply catalog: Resource type 'Augeas' was not found
root@baz:~#
I have installed a brand new Debian 12 (bookworm) machine
(it is an EC2 instance in AWS, based on the official Debian 12 AMI from Debian).
Since the puppet-agent
.deb package now seem to be located in the official Debian APT repo, I have not added any custom APT source.
(comparison: on my Debian 11 machines I have /etc/apt/sources.list.d/puppet7.list : deb http://apt.puppetlabs.com bullseye puppet7)
echo 'APT::Install-Recommends "0";' > /etc/apt/apt.conf.d/01_no_recommends
apt update && apt -y full-upgrade
apt -y install puppet-agent
mkdir /var/lib/puppet
echo -e 'vardir = /var/lib/puppet\nssldir = /var/lib/puppet/ssl\nserver = puppet-master2.foo.bar' > /etc/puppet/puppet.conf
puppet ssl bootstrap
<I get a signed cerificate>
I declare the node baz.foo.bar in nodes.pp and assign a profile.
As long as the profile and manifest don't contain any augeas
calls, everything is working fine:
root@baz:~# puppet agent -t
Info: Using environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for baz.foo.bar
Info: Applying configuration version 'ea40e02a'
<...doing my changes...>
Notice: Applied catalog in 3.05 seconds
root@baz:~#
But if I add any augeas call - for example:
augeas { '/etc/default/puppet: enable puppet daemon':
changes => [
'set /files/etc/default/puppet/START yes',
],
}
...then puppet-agent immediately bail with "Error: Failed to apply catalog: Resource type 'Augeas' was not found". (see example run at the top).
Package: puppet-agent
Version: 7.23.0-1
I also tried this:
apt install augeas-tools
apt install puppet-module-puppetlabs-augeas-core
apt install puppet-module-camptocamp-augeas
apt install libaugeas-dev
reboot
...but it is still the same.
During my debugging I found that on my Deb11 machines, augeas seem to be installed here:
/opt/puppetlabs/puppet/vendor_modules/augeas_core/
Also I see that no .deb libs or packages are installed:
dpkg -l | grep aug
Also if I run puppet module list
it says no module installed:
/etc/puppetlabs/code/environments/production/modules (no modules installed)
/etc/puppetlabs/code/modules (no modules installed)
/opt/puppetlabs/puppet/modules (no modules installed)
...so I'm not sure how and where puppet agent actually manages to find the augeas functions.
Perhaps via puppet config print vendormoduledir
= /opt/puppetlabs/puppet/vendor_modules
?
Compared to the Deb12 machine:
Augeas seem to be installed here:
/usr/share/puppet/modules.available/puppetlabs-augeas-core/
Libs and packages are installed. Example from before I manually try to add additional packages:
root@baz:~# dpkg -l | grep aug
ii augeas-lenses 1.14.0-1 all Set of lenses needed by libaugeas0 to parse config files
ii libaugeas0:amd64 1.14.0-1 amd64 Augeas configuration editing library and API
ii ruby-augeas 1:0.5.0+gem-1 amd64 Augeas bindings for the Ruby language
root@baz:~#
...and after I added the additional packages:
root@baz:~# dpkg -l | grep aug
ii augeas-lenses 1.14.0-1 all Set of lenses needed by libaugeas0 to parse config files
ii augeas-tools 1.14.0-1 amd64 Augeas command line tools
ii libaugeas-dev:amd64 1.14.0-1 amd64 Development files for writing applications based on libaugeas0
ii libaugeas0:amd64 1.14.0-1 amd64 Augeas configuration editing library and API
ii puppet-module-camptocamp-augeas 1.9.0-2 all Puppet module for Augeas
ii puppet-module-puppetlabs-augeas-core 1.1.2-1 all Puppet module for Augeas Core
ii ruby-augeas 1:0.5.0+gem-1 amd64 Augeas bindings for the Ruby language
root@baz:~#
If I run puppet module list
it says the module installed, but apparently it is not found/used since I get that error.
root@baz:~# puppet module list
/usr/share/puppet/modules
└── puppetlabs-augeas_core (v1.1.2)
root@baz:~#
If I run puppet config print vendormoduledir
on Deb12 I get /usr/share/puppet/vendor_modules
.
Now, when writing this report, I thought: what if I put a symlink to that augeas_core module in the vendor_modules dir?
root@baz:~# cd /usr/share/puppet/vendor_modules
root@baz:/usr/share/puppet/vendor_modules# ln -s /usr/share/puppet/modules.available/puppetlabs-augeas-core
root@baz:/usr/share/puppet/vendor_modules# puppet agent -t--noop
Info: Using environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Applying configuration version '219979d0'
--- /etc/default/puppet 2023-02-10 15:54:26.000000000 +0100
+++ /etc/default/puppet.augnew 2023-12-07 15:54:41.901167922 +0100
@@ -2,3 +2,4 @@
# Startup options
DAEMON_OPTS=""
+START=yes
Notice: /Stage[main]/Profiles::Puppetnode/Augeas[/etc/default/puppet: enable puppet daemon]/returns: current_value 'need_to_run', should be 0 (noop)
Notice: Applied catalog in 1.94 seconds
root@baz:/usr/share/puppet/vendor_modules#
Wheee! It worked.
I leave it to you to find the correct solution to this, if this is a bug.
If not, should I have run some special command to "register" that available module?
Using an environment name composed only of numeric characters makes the catalog compilation fail.
The puppet documentation explains that environment names can contain lowercase letters, numbers and underscores. It does not prohibit the use of numeric characters only.
When using an environment like 42
, we get ruby stacktrace like this one:
2024-01-23T11:02:48.938+01:00 ERROR [qtp472974372-11877] [puppetserver] Puppet Server Error: undefined method `match' for 42:Integer
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/node/environment.rb:127:in `valid_name?'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:90:in `uri2indirection'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:35:in `call'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/server/v3.rb:17:in `block in wrap'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:82:in `block in process'
org/jruby/RubyArray.java:1865:in `each'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:81:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:88:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:88:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:86:in `block in process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:69:in `block in with_request_profiling'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:65:in `with_request_profiling'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:85:in `block in process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:92:in `respond_to_errors'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:84:in `process'
uri:classloader:/puppetserver-lib/puppet/server/master.rb:69:in `block in handleRequest'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:62:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:289:in `override'
uri:classloader:/puppetserver-lib/puppet/server/master.rb:68:in `handleRequest'
An environment name composed only of numeric characters should be valid.
Steps to reproduce the behavior:
puppet agent -t --environment 42
We are deploying environments from a Gitlab repository with r10k.
GitLab uses the default pattern %{id}-%{title}
when creating a branch from an issue.
If we want keep branch names equals to environment names (i.e. without hyphens), the easy solution is to replace the pattern with %{id}
only. Otherwise the spaces and special characters in %{title}
xould be converted to hyphens.
Using this workflows, we end up with some numeric only branches/puppet environments.
According to puppet documentation, this shouldn't have been a problem.
A pull-request to fix this in on the way.
When a type is incorrectly specified, e.g. Integer[1-3]
for a class parameter, Puppet's class_information_service ignores this error, and produces an empty type
specification.
puppet/lib/puppet/info_service/class_information_service.rb
Lines 76 to 80 in 298762f
A warning should be produced, and a default type should be assigned in the nil
case to prevent incorrect behavior. Note that this issue does not impact catalog compilation, but does impact the ability to get classes manifest information.
Impacts both 7.x and 8.x
In puppetlabs/puppetlabs-apt#1170, @beliys report an issue with a ASCII armored GPG key that appear as a binary file after download. The issue can be reproduced using this minimal example:
file { '/tmp/with-a-redirect':
source => 'https://packagecloud.io/timescale/timescaledb/gpgkey',
}
file { '/tmp/no-redirect':
# this URL is not stable, get a "fresh" one with:
# curl -sI https://packagecloud.io/timescale/timescaledb/gpgkey | awk '$1 == "location:" { print $2 }'
source => 'https://d3fo0g5hm7lbuv.cloudfront.net/9356/8817/gpg/timescale-timescaledb-E7391C94080429FF.pub.gpg?Expires=1712081971&Signature=Zsi2Zk5uchwGUGflDbFNncWN4Nj8tBhsgg2tho5CaEcAwJ3MpkphBxz9kSBAAr4zlyp45j8zPV8Cehc9HCWY0FwmrPiONjmibmCWXuF9DyluaH-qd2mL9WTQ3WOLMVoAEuEM8-u~JfBeANoxevR-hSGf7rMjgVH1HdseYkaRe8Tr729YE5GOkIkA~Xa93l~qLCuESTVw2-7P2BcWqd2c-4PAgdBBuqLzJfA9NUTOnLJqLGa76vn7rnGmhL2z9SSZSbrW7hOzRonMSbV3wgMfXflc98dXjq-qrRR4z7QWIAf6bPQf3OBcenGIXyk5Xo~2HsrhlGPP62gwANnvynNkWg__&Key-Pair-Id=K1ZMSF1EKC3AZL'
}
The URL in the first file source
redirects to the URL in the second file source
.
Both downloaded files should have the same content in the end, but it happens that the first one seems to save the gzip compressed response from the final request, while the second one save the actual decompressed file (as expected).
romain@zappy ~ % ls -l /tmp/with-a-redirect /tmp/no-redirect
-rw-r--r-- 1 romain wheel 3949 2 avr. 08:34 /tmp/no-redirect
-rw-r--r-- 1 romain wheel 3012 2 avr. 08:33 /tmp/with-a-redirect
romain@zappy ~ % file /tmp/with-a-redirect /tmp/no-redirect
/tmp/with-a-redirect: gzip compressed data, from Unix, original size modulo 2^32 3949
/tmp/no-redirect: PGP public key block Public-Key (old)
romain@zappy ~ % md5 /tmp/with-a-redirect /tmp/no-redirect
MD5 (/tmp/with-a-redirect) = 951e9f5f3c5b204750574dc8d93fa709
MD5 (/tmp/no-redirect) = fdd5cb16b7250bb41f2976215afb5ce0
romain@zappy ~ % gunzip < /tmp/with-a-redirect > /tmp/with-a-redirect-decompressed
romain@zappy ~ % md5 /tmp/with-a-redirect /tmp/no-redirect /tmp/with-a-redirect-decompressed
MD5 (/tmp/with-a-redirect) = 951e9f5f3c5b204750574dc8d93fa709
MD5 (/tmp/no-redirect) = fdd5cb16b7250bb41f2976215afb5ce0
MD5 (/tmp/with-a-redirect-decompressed) = fdd5cb16b7250bb41f2976215afb5ce0
The saved file should be the same, regardless of the presence of HTTP redirects.
See above example.
n/a
The rake syntax
task fails when using puppet 7.29.1 or 8.5.1 with non-literal class parameter types. This is because puppet reports the issue as warning
(in 7.29.1) and error
(in 8.5.1).
Additionally the message not a Puppet::Pops::Model::AccessExpression
is not helpful.
It should be possible to detect and report on non-literal class parameter types without causing rake syntax
to fail. The error message should also use the LabelProvider
to print a meaningful message like:
The parameter '$i' must be a literal type, not a '*' expression (line: 1, column: 18)
Given a manifest whose class parameter contains an arithmetic expression:
class foo(Integer[0, 2*2] $i = 0) {}
include foo
In 7.29.1, a warning is generated:
$ bundle exec puppet parser validate classes.pp
Warning: The parameter '$i' must be a literal type, not a Puppet::Pops::Model::AccessExpression (file: /home/josh/work/puppet/classes.pp, line: 1, column: 27)
$ echo $?
0
In 8.5.1, an error is generated:
$ bundle exec puppet parser validate classes.pp
Error: Could not parse for environment production: The parameter '$i' must be a literal type, not a Puppet::Pops::Model::AccessExpression (file: /home/josh/work/puppet/classes.pp, line: 1, column: 27)
$ echo $?
1
The puppet parser valdiate
command is called by the syntax
rake task and any warnings or greater cause the task to fail in both 7.29.1 and 8.5.1. The syntax
task is also called by rake validate
in puppetlabs_spec_helper.
See #9269
When running puppet agent provided with Raspberry Pi OS, the message "Warning: Found multiple default providers for service: init, systemd; using init". This causes some services such as firewalld
to be unmanageable.
Puppet uses the correct Service provider, systemd
puppet
packagepuppet agent -t
I don't know enough about Debian to know if this is the case in all Debian platforms.
For "reasons" our configuration refererence docs are often incorrect, especially when a setting's default value is different between 7.x and 8.x. For example, the default value of strict_variables
is correct in 7.x, but wrong in 8.x.
The reference documentation should be accurate for each branch.
I would like to see the configuration reference pages generated in CI and checked into the repo, similar to how we handle man pages. That way the list of settings and default values are guaranteed to be accurate for each branch.
This is mostly an issue with the configuration reference, but also includes other documentation like the type & provider docs, which providers support which features, etc. Other changes may be required to puppet-strings for that. See PDOC-262
https://perforce.atlassian.net/browse/PUPDOC-5594
https://perforce.atlassian.net/browse/PUPDOC-5591
https://perforce.atlassian.net/browse/PUPDOC-5562
https://perforce.atlassian.net/browse/PUPDOC-5491
https://perforce.atlassian.net/browse/PUPDOC-5481
puppetlabs/puppet-docs#1166
We recently added "notice" level messages specifying the server that the agent is requesting a catalog from and the server that actually handled the request, e.g. Notice: Requesting catalog from puppet.example.com:8140.
See #9126
As described in #9223, the agent may hang for 2 minutes trying to resolve the DNS name into an IPv6 address. Other users have reported not wanting these messages to be printed (or at least not at the Notice
level).
Add a boolean puppet setting to allow the DNS resolution to be disabled. It should be enabled by default.
Creating an enum to allow the behavior to be disabled or logged at a different verbosity level.
https://puppetcommunity.slack.com/archives/C0W298S9G/p1706879255760149
https://puppetcommunity.slack.com/archives/C0W298S9G/p1705964713457719
We should remove the gem metadata contained in ext/project_data.yaml
for the
puppet gem, instead use the gemspec as the source of truth
when building it.
Puppet (and Facter's) gemspec files are only used when running under bundler,
but not when building the gems!! Since there are multiple sources of truth, we
have frequently updated one but not the other. Most recently, the puppet 8.0.0
gem listed the wrong minimum required ruby version, forcing us to release 8.0.1
One complexity is we publish platform-specific gems for puppet:
https://rubygems.org/downloads/puppet-8.5.1.gem
https://rubygems.org/downloads/puppet-8.5.1-x86-mingw32.gem
https://rubygems.org/downloads/puppet-8.5.1-x64-mingw32.gem
https://rubygems.org/downloads/puppet-8.5.1-universal-darwin.gem
The reason is because puppet depends on CFPropertyList on macOS and ffi on
Windows. By publishing platform specific gems, we can add runtime dependencies
that only affect that platform.
You might be asking why not just add CFPropertyList and ffi for all platforms?
The reason is those gems contain native extensions, and so would require a
compiler and ruby-dev packages to be installed in order to gem install puppet
.
Technically, CFPropertyList itself doesn't contain native extensions, but it has
at various times had a runtime dependency on nokogiri, which does.
Move and reconcile all of the common metadata from ext/project_data.yaml
to
the gemspec, such as author, description, required ruby version, etc.
Move .gemspec
back to puppet.gemspec
, since it should be used to build the
gem, see commit 8f81b522a98472
For the platform-specific gems, create corresponding
puppet-<platform>.gemspec
files. Each gemspec can append platform specific
dependencies like:
$ cat puppet-x64-mingw32.gemspec
spec = Gem::Specification.load('puppet.gemspec')
spec.platform = 'x64-mingw32'
spec.add_runtime_dependency('ffi', '1.15.5')
spec
Modify our build process to build the generic gem:
gem build <project>.gemspec
Optionally, build platform specific gems in a data driven way. In other
words, the build automation should not have a list of platforms to build. It
should be based on the names of the gemspec files. For example, we could
conceivably have different platform-specific gemspec in 7.x vs main branches.
for file in <project>-*.gemspec
do
gem build $file
done
One downside of this approach is Gemfile
will need to contain conditional logic to include the right gemspec
file, taking into account the bundler and rubygems use different platform names:
source ENV['GEM_SOURCE'] || "https://rubygems.org"
case platform
when 'x64-mingw32'
gemspec(name: 'puppet-x64-mingw32.gemspec')
...
else
gemspec
end
It's possible to add conditional logic in the gemspec, to include additional runtime
dependencies based on the platform we're packaging for:
Gem::Specification.new do |spec|
...
# something like
if Gem::Platform.match('x64-mingw32')
spec.add_runtime_dependency('ffi', '1.15.5')
end
end
And then specify the platform to build for gem build --platform x64-mingw32 puppet.gemspec
. One nice thing is the Gemfile
would not need to change. However, the list of the platforms would need to be stored somewhere, and could be different for 7.x vs main.
1342839
2707fd8
https://puppet.atlassian.net/browse/PA-1077
https://puppet.atlassian.net/browse/PA-2058
https://puppet.atlassian.net/browse/PUP-8685
I tried to convert an exec
resource from multiple invocations of test -f
to use the built-in creates
option, but it doesn't work as expected, it will not execute unless none of the files are present.
Original:
exec {'generate snakeoil certificate':
command => "/usr/bin/sscg -q \
--cert-file /etc/pki/tls/certs/localhost.crt \
--cert-key-file /etc/pki/tls/private/localhost.key \
--ca-file /etc/pki/tls/certs/localhost.crt \
--dhparams-file /tmp/dhparams.pem \
--lifetime 3650 \
--hostname ${facts['networking']['fqdn']} \
--email root@${facts['networking']['fqdn']}",
unless => [
'/usr/bin/test -f /etc/pki/tls/certs/localhost.crt',
'/usr/bin/test -f /etc/pki/tls/private/localhost.key',
],
notify => Service['httpd'],
}
Expected equivalent:
exec {'generate snakeoil certificate':
command => "/usr/bin/sscg -q \
--cert-file /etc/pki/tls/certs/localhost.crt \
--cert-key-file /etc/pki/tls/private/localhost.key \
--ca-file /etc/pki/tls/certs/localhost.crt \
--dhparams-file /tmp/dhparams.pem \
--lifetime 3650 \
--hostname ${facts['networking']['fqdn']} \
--email root@${facts['networking']['fqdn']}",
creates => [
'/etc/pki/tls/certs/localhost.crt',
'/etc/pki/tls/private/localhost.key',
],
notify => Service['httpd'],
}
Debug output if both exist:
Info: Applying configuration version '1700663411'
Debug: /Stage[main]/Main/Exec[generate snakeoil certificate]/creates: Checking that 'creates' path '/etc/pki/tls/certs/localhost.crt' exists
Debug: /Stage[main]/Main/Exec[generate snakeoil certificate]: '/usr/libexec/httpd-ssl-gencerts' won't be executed because of failed check 'creates'
Debug: Finishing transaction 12600
Debug output if the first file is missing and the second one exists:
Info: Applying configuration version '1700663438'
Debug: /Stage[main]/Main/Exec[generate snakeoil certificate]/creates: Checking that 'creates' path '/etc/pki/tls/certs/localhost.crt' exists
Debug: /Stage[main]/Main/Exec[generate snakeoil certificate]/creates: Checking that 'creates' path '/etc/pki/tls/private/localhost.key' exists
Debug: /Stage[main]/Main/Exec[generate snakeoil certificate]: '/usr/libexec/httpd-ssl-gencerts' won't be executed because of failed check 'creates'
Debug: Finishing transaction 12600
Debug output if neither file exists (debug run without notify httpd
):
Info: Applying configuration version '1700663608'
Debug: /Stage[main]/Main/Exec[generate snakeoil certificate]/creates: Checking that 'creates' path '/etc/pki/tls/certs/localhost.crt' exists
Debug: /Stage[main]/Main/Exec[generate snakeoil certificate]/creates: Checking that 'creates' path '/etc/pki/tls/private/localhost.key' exists
Debug: Exec[generate snakeoil certificate](provider=posix): Executing '/usr/libexec/httpd-ssl-gencerts'
Debug: Executing: '/usr/libexec/httpd-ssl-gencerts'
Notice: /Stage[main]/Main/Exec[generate snakeoil certificate]/returns: executed successfully
Debug: /Stage[main]/Main/Exec[generate snakeoil certificate]: The container Class[Main] will propagate my refresh event
Debug: Class[Main]: The container Stage[main] will propagate my refresh event
Debug: Finishing transaction 12600
I would expect the command to be executed if any of the files listed in creates
are missing.
Originally posted by samuraiii December 2, 2023
Hi all,
there is a regression for puppet-agent on gentoo linux, presumably between versions >8 and 8<=.
The 7.X version was detecting two inits on my box but was choosing the right every time (systemd).
Version 8.2.0 detects also both inits but chooses the "init" one, which breaks all systemd related facts.
I am not sure where to post this since tickets are obsolete, so I am sorry for possible misposting.
Cheers
S
When using the 'dnfmodule' provider, the following command is run: "/usr/bin/dnf module list -d 0 -e 1". Because dnf module does not seem to use rpm imported gpg keys, the system prompts to trust the key.
A puppet run should not fail due to a prompt for trusting the gpg key. Perhaps adding the option "--assumeyes" or "-y" is needed.
Steps to reproduce the behavior:
package {'php-module':
name => 'php',
ensure => "remi-8.1",
provider => dnfmodule,
require => Package['remi-release'],
}
This was originally reported in https://puppet.atlassian.net/browse/PUP-11446.
Ways to reproduce -
$ echo '{"mystructuredfact": { "inner_key": null } }' > /etc/facter/facts.d/myfact.json
When that is sent to PuppetDB, queries such as "inventory[facts.mystructuredfact.inner_key]" will return "" instead of null.
Further details are in ticket PUP-11446.
The puppet catalog download
command produces a 500 error on server using the default options:
# puppet catalog download
Notice: Requesting catalog from puppetserver:8140 (xxx.xxx.xxx.xxx)
Error: Could not call 'find' on 'catalog': Error 500 on SERVER: Server Error: Could not intern from json: undefined method `[]' for nil:NilClass
Error: Could not call 'find' on 'catalog': Error 500 on SERVER: Server Error: Could not intern from json: undefined method `[]' for nil:NilClass
Error: Try 'puppet help catalog download' for usage
A catalog is downloaded
Steps to reproduce the behavior:
puppet catalog download
The catalog rest terminus expects a facts_for_catalog
option to be present in the request:
puppet/lib/puppet/face/catalog.rb
Line 140 in 1d188e1
puppet catalog download
documentation, it seems like the catalog rest terminus supported providing no facts at some point, which would presumably use facts from PuppetDB instead:puppet/lib/puppet/face/catalog.rb
Lines 130 to 132 in 1d188e1
facts_for_catalog
option is nil as shown in the error above. Either the catalog rest terminus should allow providing no facts:
diff --git a/lib/puppet/http/service/compiler.rb b/lib/puppet/http/service/compiler.rb
index 1ead5248e3..9b8731cc86 100644
--- a/lib/puppet/http/service/compiler.rb
+++ b/lib/puppet/http/service/compiler.rb
@@ -79,7 +79,7 @@ class Puppet::HTTP::Service::Compiler < Puppet::HTTP::Service
# the server
#
# @api public
- def post_catalog(name, facts:, environment:, configured_environment: nil, check_environment: false, transaction_uuid: nil, job_uuid: nil, static_catalog: true, checksum_type: Puppet[:supported_checksum_types])
+ def post_catalog(name, facts: nil, environment:, configured_environment: nil, check_environment: false, transaction_uuid: nil, job_uuid: nil, static_catalog: true, checksum_type: Puppet[:supported_checksum_types])
if Puppet[:preferred_serialization_format] == "pson"
formatter = Puppet::Network::FormatHandler.format_for(:pson)
# must use 'pson' instead of 'text/pson'
@@ -93,8 +93,6 @@ class Puppet::HTTP::Service::Compiler < Puppet::HTTP::Service
# query parameters are sent in the POST request body
body = {
- facts_format: facts_format,
- facts: Puppet::Util.uri_query_encode(facts_as_string),
environment: environment,
configured_environment: configured_environment || environment,
check_environment: !!check_environment,
@@ -102,7 +100,12 @@ class Puppet::HTTP::Service::Compiler < Puppet::HTTP::Service
job_uuid: job_uuid,
static_catalog: static_catalog,
checksum_type: checksum_type.join('.')
- }.map do |key, value|
+ }
+ unless facts.nil?
+ body[:facts_format] = facts_format
+ body[:facts] = Puppet::Util.uri_query_encode(facts_as_string)
+ end
+ body = body.map do |key, value|
"#{key}=#{Puppet::Util.uri_query_encode(value.to_s)}"
end.join("&")
Or the puppet catalog download face should be modified to include facts in the request, e.g.
diff --git a/lib/puppet/face/catalog.rb b/lib/puppet/face/catalog.rb
index 3a7ea1a975..9ab47bcdf2 100644
--- a/lib/puppet/face/catalog.rb
+++ b/lib/puppet/face/catalog.rb
@@ -137,7 +137,8 @@ Puppet::Indirector::Face.define(:catalog, '0.0.1') do
Puppet::Resource::Catalog.indirection.cache_class = nil
catalog = nil
retrieval_duration = thinmark do
- catalog = Puppet::Face[:catalog, '0.0.1'].find(Puppet[:certname])
+ facts = Puppet::Face[:facts, '0.0.1'].find(Puppet[:certname])
+ catalog = Puppet::Face[:catalog, '0.0.1'].find(Puppet[:certname], { extra: { facts_for_catalog: facts } })
end
catalog.retrieval_duration = retrieval_duration
catalog.write_class_file
Hello,
Debian 12 (Bookworm) stable release was published in July 2023, however there are still no Puppet packages released at https://apt.puppetlabs.com/ or https://nightlies.puppet.com/apt/index.html (I'm referring to Puppetserver, Puppet Agent, PDK, and Bolt packages).
While Debian 12 ships their packages for Puppet 7, but it would be beneficial for all Debian users to have Puppet 8 available from the official Puppetlabs APT repository.
Other's have asked the same questions a while back, but no updates were provided about Debian 12 packages so far.
Cheers!
Calling puppet module install
or any other code that internally uses Puppet::HTTP::Client
may break with obscure invalid encoding:"783: unexpected token at '\x1F\x8B\b'"
or similar errors if HTTP redirects are involved and the final server supports HTTP compression.
This is caused by Puppet::HTTP::Redirector
internally used by Puppet::HTTP::Client
to follow HTTP redirects. The redirect logic copies all request headers from the original Net::HTTPRequest
object to a new one, including an Accept-Encoding
header that was implicitly added by Net::HTTP
during the first request.
Net::HTTP
supports HTTP compression. It will automatically add an Accept-Encoding
header and later decompress the response body, but only if that header was not already set by the caller. Puppet::HTTP::Redirector
adds this header explicitly, disabling the transparent compression handling in Net::HTTP
and resulting in a response body that may contain gzip compressed binary data instead of the expected result.
Feeding gzip compressed binary data into a JSON decoder triggers the unexpected token at '\x1F\x8B\b'
error from above.
HTTP redirects should be handled gracefully and not cause obscure errors.
Steps to reproduce the behavior:
puppet module install --module_repository http://forge.puppetlabs.com puppetlabs-stdlib
We have node manifests of the following form:
node /^service-(server|client)(?:-test)?-(\d+)/ {
$myservice_role=$1
$myservice_id=$2
case $myservice_role {
'server': {
# do one set of things
}
'client': {
# do some different things
}
<...etc...>
}
We have observed these nodes randomly swapping server and client behavior, as though their pattern match was for the wrong hostname. This only occurs with threading enabled.
A host's sense of its hostname should not be transposed with that of another host. This may in fact be a problem with regexp backreferences generally.
We're able to quickly reproduce the issue for debugging using ~20 hosts, all matching the node pattern, kicking off agent runs continuously (e.g. while /bin/true;do puppet agent --test;done
). We use the following manifest to capture the misbehavior:
node /^thing-(foo|bar)(?:-test)?-(\d+)/ {
$first_blob=$1
$second_blob=$2
if $::hostname =~ /^thing-(foo|bar)(?:-test)?-(\d+)/ {
if $1==$first_blob and $2==$second_blob {
# all is good, just notify
notify { 'regex_test_debug':
message => "hostname=${::hostname} first_blob=${first_blob} second_blob=${second_blob}"
}
} else {
# fail loudly
fail ("regex_test_debug: \$1=$1 \$2=$2 but \$first_blob=$first_blob and \$second_blob=$second_blob")
}
} else {
fail ("regex_test_debug: $::hostname does not match node regex")
}
}
This gives us appropriate fail
log entries whenever the node pattern match results do not correspond to the hostname:
2024-04-24T16:17:53.402-04:00 ERROR [qtp943675003-391] [puppetserver] Puppet Evaluation Error: Error while evaluating a Function Call, regex_test_debug: $1=bar $2=8 but $first_blob=foo and $second_blob=5 (file: /export/home/puppet/environments/production/manifests/regex_test.pp, line: 22, column: 7) on node thing-bar-8.example
I believe I've identified where these nodes step on one another during concurrent agent runs, however the fix isn't as trivial as adding a sempahore[1].
In Resource::Type
(lib/puppet/resource/type.rb
) it seems that all nodes which match a regexp node
pattern share a common instance of this class. In my test case above we find it running around with the name __node_regexp__thing-foobar:-test-d
, and concurrent agent runs report the same Ruby object_id.
This object is slightly stateful, in that the @match
attribute seems to contain the most recent string matched against the name pattern. We observe a 1:1 correlation between the state of this variable and what the agent run associated with multiple relevant threads thinks the hostname pattern match object was supposed to contain. I'm not quite sure how it propagates back to the DSL, but it seems likely that this is where it's coming from.
The correct fix isn't immediately obvious, but is probably of one of the following forms:
@match
between the time it's set and the time a node is done with it. I am not sure how to definitively tell when it would be safe to unlock.@match
based on the source string. This would mean that calling objects would need to be able to identify what pattern they mean by the string that matched the pattern, and I haven't found an obvious attribute to use for the purpose.Resource::Type
objects for individual agentsAny approach I've thought of is nontrivial and should have some input from someone more expert in this codebase.
Our situation manifesting this bug is peculiar to the use of backreferences in node
blocks in the DSL, but I can see it impacting anything that uses a pattern and results in a Resource::Type
object being created.
...oh, and lest I forget, calls to #evaluate_code
can run concurrently on the same Resource::Type
object. This may (I am not sure and cannot prove it) cause recycled MatchScope
objects downstream of both scope.ephemeral_from
and code.safeevaluate
to get overwritten across threads. Comments imply that those objects do get reused, but I haven't actually seen this to be the case so I'm not sure this is an issue. This aspect would be trivial to solve with a semaphore, if indeed it is a problem.
[1] Technically if I wrap the whole of Parser::Compiler#evaluate_ast_node
in an environment.lock.synchronize
block it makes the problem go away. However this seems like too high-level a bottleneck. The correct fix is probably to Resource::Type
.
When sourcing files from an http(s) source, in order to check if the file is already present Puppet, searches for these headers:
X-Checksum-Sha256
X-Checksum-Sha1
X-Checksum-Md5
Content-MD5
Last-Modified
Some servers (such as Gitlab) do not provide any of these headers, but they do provide an Etag
header, which indicates the version of the resource that is going to be served.
Modify the code responsible for retrieving the metadata from http(s) resources (I think is this) to take into account the Etag
header with more priority than Last-Modified
since it seems that Last-Modified
should be considered a fallback when there is no Etag
(as stated in MDN)
Use other resource to download files, but I have not found any module in the forge that uses Etag
as metadata of the version of the file.
N/A
When a group resource which has not been created(new), and is add/managing member to the group on creation "fails" to purge the group if provider is not explicitly set to 'groupadd'.
It seems that "groupmod" is selected for the localmodify command when defining the purge_members and results in error as groupmod does not have a "-m" option.
A new group resource should be created without need to explicitly set provider.
Steps to reproduce the behavior:
Make sure this is a new group that does not exist on system.
Behavior will not appear if group exists.
$members = [ 'larry', 'moe', 'curly' ]
group { 'sftponly':
ensure => 'present',
auth_membership => true,
members => $members,
forcelocal => true,
}
results in
Debug: Executing: '/usr/sbin/groupadd sftponly'
Debug: Success: /usr/sbin/groupadd sftponly
Debug: Executing: '/usr/bin/gpasswd -a larry sftponly'
Debug: Success: /usr/bin/gpasswd -a larry sftponly
...
Debug: Executing: '/usr/sbin/groupmod -m larry moe curly sftponly'
Error: Could not set members on group[sftponly]: Execution of '/usr/sbin/groupmod -m larry moe curly sftponly' returned 2: /usr/sbin/groupmod: invalid option -- 'm'
A work around is to explicitly set provider => 'groupdadd'
$members = [ 'larry', 'moe', 'curly' ]
group { 'sftponly':
ensure => 'present',
auth_membership => true,
members => $members,
forcelocal => true,
provider => 'groupadd',
}
results in
Debug: Executing: '/usr/sbin/lgroupadd sftponly'
Debug: Executing: '/usr/sbin/lgroupmod -M larry moe curly sftponly'
For a project, we operate approx. 40 servers of exact same hardware specifications. On most of them, puppet runs stable. But on several others, puppet seems to be instable and throws random errors. We do not have any clue, what the problem could be.
Puppet agent should not throw random errors.
To reproduce, we can just use any simple puppet agent command and execute it multiple times after one another.
e.g. configprint:
root@host ~ # puppet agent --configprint ssldir
Error: Could not initialize global default settings: can't modify frozen File: #<File:/opt/puppetlabs/puppet/lib/ruby/vendor_gems/gems/hocon-1.3.1/lib/hocon/impl/config_node_simple_value.rb>
root@host ~ # puppet agent --configprint ssldir
/etc/puppetlabs/puppet/ssl
root@host ~ # puppet agent --configprint ssldir
/etc/puppetlabs/puppet/ssl
root@host ~ # puppet agent --configprint ssldir
/etc/puppetlabs/puppet/ssl
root@host ~ #
root@host ~ # puppet agent --configprint ssldir
Traceback (most recent call last):
21: from /opt/puppetlabs/puppet/bin/puppet:4:in `<main>'
20: from /opt/puppetlabs/puppet/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
19: from /opt/puppetlabs/puppet/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
18: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:12:in `<top (required)>'
17: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:12:in `require_relative'
16: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:335:in `<top (required)>'
15: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:335:in `require_relative'
14: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser.rb:6:in `<top (required)>'
13: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser.rb:6:in `require_relative'
12: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:8:in `<top (required)>'
11: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:8:in `require_relative'
10: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops.rb:1:in `<top (required)>'
9: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops.rb:120:in `<module:Puppet>'
8: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/model/ast.rb:4669:in `register_pcore_types'
7: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loaders.rb:111:in `static_loader'
6: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/pcore.rb:65:in `init'
5: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/types.rb:61:in `register_ptypes'
4: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/types.rb:61:in `each'
3: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/types.rb:65:in `block in register_ptypes'
2: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/types.rb:760:in `register_ptype'
1: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/types.rb:53:in `create_ptype'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/pcore.rb:103:in `create_object_type': can't modify frozen Hash: {} (FrozenError)
Example: confdir
root@host ~ # puppet agent --configprint confdir
/etc/puppetlabs/puppet
root@host ~ # puppet agent --configprint confdir
/etc/puppetlabs/puppet
root@host ~ # puppet agent --configprint confdir
Traceback (most recent call last):
27: from /opt/puppetlabs/puppet/bin/puppet:4:in `<main>'
26: from /opt/puppetlabs/puppet/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
25: from /opt/puppetlabs/puppet/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
24: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:12:in `<top (required)>'
23: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:12:in `require_relative'
22: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:335:in `<top (required)>'
21: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:335:in `require_relative'
20: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser.rb:6:in `<top (required)>'
19: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser.rb:6:in `require_relative'
18: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:8:in `<top (required)>'
17: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:8:in `require_relative'
16: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops.rb:1:in `<top (required)>'
15: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops.rb:120:in `<module:Puppet>'
14: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/model/ast.rb:4669:in `register_pcore_types'
13: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loaders.rb:111:in `static_loader'
12: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/pcore.rb:65:in `init'
11: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/types.rb:68:in `register_ptypes'
10: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/types.rb:68:in `each'
9: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/types.rb:68:in `block in register_ptypes'
8: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/p_meta_type.rb:48:in `resolve'
7: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/p_object_type.rb:738:in `_pcore_init_from_hash'
6: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/p_object_type.rb:738:in `map'
5: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/p_object_type.rb:738:in `each'
4: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/p_object_type.rb:744:in `block in _pcore_init_from_hash'
3: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/p_object_type.rb:744:in `new'
2: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/p_object_type.rb:286:in `initialize'
1: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/type_asserter.rb:33:in `assert_instance_of'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/type_asserter.rb:41:in `report_type_mismatch': Puppet::Pops::Types::TypeAssertionError
root@host ~ # puppet agent --configprint confdir
/etc/puppetlabs/puppet
or even only version output:
root@host ~ # puppet agent --version
7.30.0
root@host ~ # puppet agent --version
Traceback (most recent call last):
11: from /opt/puppetlabs/puppet/bin/puppet:4:in `<main>'
10: from /opt/puppetlabs/puppet/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
9: from /opt/puppetlabs/puppet/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
8: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:12:in `<top (required)>'
7: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:12:in `require_relative'
6: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:41:in `<top (required)>'
5: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:129:in `<module:Puppet>'
4: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/defaults.rb:231:in `initialize_default_settings!'
3: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/settings.rb:1019:in `define_settings'
2: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/settings.rb:1019:in `each'
1: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/settings.rb:1026:in `block in define_settings'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/settings.rb:763:in `newsetting': can't modify frozen Hash: {:default=>"", :desc=>"A command to run before every agent run. If this command returns a non-zero\\n return code, the entire Puppet run will fail.", :name=>:prerun_command, :section=>:main} (FrozenError)
Version 7.30.0
and Version 7.23.0
Debian Bookworm
and also on some servers running on Debian Bullseye
.Linux host 6.5.13-5-pve #1 SMP PREEMPT_DYNAMIC PMX 6.5.13-5 (2024-04-05T11:03Z) x86_64 GNU/Linux
Linux host 6.5.11-8-pve #1 SMP PREEMPT_DYNAMIC PMX 6.5.11-8 (2024-01-30T12:27Z) x86_64 GNU/Linux
Nothing.
Using a negative value with the Integer type assertion on a class causes a rather surprising pops error.
Example code which triggers the error:
Integer[-1] $slowlog_log_slower_than = 10000,
from
causes
error during compilation: The parameter '$slowlog_log_slower_than' must be a literal type, not a Puppet::Pops::Model::AccessExpression (file: /home/jhoblitt/github/lsst-control/spec/fixtures/modules/redis/manifests/init.pp, line: 432, column: 15) on node foreman.cp.lsst.org
I expect my battleship to float!
Steps to reproduce the behavior:
Integer[<a filthy no good negative integer>]
When running Puppet agent 7.28 interactively (e.g. puppet agent --test --noop
) on a host with an IPv6 address but without an IPv4 address, the run consistently hangs for over two minutes after printing Info: Loading facts
and before printing Notice: Requesting catalog from puppet.example.com:8140
.
When running on a dual stack host with both IPv4 and IPv6 addresses, there is no pause and the Notice
includes an IPv4 address in parentheses: Notice: Requesting catalog from puppet.example.com:8140 (10.0.0.1)
.
There should be no delay when running Puppet on hosts with only an IPv6 address.
On an IPv6 only host that uses a Puppet server with both IPv4 and IPv6 addresses, run puppet agent --test --noop
.
The Puppet server is dual stack with routable IPv4 and IPv6 addresses. These are in DNS and can be resolved from the client system using the system resolver as normal.
This delay was not present in previous versions of Puppet. Reverting Puppet agent to 7.27.0 resolves the problem.
Looking at puppetlabs/puppet#9126, it appears that the lookup using Resolv.getaddress
is causing the delay.
Please note that all hostnames and IP addresses have been redacted and replaced in the examples below, but otherwise they are reproduced exactly as we ran them.
Using Ruby 2.7.8 on a dual stack host:
$ ruby --version; time ruby -r resolv -e 'puts Resolv.getaddress("puppet.example.com")'
ruby 2.7.8p225 (2023-03-30 revision 1f4d455848) [x86_64-linux]
10.0.0.1
real 0m0.162s
user 0m0.137s
sys 0m0.024s
Using Ruby 2.7.8 on an IPv6 only host:
$ ruby --version; time ruby -r resolv -e 'puts Resolv.getaddress("puppet.example.com")'
ruby 2.7.8p225 (2023-03-30 revision 1f4d455848) [x86_64-linux]
Traceback (most recent call last):
2: from -e:1:in `<main>'
1: from /opt/rbenv/versions/2.7.8/lib/ruby/2.7.0/resolv.rb:44:in `getaddress'
/opt/rbenv/versions/2.7.8/lib/ruby/2.7.0/resolv.rb:94:in `getaddress': no address for puppet.example.com (Resolv::ResolvError)
real 2m40.428s
user 0m0.182s
sys 0m0.056s
Using Ruby 3.2.2 on an IPv6 only host does return a result, but it's an IPv4 address which is arguably not what you would expect, so I suspect this issue is less visible in Puppet 8.x but may represent a related but slightly different bug:
$ ruby --version; time ruby -r resolv -e 'puts Resolv.getaddress("puppet.example.com")'
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux]
10.0.0.1
real 0m0.191s
user 0m0.152s
sys 0m0.039s
Noting that if you use the Resolv
gem under Ruby 2.7.8 rather than the builtin, this works as per 3.2.2 which would seem to be what you'd expect:
$ cat Gemfile; ruby --version; time ruby -r 'bundler/setup' -r resolv -e 'puts Resolv.getaddress("puppet.example.com")'
source "https://rubygems.org"
gem "resolv", "0.3.0"
ruby 2.7.8p225 (2023-03-30 revision 1f4d455848) [x86_64-linux]
10.0.0.1
real 0m0.328s
user 0m0.296s
sys 0m0.031s
dnfmodule provider can (un)install DNF profiles. But it has some limitations:
dnfmodule
provider - profiles, instead of packagesSo I'd like to propose, and try to implement, these changes:
flavor
parameter accept arrays, so to (un)install multiple profilesensure
* - as opposed to installed
, absent
, latest
, and so ondnfmodule
provider so to make all functioning clear* puppet-yum module will soon support full DNS stream management, so this could be deprecated altogether from dnfmodule
provider
I discussed this at length in Puppet Slack, with ideas ranging from implementing stuff in Puppet core, puppet-yum module and a dedicated module. We ended up agreeing in managing profiles (un)installation in core package
type (which is the official (un)installer), requiring the changes proposed in this report, and moving DNF streams management (which is a configuration, not an (un)installation) to puppet-yum module. I'm still open to ideas.
The function ModuleDataProvider::validate_data_hash
is supposed to prune all hiera keys that are not prefixed with the configured module_name
. It does not.
Instead it returns the hash that it was called with unchanged. All pruning is done on a cloned version of the supplied hash that’s discarded and not returned to the caller.
Puppet.warning("Module '#{module_name}': #{msg}, key=#{k}")
to help identify the offending key(s).Delete this line and the pruned hash is returned:
https://github.com/puppetlabs/puppet/blob/main/lib/puppet/pops/lookup/module_data_provider.rb#L57
This issue was detected when using Puppet Bolt. The underlying version of Puppet, used by Bolt, was 7.20.0
. Note that this is a bug with Puppet, not with Bolt.
When this issue is fixed it might well result in other bugs in how Bolt handles such Hiera key and their lookups as Bolt doesn't prefix key names with the module_name
as the module_name
IS the Bolt project name itself.
If you accidentally include binary data in the catalog, then the problem is difficult to troubleshoot as the error doesn't specify which environment/module/resource/parameter is causing the issue.
Puppetserver 8 will fail compilation and its log will contain:
2024-02-14T20:26:59.001Z ERROR [qtp1784649573-50] [puppetserver] Puppet Server Error: Failed to serialize Puppet::Resource::Catalog for 'XXX': Could not render to Puppet::Network::Format[rich_data_json]: source sequence is illegal/malformed utf-8
which is surfaced on the agent as:
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Failed to serialize ...
In puppet7, puppetserver will silently downgrade to PSON (though this can be disabled with allow_pson_serialization
setting). The agent will deserialize the catalog as PSON, and then report a warning when trying to cache the catalog as JSON:
# puppet agent -t
...
Info: Unable to serialize catalog to json, retrying with pson. PSON is deprecated and will be removed in a future release
There are two variations to this problem. First, the string needs to be "labeled" as UTF-8, so String#encoding
should return UTF-8
and not ASCII_8BIT
(aka BINARY
). Second, the string needs to be valid UTF-8, so String.valid_encoding?
must be true. The latter case, can easily occur when using the file
function instead of binary_file
.
If binary data is accidentally introduced into the catalog and is not wrapped in Binary
, as can occur when using the file
function, then the compilation should fail indicating which resource caused the issue.
Steps to reproduce the behavior:
binary_content = "\xC0\xFF".force_encoding('binary')
File.binwrite('/tmp/src.bin', binary_content)
# cat <<END > /etc/puppetlabs/code/environments/production/manifests/site.pp
file { '/tmp/dst.bin':
ensure => file,
content => file('/tmp/src.bin'),
}
END
#9102
https://puppet.atlassian.net/browse/PUP-10096
https://puppetcommunity.slack.com/archives/C0W298S9G/p1707246678595899
On a Solaris system, running "pkg update -n" in the global zone at the same time that it is running in a non-global zone, can cause the command to fail in one or the other.
If we have puppet running this command, it causes the puppet run to fail for that specific resource.
On a T7-4 with more than 70 non-global zones, this happens, ... a lot.
insync? to return either true or false without having unexpected errors in the package subsystem.
Not so much steps as having an environment with a lot of non-global zones with puppet running on the global zone AND the non-global zones. If we find ourselves running pkg update -n in more than one place at once, one of them will fail.
The existence of puppet running in no-global zones as well as the global zone. I've been running something that addresses this now for about a year.
gMSAs are domain accounts used to run services, and are a more secure alternative than traditional managed service accounts for running services or other automation.
When attempting to provide a gMSA as the logonaccount for a service
resource, an error is encountered:
> puppet resource service someservice logonaccount='DOMAIN\msvc_automation$'
Error: The given password is invalid for user 'DOMAIN\msvc_automation$'.
Error: /Service[someservice]/logonaccount: change from 'DOMAIN\svc_user' to 'DOMAIN\msvc_automation$' failed" The given password is invalid for user 'DOMAIN\msvc_automation$'.
This continues to occur event with alternative attempts, such as with the ~
used as a password placeholder in PsExec.
> puppet resource service someservice logonaccount='DOMAIN\msvc_automation$' logonpassword=undef
> puppet resource service someservice logonaccount='DOMAIN\msvc_automation$' logonpassword='~'
Support configuring services as gMSAs with the oob service resource on Windows.
Alternatives include using an exec resource that reconfigures the service to run as a gMSA:
exec { "Configure someservice logon account":
command => "${facts['os']['windows']['system32']}\\sc.exe config someservice obj= 'DOMAIN\\msvc_automation$'",
unless => "${facts['os']['windows']['system32']}\\sc.exe qc someservice | ${facts['os']['windows']['system32']}\\findstr.exe -li 'DOMAIN\\msvc_automation$'",
provider => powershell,
}
In the windows service provider, there is a check done to validate the logon credentials here
That eventually takes us here where we call the Windows API to get an impersonation token. The logon type LOGON32_LOGON_NETWORK
is attempted, and if that fails, LOGON32_LOGON_INTERACTIVE
is used.
gMSA cleartext password retrieval is fairly complex, but can be done with assistance from the PS Module DSInternals in the ConvertFrom-ADManagedPasswordBlob
function , and the general process is described in this blog post. Forcing a Puppet user to provide this value is not user friendly though.
Alternatively, if user context is SYSTEM (or another account with the "Act as part of the operating system" privledge) during the LogonUserW call, and if the logontype is specified as LOGON32_LOGON_SERVICE
, the cleartext password can be provided as _SA_{262E99C9-6160-4871-ACEC-4E61736B6F21}
, which is defined in lmaccess.h as the constant SERVICE_ACCOUNT_PASSWORD
. This is detailed in this MS discussion and the constant is noted in this Rust winapi wrapper (among other places like the windows sdk).
I see a few possible solutions when credentials are being validated, or the password is being set for the resource:
SERVICE_ACCOUNT_PASSWORD
, and a logon type of LOGON32_LOGON_SERVICE
when authenticating. I think this is most preferred, although it would only work when Puppet is run as SYSTEM.LOGON32_LOGON_SERVICE
when authenticating. Isnt user friendly since the user needs to provide the SERVICE_ACCOUNT_PASSWORD
value, but would at least allow Puppet, in some contexts (needs to be running as SYSTEM), to set a gMSA as a service logon.for the scheduled_task
resource here, testing if a user is a gMSA is done simply by checking if the last character is $
which seems reasonable. However it seems like proper testing is doable via NetIsServiceAccount
Hi, I've the following code:
braces1: "%{literal('%')}{}"
braces2: "%{lookup('braces1')}"
braces3: "%{alias('braces1')}"
echo { 'braces1:':
message => "${lookup('braces1')}",
withpath => false,
}
echo { 'braces2:':
message => "${lookup('braces2')}",
withpath => false,
}
echo { 'braces3:':
message => "${lookup('braces3')}",
withpath => false,
}
(echo
resource from https://forge.puppet.com/modules/ipcrm/echo/readme which is basically a notify)
This outputs:
Notice: %{}
Notice:
Notice: %{}
I expect the same output three times:
Notice: %{}
Notice: %{}
Notice: %{}
Above code with puppet 8.5.1 or 7.29.1
I'm not sure if this is the intended behavior for lookup or a bug. Also the documentation for the literal()
seems to be wrong: puppetlabs/puppet-docs#1149 (which made this bug even more confusing).
An undefined variable produce a warning. Later on, a value can be assigned to this variable, which somewhat clash with the fact that variables are immutable.
The following show illustrate this issue:
if defined('$foo') {
warning("A: foo=${foo}")
} else {
warning("A: foo is not defined")
}
$foo = 'value'
if defined('$foo') {
warning("B: foo=${foo}")
} else {
warning("B: foo is not defined")
}
Applying this catalog produce:
Warning: Scope(Class[main]): A: foo is not defined
Warning: Scope(Class[main]): B: foo=value
The above example is intended to illustrate the issue. In the wild, this problem is often found when a module provide a defined type which has parameters that default to the value of a variable in another class of the module, for example:
class stuff {
$color = 'red'
}
define stuff::thing (
Optional[String] $color = $stuff::color,
) {
include stuff
warning("Look, a ${color} ${title}!")
}
stuff::thing { 'something':
}
# Warning: Unknown variable: 'stuff::color'.
#=> "Look, a something!"
Instead of immediately producing a warning if a variable is used (or returning false when testing if the variable is defined), stopping evaluation of the current statement and re-evaluating it later may help have a better behavior.
If deferring evaluation is not possible, assigning a value to a variable which has already resolved as undefined should not be possible and raise an error.
puppet lookup gives error when key's value is ::ff:10.10.10.1
no error expected
Steps to reproduce the behavior:
platform::dns::resolv::servers:
- fd01::1
- ::ff:10.10.10.1
---
version: 5
hierarchy:
- name: "yaml"
datadir: /tmp/puppet/hieradata
data_hash: yaml_data
paths:
- system.yaml
wget https://apt.puppet.com/puppet8-release-bulleye.deb
dpkg -i puppet8-release-bulleye.deb
apt-get update
apt-get install puppet-agent
/opt/puppetlabs/bin/puppet --version
8.4.0
Error: Could not run: Lookup of key 'platform::dns::resolv::servers' failed: Value for key 'platform::dns::resolv::servers', in hash returned from data_hash function 'yaml_data', when using location '/tmp/puppet/hieradata/system.yaml', has wrong type, expects Puppet::LookupValue, got Array[Any, 2, 2]
Originally reported in https://puppet.atlassian.net/browse/PUP-11719
The function ModuleDataProvider::validate_data_hash
is supposed to prune all hiera keys that are not prefixed with the configured module_name. It does not. Instead it returns the hash that it was called with unchanged. All pruning is done on a cloned version of the supplied hash that’s discarded.
https://github.com/puppetlabs/puppet/blob/main/lib/puppet/pops/lookup/module_data_provider.rb#L46
It should prune all hiera keys that are not prefixed with the configured module name.
It should include the name of any offending key(s) in the warning message that it prints. The warning message as currently implemented doesn't provide the developer with any meaningful actionable information.
https://github.com/puppetlabs/puppet/blob/main/lib/puppet/pops/lookup/module_data_provider.rb#L54 should be modified to be something like:
Puppet.warning("Module '#{module_name}': #{msg}, key=#{k}") to help identify the offending key(s).
Hiera keys not prefixed with the module name are not pruned from the supplied hash. No warning advising on the offending key(s) is printed.
Delete this line and the pruned hash is returned:
https://github.com/puppetlabs/puppet/blob/main/lib/puppet/pops/lookup/module_data_provider.rb#L57
Puppet Version: 7.20
Puppet Server Version:
OS Name/Version:
Add any other context about the problem here.
We observe the following error across ~5% of all Puppet agent runs in our organization against our Puppet 7.x masters when we set multithreaded: true
, with any value for max-active-instances
(even 2
):
ERROR [qtp735649759-289] [puppetserver] Puppet Server Error: Evaluation Error: Error while evaluating a Type-Name, Internal Error: Attempt to redefine loader named '<foo> private' (file: /path/to/our/modules/foo/manifests/init.pp, line: 12, column: 18) on node clientnode.example.com
The error disappears if we disable threading, or if we hand-insert thread synchronization into the offending method or its caller. It is not consistent to any particular client node, and nodes on which agent runs throw the error will work fine the other ~95% of the time.
The source of the error, add_loader_by_name
in puppet/lib/ruby/vendor_ruby/puppet/pops/loaders.rb
, is fairly straightforward, adding a new key to an instance variable and throwing the error we observe if the key is already present:
def add_loader_by_name(loader)
name = loader.loader_name
if @loaders_by_name.include?(name)
raise Puppet::ParseError, _("Internal Error: Attempt to redefine loader named '%{name}'") % { name: name }
end
@loaders_by_name[name] = loader
end
Interestingly, adding Puppet.info
or Puppet.warn
output before the if
statement, to debug the call stack and thread state, causes the issue to disappear. I'd theorize that whatever output synchronization the logging system uses adds sort of back-door thread safety. Adding logging inside the if
statement doesn't do this, which makes sense.
The calling method in all instances of this error we've logged to date has been []
. If we add naïve Ruby-native synchronization to that method as below, our problem disappears:
def [](loader_name)
loader = @loaders_by_name[loader_name]
if loader.nil?
# Unable to find the module private loader. Try resolving the module
@my_semaphore.synchronize {
loader = private_loader_for_module(loader_name[0..-9]) if loader_name.end_with?(' private')
raise Puppet::ParseError, _("Unable to find loader named '%{loader_name}'") % { loader_name: loader_name } if loader.nil?
}
end
loader
end
That's presumably not a correct-for-this-codebase fix by itself, but does point in the general direction of "it's threads stepping on each other, add a sempahore somewhere."
It's not clear why our ecosystem in particular triggers this issue. While we're a medium-large organization, our Puppet masters are spread around to various regional collections of nodes. The one on which we did the above debugging manages only about a dozen nodes. The issue manifests on all of them.
Our module ecosystem is old growth forest, with a lot of interwoven dependency. It's possible that the trigger here is multiple references to the same dependent object being resolved in parallel.
It's of note that we have a number of modules with names of the form *_private
(in a search path distinct from our "main" module repo). But this seems to occur with any module and does not occur with threading disabled, so is unlikely to be some kind of namespace collision.
the old code would work correctly for ensure => present
, but not for ensure => 'some_stream'
, as dnf does include the whole module spec in the error message and the old regex didn't match anymore.
# dnf module install -d 0 -e 1 -y 389-ds
Error: Problems in request:
broken groups or modules: 389-ds
# dnf module install -d 0 -e 1 -y 389-ds:1.4
Error: Problems in request:
broken groups or modules: 389-ds:1.4
module is correctly enabled by switching to the "enable only" codepath automatically
package { '389-ds':
ensure => '1.4',
provider => dnfmodule,
}
Setting enable_only => true
avoids the issue, but the code should be able to figure this out itself.
See voxpupuli/puppet-nodejs#488
Puppet agents print a warning if the number of fact values exceeds its soft limit:
Warning: The current total number of facts: 5787 exceeds the number of facts limit: 2048
However, the wording "total number of facts" is confusing, because it sounds like it's the number of facts, when really the problem is the number of values that the facts are producing, due to nested arrays and hashes like the _puppet_inventory_1
fact in PE.
The message should say something like
The total number of fact values X exceeds the fact values limit Y
The documentation for the number_of_facts_soft_limit
should be updated so its clear we're counting the leaves of structured facts.
Steps to reproduce the behavior:
# cat <<END > /opt/puppetlabs/facter/facts.d/many.rb
#!/opt/puppetlabs/puppet/bin/ruby
require 'json'
puts JSON.dump("many" => [*1..3000])
END
# puppet agent -t
/opt/puppetlabs/puppet/bin/puppet agent -t
Info: Using environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Warning: The current total number of facts: 3323 exceeds the number of facts limit: 2048
This behavior is occurring because we corrected how fact values are counted, see f33de24
Ruby 3.3 is available since some months. Some people build puppet against system Ruby and some distros already ship Ruby 3.3 or prepare it. As a Puppet user I would like to see support for current Ruby versions in latest Puppet
Add Ruby 3.3 to CI.
This also correlates to puppetlabs/puppet-enterprise_issues#8
Puppet returns exit code 0 when it fails to modify the system. This is not specific to any resource type, for example:
❯ bundle exec puppet resource file / content=123
Notice: /File[/]: Not removing directory; use 'force' to override
Notice: /File[/]: Not removing directory; use 'force' to override
Error: Could not set 'file' on ensure: Permission denied @ dir_s_mkdir - /20240312-324986-1f7prcz.lock
Error: Could not set 'file' on ensure: Permission denied @ dir_s_mkdir - /20240312-324986-1f7prcz.lock
Wrapped exception:
Permission denied @ dir_s_mkdir - /20240312-324986-1f7prcz.lock
Error: /File[/]/ensure: change from 'directory' to 'file' failed: Could not set 'file' on ensure: Permission denied @ dir_s_mkdir - /20240312-324986-1f7prcz.lock
file { '/':
ensure => 'directory',
provider => 'posix',
}
❯ echo $?
0
It should return non-zero, but do so in a backwards-compatible way through a feature flag. Proposal is to add --fail
flag (inspiration taken from curl) like:
❯ bundle exec puppet resource file --fail / content=123
...
❯ echo $?
1
On a Masterless Puppet Environment using the last version of puppet-agent, we get two warning messages about the fact that there is no private key or certificate, what is expected in this environment.
The puppet-agent-7.16.0-1.el9.x86_64
is the most recent version that does not show the warning, from version puppet-agent-7.17.0-1.el9.x86_64
the warnings are displayed.
Can you please take a look?
No warning messages on a Masterless Puppet Environment.
It's a bug with the file resource https://www.puppet.com/docs/puppet/5.5/types/file.html
# mkdir -p testing/filebug/manifests
# vi testing/filebug/manifests/init.pp
# cat testing/filebug/manifests/init.pp
class filebug {
$test_file = 'https://link.testfile.org/PDF10MB'
file { '/tmp/test_file':
ensure => 'file',
source => $test_file,
}
}
# puppet apply --modulepath=/root/testing -e "include filebug"
Notice: Compiled catalog for testhost.cern.ch in environment production in 0.01 seconds
Warning: Private key for ‘testhost.cern.ch' does not exist
Warning: Client certificate for ‘testhost.cern.ch' does not exist
Notice: /Stage[main]/Filebug/File[/tmp/test_file]/ensure: defined content as '{mtime}2022-10-08 01:39:09 UTC'
Notice: Applied catalog in 4.88 seconds
# rpm -qa puppet-agent
puppet-agent-7.28.0-1.el9.x86_64
Puppet7-agent package for bookworm is maintained by Debian, I know. Still, the puppet_gem provider on the package is calling /opt/puppetlabs/puppet/bin/gem
binary which doesn't exist in this package.
Except for bookworm, we had three different gem options: gem (system provider), puppet_gem (provider) and puppetserver_gem (provider/module). Looks like the p7 package from debian is missing the puppet_gem provider as it only installs /usr/bin/gem which is the system provider.
Calling the puppet_gem provider on the puppet7 agent installation from Bookworm breaks because there is no /opt/puppetlabs/puppet/bin
directory on the debian package
Puppet should call the correct (?) gem binary.
Steps to reproduce the behavior:
This could potentially be considered more of a documentation bug, but it is part of the code, not the docs...).
The examples in the hoera section (https://github.com/puppetlabs/puppet/tree/main/examples/hiera) are significantly out of date, referring to hiera3 syntax.
As a service delivery partner we want to engage into the development. This isn't really possible without at least read-only access to the pipelines. We don't know which jobs exist and what they do. That makes it impossible to add support for new platforms. And as we can see on the existing requests Puppet itself is quite slow for adding support for new distributions/architectures.
use GitHub actions for supported platforms to build packages (preferred) or at least grant partners access to the Jenkins (which was public in the past).
While I hate it that an open source tool uses private pipelines, you could also workaround this by providing new packages faster. However I think reproducible builds are required in the future and that also requires logs.
Cannot install puppet7 on a clean ubuntu bionic instance.
packages in http://apt.puppet.com/dists/bionic/puppet7/binary-amd64/ were updated yesterday evening, hash mismatch.
apt repos should be updated
Steps to reproduce the behavior:
On a ubuntu bionic image:
wget https://apt.puppetlabs.com/puppet7-release-bionic.deb -O puppet.deb
dpkg-reconfigure debconf -f noninteractive -p critical
dpkg -i puppet.deb
apt update
this returns:
Get:6 http://apt.puppet.com bionic/puppet7 amd64 Packages [40.1 kB]
Err:6 http://apt.puppet.com bionic/puppet7 amd64 Packages
Hash Sum mismatch
Hashes of expected file:
- Filesize:40149 [weak]
- SHA512:7a5cf571e1682d32b988cacdcaa5d922ec4dcf85fc260f6c277adc1c0b5eafc8377091f614e05ebb04c05a33219dcfd23a143a925a1599cdfb1272d8004630bb
- SHA256:5b095bcd8b7643af54dcee67eb4c2bc7112bc28bc73d157ae4a006f204507ebc
- SHA1:7abbdaedd9cfed9de77c7cb4f8693dfa0b48e5cc [weak]
- MD5Sum:6fd19ae4c5ef33f101eced99ed46965b [weak]
Hashes of received file:
- SHA512:a8fc2d224e85b787d2e99d17336f2a31637b13f7720b86eb1366b708dded7e1ff57e5da0b110cc0b6e187e48c2669b6639a2f379b1848cd26d963dbb0d55eee9
- SHA256:1c686de9f77bb4a52846a22fdae23ee83956a3fe485955700542a975b2e7bf6a
- SHA1:b1799733c0d6613a37751832d43178639fbcdb69 [weak]
- MD5Sum:5e5391bb667ae3c91b598ea78f0aaefe [weak]
- Filesize:40149 [weak]
Last modification reported: Tue, 30 Apr 2024 03:39:01 +0000
Release file created at: Tue, 30 Apr 2024 03:36:55 +0000
Get:7 http://apt.puppet.com bionic/puppet7 all Packages [21.2 kB]
Err:7 http://apt.puppet.com bionic/puppet7 all Packages
Fetched 218 kB in 1s (351 kB/s)
Reading package lists... Done
E: Failed to fetch http://apt.puppet.com/dists/bionic/puppet7/binary-amd64/Packages.gz Hash Sum mismatch
Hashes of expected file:
- Filesize:40149 [weak]
- SHA512:7a5cf571e1682d32b988cacdcaa5d922ec4dcf85fc260f6c277adc1c0b5eafc8377091f614e05ebb04c05a33219dcfd23a143a925a1599cdfb1272d8004630bb
- SHA256:5b095bcd8b7643af54dcee67eb4c2bc7112bc28bc73d157ae4a006f204507ebc
- SHA1:7abbdaedd9cfed9de77c7cb4f8693dfa0b48e5cc [weak]
- MD5Sum:6fd19ae4c5ef33f101eced99ed46965b [weak]
Hashes of received file:
- SHA512:a8fc2d224e85b787d2e99d17336f2a31637b13f7720b86eb1366b708dded7e1ff57e5da0b110cc0b6e187e48c2669b6639a2f379b1848cd26d963dbb0d55eee9
- SHA256:1c686de9f77bb4a52846a22fdae23ee83956a3fe485955700542a975b2e7bf6a
- SHA1:b1799733c0d6613a37751832d43178639fbcdb69 [weak]
- MD5Sum:5e5391bb667ae3c91b598ea78f0aaefe [weak]
- Filesize:40149 [weak]
Last modification reported: Tue, 30 Apr 2024 03:39:01 +0000
Release file created at: Tue, 30 Apr 2024 03:36:55 +0000
E: Failed to fetch http://apt.puppet.com/dists/bionic/puppet7/binary-all/Packages.gz
E: Some index files failed to download. They have been ignored, or old ones used instead.
Trying to install puppet7 on a fresh ubuntu bionic instance.
Type & providers that implement eval_generate
to create child resource may be called before the parent deferred resource is resolved. This was first reported in puppetlabs/puppetlabs-concat#789 where the concat
parent resource generates concat_fragments
.
Commit a76f498 made it possible to lazily resolve deferred resources. The behavior was guarded by the preprocess_deferred=true
setting. In puppet8, the default value was changed to false
, so deferred resources are no longer preprocessed.
The intention of the change was to allow dependencies of a deferred resource to be managed prior to the deferred resource. For example, from @bastelfreak in https://puppetcommunity.slack.com/archives/CFD8Z9A4T/p1700590993590929
gitlab_runner module uses a deferred function to register itself to the gitlab server. the gitlab runner has a local firewall for outgoing traffic. so we have to apply puppet code to modify the firewall before the function is executed
However, as a result of this change, type and provider lifecycle methods (such as validation, autorequires, prefetch, generate and eval_generate) may be called with deferred resources still in the catalog, since those calls occur after the catalog is created, but before the deferred resource is evaluated.
I would expect the eval_generate
method to be called after deferred values in the resource have been resolved, so that DeferredValue
objects don't leak into custom types/providers.
# rpm -q puppet-agent
puppet-agent-8.3.1.86.g719cf6147-1.el7.x86_64
# /opt/puppetlabs/puppet/bin/puppet module install puppetlabs-concat
Notice: Preparing to install into /etc/puppetlabs/code/environments/production/modules ...
Notice: Downloading from https://forgeapi.puppet.com ...
Notice: Installing -- do not interrupt ...
/etc/puppetlabs/code/environments/production/modules
└─┬ puppetlabs-concat (v9.0.0)
└── puppetlabs-stdlib (v9.4.1)
# cat <<END > concat.pp
concat { '/tmp/deferred_file': }
concat::fragment { '1':
target => '/tmp/deferred_file',
content => Deferred('md5', ['test']),
}
END
# puppet apply concat.pp
Warning: Deferred function md5 has no return_type, unable to guarantee value type during compilation.
(file & line not available)
Notice: Compiled catalog for papal-mandrel.delivery.puppetlabs.net in environment production in 0.07 seconds
Notice: /Stage[main]/Main/Concat[/tmp/deferred_file]/File[/tmp/deferred_file]/ensure: defined content as '{sha256}276179e40d8de773d3819996cd861446eeeef83fb607df4e72a3768c3390c1d0'
Notice: Applied catalog in 0.01 seconds
# cat /tmp/deferred_file
#<Puppet::Pops::Evaluator::DeferredValue:0x00007f8c92dba4b8>
This affects puppet 8 (by default) and puppet 7 (when running with preprocess_deferred=false
).
Hi, Ubuntu 24.04 is the latest stable release. Please provide packages for it.
A clear and concise description of what you want to happen.
A clear and concise description of any alternative solutions or features you've considered.
Add any other context or screenshots about the feature request here.
The package
provider "pip" does not handle correctly the installation of python modules via network urls (e.g. "git+https://github.com/") on Ubuntu 22.04: this is because on newer pip
versions the output of pip freeze --all
(which is used by puppet to test if a python module is already installed) is generated in an unexpected format:
puppet/lib/puppet/provider/package/pip.rb
Lines 102 to 108 in f9bcd09
root@831e7bc5baed:/# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04 LTS
Release: 22.04
Codename: jammy
root@831e7bc5baed:/# pip --version
pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)
root@831e7bc5baed:/# pip freeze --all | grep -i ^vSphere-Automation-SDK
vsphere-automation-sdk @ git+https://github.com/vmware/vsphere-automation-sdk-python.git@199b26f1d523023927c172afa6f5b2ebb85dd8f9
root@831e7bc5baed:/#
On Ubuntu 20.04 this issue does not happen:
root@70b080cc5203:/# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.6 LTS
Release: 20.04
Codename: focal
root@70b080cc5203:/# pip --version
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)
root@70b080cc5203:/# pip freeze --all | grep -i ^vSphere-Automation-SDK
vsphere-automation-sdk==1.86.0
root@70b080cc5203:/#
puppet apply
should skip installing the python module when run a second time since it's already installed
From an empty Docker container, image ubuntu:22.04
:
# install puppet
apt update
apt install --yes lsb-release wget python3-pip git
wget https://apt.puppetlabs.com/puppet8-release-$(lsb_release -sc).deb
dpkg -i puppet8-release-$(lsb_release -sc).deb
apt update
apt install --yes puppet-agent
# apply manifest
cat > pip_freeze_issue.pp <<'EOF'
package { 'vSphere-Automation-SDK':
ensure => 'present',
provider => 'pip',
source => 'git+https://github.com/vmware/vsphere-automation-sdk-python.git'
}
EOF
/opt/puppetlabs/bin/puppet apply pip_freeze_issue.pp
# check python module
pip freeze --all | grep -i ^vSphere-Automation-SDK
pip list --format=freeze | grep -i ^vSphere-Automation-SDK
# apply a second time, should not install the python module again
/opt/puppetlabs/bin/puppet apply pip_freeze_issue.pp
root@831e7bc5baed:/# /opt/puppetlabs/bin/puppet agent --version
8.4.0
root@831e7bc5baed:/# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04 LTS
Release: 22.04
Codename: jammy
root@831e7bc5baed:/#
The package provider for "pip" should use pip list --format=freeze
instead of pip freeze --all
, when supported (sorry i don't know the exact version)
After upgrading, either by running an 8.x install either by first uninstalling a 7.x install or by running it over the top, puppet installs successfully on windows 2019/2022 however I receive errors on the starting of the service. Seems to happen moreso on core, but it occurs on both desktop experience and regular. Sometimes after 5-10 starts it will actually start, but usually if stopped, it refuses to start again.
Failed to transition the puppet service to the SERVICE_RUNNING state. Detail: Failed to start the service: The service did not respond to the start or control request in a timely fashion.
Running puppet agent interactively works perfectly fine. rolling back to 7.x (even 7.27.0) works, as does starting the 7.x service.
The issue is exclusively on the service starting cleanly after installation. A reboot does not fix it. Nor does removing C:\programdata\puppetlabs, or refreshing to certs.
Any other additional information needed, let me know.
The gemspec still requires a minimum ruby version of 2.5 here.
Raise minimum ruby requirement to ruby 2.6 (which I believe is needed for puppet server). Will help keep consistency with other tooling and modules in the puppet ecosystem.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.