Comments (11)
FTR: Even though I discovered this on Chef 12.x, I can’t see why this would not affect other Chef versions or why this wasn’t an issue earlier. It really makes no sense to me. I’m probably being dense, but I can’t see how this code ever worked or why kitchen tests would pass with this (note that the kitchen tests are not executed in the CI!)
from iptables.
The behaviour of resource is correct as far as I can see, this is how accumulator pattern template resources works.
Every time the iptables_rule
resource is called it adds the rule that the resource defines to the template variables and as the template is defined with action :nothing
and delayed_action :create
the templated file is created at the end of the run when all rules are present in the template variables. See the haproxy cookbook for another example of creating a file like this.
From the trace that you posted in #109 the error you get:
Chef::Exceptions::FileNotFound
------------------------------
Cookbook 'ch-openvpn-cluster' (0.1.0) does not contain a file at any of these locations:
templates/ubuntu-16.04/rules.v4.erb
templates/ubuntu/rules.v4.erb
templates/default/rules.v4.erb
templates/rules.v4.erb
Is because something is altering the source
property of the template to the name of the file that is being created rather than the actual erb source file which is iptables.erb
. This facility only exists in the case that someone wants to use a custom rule file template which is going to be an edge case as the format is pretty much defined by iptables-save.
template("/etc/iptables/rules.v4") do
action [:nothing]
retries 0
retry_delay 2
default_guard_interpreter :default
source "rules.v4.erb" <----- problem
variables {"iptables"=>{"filter"=>{"chains"=>{"INPUT"=>"ACCEPT [0:0]", "FORWARD"=>"ACCEPT [0:0]", "OUTPUT"=>"ACCEPT [0:0]"}, "rules"=>["-A FORWARD -i brvpn -o brvpn -j ACCEPT -m comment --comment \"/etc/iptables/rules.v4\""]}}}
declared_type :template
cookbook_name "ch-openvpn-cluster" <----- problem
owner nil
group nil
mode "0600"
path "/etc/iptables/rules.v4"
verifications []
end
If you copy the iptables.erb from this cookbook to your wrapper then it will work but it still is not correct.
from iptables.
So what I did to trace this down further was to add log lines to the resource:
…
property :config_file, String, default: lazy { node['iptables']['persisted_rules_iptables'] }
…
…
action :create do
Chef::Log.warn("#{new_resource}")
Chef::Log.warn(new_resource.source)
Chef::Log.warn(new_resource.cookbook)
Chef::Resource::Template.send(:include, Iptables::RuleHelpers)
Chef::Resource::Template.send(:include, Iptables::ChainHelpers)
with_run_context :root do
edit_resource(:template, new_resource.config_file) do |new_resource|
Chef::Log.warn("#{new_resource}")
Chef::Log.warn("#{source} #{new_resource.source} -- #{cookbook} #{new_resource.cookbook}")
source new_resource.source
cookbook new_resource.cookbook
sensitive new_resource.sensitive
mode new_resource.filemode
…
[2020-01-15T14:04:16+00:00] WARN: iptables_rule[bridge_forwarding]
[2020-01-15T14:04:16+00:00] WARN: iptables.erb
[2020-01-15T14:04:16+00:00] WARN: iptables
[2020-01-15T14:04:16+00:00] WARN: template[/etc/iptables/rules.v4]
[2020-01-15T14:04:16+00:00] WARN: rules.v4.erb rules.v4.erb --
followed by the failure I posted.
So, I get that this pattern is common. I read the haproxy source and they do the same thing over there O_o. So I guess this must be correct, but I don’t understand why. On the other hand, I fully understand why it does not work, and this is what is confusing to me. Here’s what I think happens:
Inside the edit_resource block, new_resource
refers to the template resource instance. The log output indicates that the source
property of the template resource is defaulted to the name of the template resource (new_resource.config_file
= '/etc/iptables/rules.v4'
via default attributes) plus the .erb
suffix. The cookbook property is undefined and thus defaults to the cookbook which is currently being compiled/run (in my case, ch-openvpn-cluster
).
So that is where I understand that the values come from.
I do not understand why the pattern would work with the block as written, but apparently it does, because it’s the same in the haproxy cookbook. On the other hand, I fully understand how it fails. This is very weird. I’m probably overlooking something obvious.
I see two options here:
- The pattern does in fact not work inside this cookbook as written, and nobody noticed yet because they trigger corner cases where it doesn’t matter or haven’t upgraded yet. The CI is not running in travis and thus nobody really ever ran the tests (or the tests also trigger the corner cases). I consider this highly unlikely.
- The pattern does not work on Chef 12.x but works on newer versions. I’d be fine with that, in which case the bug would be that this cookbook claims compatiblity. We can wait, we’re in the process of upgrading to 14 either way. However, then I don’t get why you were able to test #109 with Chef 12 successfully.
I’m very confused by this. Maybe it’s because it’s after lunch and in the afternoon, but if there’s any chance you can show me where I’m wrong, that’d be highly appreciated. In the meantime, I’m going to try to set up vagrant et al. (we typically use dokken or openstack drivers) to run the CI tests locally and try to write a reproducer.
(Unfortunately, I can’t simply publish the wrapper cookbook or a stripped down version of it right now.)
If I can get to a point where I understand why this pattern is supposed to work, I can probably figure out why it doesn’t work for us and what we need to do to fix it.
from iptables.
Around the use of new_resource
for the block variable I agree, I still don't fully understand how that works and for all the digging I've done I can't find anything concrete to explain it.
Things to try:
- Do the kitchen tests work if you clone the branch and run them? I've tried with Dokken and Vagrant/VB and I don't get any differing results between them. (with this cookbook at least).
- Try this resource:
#
# Author:: Ben Hughes <[email protected]>
# Cookbook:: iptables
# Resource:: rule
#
# Copyright:: 2020, Ben Hughes
# Copyright:: 2017-2019, Chef Software, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
property :source, String, default: 'iptables.erb'
property :cookbook, String, default: 'iptables'
property :config_file, String, default: lazy { node['iptables']['persisted_rules_iptables'] }
property :table, String, equal_to: %w(filter mangle nat raw security), default: 'filter'
property :chain, String
property :match, String
property :target, String
property :line, String
property :comment, [String, TrueClass, FalseClass], default: true
property :extra_options, String
property :filemode, [String, Integer], default: '0600'
action :create do
Chef::Resource::Template.send(:include, Iptables::RuleHelpers)
Chef::Resource::Template.send(:include, Iptables::ChainHelpers)
with_run_context :root do
new_rule = new_resource
edit_resource(:template, new_resource.config_file) do
source new_rule.source
cookbook new_rule.cookbook
sensitive new_rule.sensitive
mode new_rule.filemode
variables['iptables'] ||= {}
variables['iptables'][new_rule.table] ||= node['iptables']['persisted_rules_template'][new_rule.table].dup
variables['iptables'][new_rule.table]['rules'] ||= []
unless new_rule.line.nil? && new_rule.chain.nil? && new_rule.match.nil? && new_rule.target.nil?
rule = rule_builder(
line: new_rule.line,
chain: new_rule.chain,
match: new_rule.match,
target: new_rule.target,
extra_options: new_rule.extra_options)
rule.concat(comment_builder(name: new_rule.name, comment: new_rule.comment)) unless comment_exists?(rule)
Chef::Log.info("Running accumulator template resource for rule '#{rule}' and file #{new_rule.config_file}")
variables['iptables'][new_rule.table]['rules'] << rule
end
action :nothing
delayed_action :create
end
end
end
That makes more sense to me from the accessing the external new_resource
via new_rule
and removing the block var. See if that has a different result.
I'm going to try a few other things as well.
from iptables.
Also to add, simply removing the |new_resource|
section has a very strange failure mode which I'm still trying to work out.
from iptables.
There's some more info here as well: chef/chef#5438. Which would suggest that the resource as per the PR is correct, that was in my browser history so I'm guessing that where I came up with just accepting that it worked and not trying to work out exactly how/why.
Whether those PRs made it into Chef 12 is another question which could explain something but it doesn't explain why with the test kitchen I can get it passing with 12.21.5 without issue.
edit:
https://github.com/chef/chef/blob/master/lib/chef/dsl/declare_resource.rb#L156
https://www.rubydoc.info/gems/chef/12.13.37/Chef%2FDSL%2FDeclareResource:edit_resource
https://www.rubydoc.info/gems/chef/Chef%2FDSL%2FDeclareResource:edit_resource
https://docs.chef.io/dsl_recipe.html#edit-resource
That sort of explains the logic of how the block and new_resource are used, I'm not going to pretend I'm clever enough to understand exactly how that works completely but I can follow the logic of the declared external resource being passed in block &resource_attrs_block
and working on the edited resource object.
Doesn't explicitly help the issue you're facing though mind. If the kitchen tests in this cookbook work then it's got to be something the wrapper is influencing to cause the error.
from iptables.
Thank you for your attention. It is really appreciated.
1. Do the kitchen tests work if you clone the branch and run them? I've tried with Dokken and Vagrant/VB and I don't get any differing results between them. (with this cookbook at least).
The kitchen.dokken.yml does not define any suites, only platforms and ends in a ...
. Can it even be used? If so, how?
Vagrant+VirtualBox should be working on my work laptop tomorrow, I’ll try the main kitchen.yml
then.
Then I can also do the other tests.
from iptables.
The dokken configuration suppliments the kitchen.yml so it gets the suites from there, kitchen.dokken.yml only overwrites the platforms.
export KITCHEN_YAML=kitchen.yml
export KITCHEN_LOCAL_YAML=kitchen.dokken.yml
Also to note that i've noticed that #108 is pending so I probably won't do any more work on this as the rewrite is occurring. Doesn't help the Chef 12 issue though.
from iptables.
I have this for my dokken runes to make life easier
alias dokken="KITCHEN_LOCAL_YAML=kitchen.dokken.yml kitchen"
Yep, 108 is coming but has a minimum of Chef 14, mainly because it is the oldest supported version of Chef and I wanted to use some newer features for the cookbook like the description:
field for self documenting
from iptables.
@xorima Thanks! With that, I was able to run the dokken tests.
I have a reproducer for Chef 12, but it doesn’t fail for Chef 14. If the rewrite in #108 is going to break Chef 12 either way (which is totally fair), you can go ahead and close this and we’ll simply pin iptables to something lower until we finished our migration to Chef 14.
I also found the key difference: In Chef 14, new_resource
inside the block refers to the iptables_rule[…]
resource, and not to the newly created template resource. Which at least explains what’s wrong here.
Note that the kitchen tests in the cookbook do not fail for Chef 12; I had to add a specific reproducer (see #111 if you like). They simply do not explicitly test the forwarding of attributes to the nested template resource. It may be worth to add the reproducer to the test suite nevertheless.
from iptables.
Hey @horazont
Unfortunately I cannot see large value in continuing to support a version of chef which is 4 majors behind the latest and out of support by chef.
My advice would be to find a version of this cookbook which supports both 12 and later, upgrade nodes to the latest version then upgrade this cookbook to latest.
Thanks
Jason.
from iptables.
Related Issues (20)
- Update documentation to use `rule` instead of `iptables_rule` HOT 1
- iptables v1.8.2 (legacy): unknown option "--to-ports" HOT 2
- Getting issue while running Bootstrap command in Chef HOT 2
- debian if-pre-up.d script references nonexistent /etc/iptables/general HOT 2
- Doesn't work on ubuntu bionic with netplan HOT 1
- iptables cookbooks breaks at load time if it's in the dependency tree on non-linux platforms HOT 2
- 6.0.1 using Ruby 2.4 method causing error on chef client <13 HOT 3
- ip_tables 7.0.0 failing because of an unknown check HOT 3
- iptables_load wrapper does not obey iptables-restore exit status HOT 2
- How to order rules since 6.0? HOT 8
- iptables rule creation failing HOT 2
- Can the iptables and ip6tables be enable/disable separately? HOT 1
- iptables_service fails under Chef 16.13 with undefined method `new_resource' for Chef::Resource::Service HOT 1
- Feature request: Alternative to specifying an exact line_number HOT 2
- deprecated default recipe does a lot more than resources HOT 2
- iptables v1.8.4 (legacy): unknown option "--to-ports" HOT 1
- is this cookbook abandoned ?
- fix rebuild-iptables to load system-created rules HOT 2
- Can't start iptables cookbook on host without ipv6 support HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from iptables.