Comments (7)
@gogsbread - Just for context, Davide is one of your co-workers (he's on the kernel team), and was on the OS Team that wrote these cookbooks (before the OS Team was disbanded in the recent re-org).
And yeah, it's harder with Apache than some others for the reason you describe. This is why we made a task for it rather than doing it (if it was easy we would have made the one-line diff :) ).
However the complete lack of any verification has caused a lot of problems thus far. At Facebook the yum origin servers went down. At SCALE the website went down. So it's painful. We didn't have a great solution off hand, so we just created this as a placeholder.
That said, we added none of that context to this task, which I'm sure I'll get a (justified) earful from @NaomiReeves on, and which caused you to trip on this.
So I think at this point this has enough context to leave open for one of us one day when we feel like digging into a fun problem. But if you prefer to modify the Issue itself, and then delete these comments, that's also fine.
from chef-cookbooks.
The backup directory is easy to find, it's in the config object, but I definitely would not go that route. What I'd do is something akin to a custom resource in which you...
- Rsync existing config directory to a temporary directory, now you have a base (there's internal functions for creating temp directories and such, I use them in fb_network_scripts, then you can use shell_out to call rsync or cp).
- loop over the configs and use
build_resource
to template them all into your temp directory.build_resource
creates a resource, but doesn't put it in the resource_collection, so when you run it, it won't trigger the custom resource to have "updated". Trigger them all manually with.run_action(:create)
. - Now you have a full directory including both files the package dropped off as well as all the configs we want to test, but without having touched anything live.
- Run
httpd -t
on that temp directory. Note that on debian-ish distros this isapache2 -t
- If it works, then you loop over the configs and just use
templates
normally, but if not, throw aChef::Exceptions::ValidationFailed
exception
Then you're not reaching into the backup directory or ever fucking with live configs if they don't work. This is similar, but much less complicated, than when I do in fb_network_scripts or whatever it's called these days.
Don't feel obligated to do this, not sure what Naomi has you on, but if you're interested and want to do this, I can definitely review it.
from chef-cookbooks.
@davide125 A default approach to perform such a validation is to use verify
property of template
resource however that is tricky given how Apache configurations files are hierarchical. http -t
runs on the entire hierarchy of included files, starting from the root http.conf
, on the filesystem, however verify
only provides a temporary path to a single file(fb_apache.conf
) which doesn't have all the context to be verified in isolation; not even with httpd -t -f <config_file>
.
One can only do a non-trivial, potentially error prone, and brittle implementation which is something that may not be worth doing. We'd be happy to review PRs but not something that we'd implement. Feel free to DM me and I can provide some internal context as well.
from chef-cookbooks.
@jaymzh thank you for the context. These comments and discussion and useful and we'll continue to have this documented.
I did tag Davide in the diff which has more elaborate documentation on the thought process and some alternatives that were discussed.
In summary, there are two high level approaches I could think of and worth documenting:
- Create all template configurations inside some temp directory and run
httpd -t
in there. If the verification is successful then we go about materializing the changed configuration - Optimistically materialize the template configuration(before reloading/restarting the server) and run
httpd -t
to find errors and if wrong restore from thebackup
copy and fail the chef run.
For 1. @NaomiReeves had some discussions with chef dev
move the templates creation in to a custom resource that would collect/track all of the templates being modified and materialize them all in the same tmp dir
grab the existing configurations and de-dup against the updated configs
run the verify command on the group of new configs
if the verify command passes, move the updated configs to their correct path
if the verify command fails, do not move the updated configs and fail the chef run
For 2. I tried to provide a rough implementation, however I couldn't find an easy way currently to access the path where the file was backed-up.
from chef-cookbooks.
No obligation; I was just toying around with different ideas and learning chef in the process.
from chef-cookbooks.
@jaymzh I landed 913517d. that is currently rolled to shard 0. lmk if anything catches your eye that warrants an iteration.
from chef-cookbooks.
I left some comments for ya. Thanks for the heads up!
from chef-cookbooks.
Related Issues (20)
- Small copy-pasta in the fb_ethtool README.md HOT 2
- fb_postfix should not default mydomain to fb.com
- fb_systemd::udevd fails on Ubuntu 20.04 LTS due to missing symlink HOT 1
- fb_vsftpd is broken on debian 9
- fix shellcheck issues in fb_ipset and fb_less
- fb_tmpclean doesn't include tmpreaper defaults on debian/ubuntu - breaks /tmp cleanup
- fb_network_scripts changes should be reflected in /var/chef/backup HOT 1
- fb_helpers_reboot lies about :now HOT 2
- fb_storage always ignores override files when '_clowntown_override_file_method' not defined
- codemod internal library methods in fb_fstab to make it clear they're not part of the API
- fb_ntp shouldn't default to facebook timeservers HOT 18
- Enablement of `unified_mode` for v17+ Chef client compatibility HOT 4
- Fix CI failures HOT 2
- fb_helpers contains namespace collisions with official chef node objects HOT 13
- node.antlir_build? not defined in open source HOT 2
- Definitions for antlir2 fix are not exposed in open source HOT 2
- Compound API interactions for cookbooks included by `fb_init` are difficult to implement HOT 25
- Regression with recent log output change HOT 3
- fb_apache: hardcoded /etc/httpd path for server-status
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from chef-cookbooks.