Giter Site home page Giter Site logo

Comments (7)

jaymzh avatar jaymzh commented on March 29, 2024 1

@gogsbread - Just for context, Davide is one of your co-workers (he's on the kernel team), and was on the OS Team that wrote these cookbooks (before the OS Team was disbanded in the recent re-org).

And yeah, it's harder with Apache than some others for the reason you describe. This is why we made a task for it rather than doing it (if it was easy we would have made the one-line diff :) ).

However the complete lack of any verification has caused a lot of problems thus far. At Facebook the yum origin servers went down. At SCALE the website went down. So it's painful. We didn't have a great solution off hand, so we just created this as a placeholder.

That said, we added none of that context to this task, which I'm sure I'll get a (justified) earful from @NaomiReeves on, and which caused you to trip on this.

So I think at this point this has enough context to leave open for one of us one day when we feel like digging into a fun problem. But if you prefer to modify the Issue itself, and then delete these comments, that's also fine.

from chef-cookbooks.

jaymzh avatar jaymzh commented on March 29, 2024 1

The backup directory is easy to find, it's in the config object, but I definitely would not go that route. What I'd do is something akin to a custom resource in which you...

  • Rsync existing config directory to a temporary directory, now you have a base (there's internal functions for creating temp directories and such, I use them in fb_network_scripts, then you can use shell_out to call rsync or cp).
  • loop over the configs and use build_resource to template them all into your temp directory. build_resource creates a resource, but doesn't put it in the resource_collection, so when you run it, it won't trigger the custom resource to have "updated". Trigger them all manually with .run_action(:create).
  • Now you have a full directory including both files the package dropped off as well as all the configs we want to test, but without having touched anything live.
  • Run httpd -t on that temp directory. Note that on debian-ish distros this is apache2 -t
  • If it works, then you loop over the configs and just use templates normally, but if not, throw a Chef::Exceptions::ValidationFailed exception

Then you're not reaching into the backup directory or ever fucking with live configs if they don't work. This is similar, but much less complicated, than when I do in fb_network_scripts or whatever it's called these days.

Don't feel obligated to do this, not sure what Naomi has you on, but if you're interested and want to do this, I can definitely review it.

from chef-cookbooks.

gogsbread avatar gogsbread commented on March 29, 2024

@davide125 A default approach to perform such a validation is to use verify property of template resource however that is tricky given how Apache configurations files are hierarchical. http -t runs on the entire hierarchy of included files, starting from the root http.conf, on the filesystem, however verify only provides a temporary path to a single file(fb_apache.conf) which doesn't have all the context to be verified in isolation; not even with httpd -t -f <config_file>.

One can only do a non-trivial, potentially error prone, and brittle implementation which is something that may not be worth doing. We'd be happy to review PRs but not something that we'd implement. Feel free to DM me and I can provide some internal context as well.

from chef-cookbooks.

gogsbread avatar gogsbread commented on March 29, 2024

@jaymzh thank you for the context. These comments and discussion and useful and we'll continue to have this documented.
I did tag Davide in the diff which has more elaborate documentation on the thought process and some alternatives that were discussed.

In summary, there are two high level approaches I could think of and worth documenting:

  1. Create all template configurations inside some temp directory and run httpd -t in there. If the verification is successful then we go about materializing the changed configuration
  2. Optimistically materialize the template configuration(before reloading/restarting the server) and run httpd -t to find errors and if wrong restore from the backup copy and fail the chef run.

For 1. @NaomiReeves had some discussions with chef dev

move the templates creation in to a custom resource that would collect/track all of the templates being modified and materialize them all in the same tmp dir
grab the existing configurations and de-dup against the updated configs
run the verify command on the group of new configs
if the verify command passes, move the updated configs to their correct path
if the verify command fails, do not move the updated configs and fail the chef run

For 2. I tried to provide a rough implementation, however I couldn't find an easy way currently to access the path where the file was backed-up.

from chef-cookbooks.

gogsbread avatar gogsbread commented on March 29, 2024

No obligation; I was just toying around with different ideas and learning chef in the process.

from chef-cookbooks.

gogsbread avatar gogsbread commented on March 29, 2024

@jaymzh I landed 913517d. that is currently rolled to shard 0. lmk if anything catches your eye that warrants an iteration.

from chef-cookbooks.

jaymzh avatar jaymzh commented on March 29, 2024

I left some comments for ya. Thanks for the heads up!

from chef-cookbooks.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.