Use -z when invoking pkg_add to allow installation of alternatives

[ansible]

Use -z when invoking pkg_add to allow installation of alternatives

[landryb]

Ok, the example i provided wasnt totally clear - after more testing on 5.5, 5.6 and -current, -z is needed for fuzzy matching but so far only on the pX part of the pkgname (the patchlevel).

Ie in the case where the available packages are say php-fpm-5.3.29p2 php-fpm-5.5.18p1 php-fpm-5.4.34p0, pkg_add -Iz php-fpm-5.3 would fail (not precise enough for the tools) but pkg_add -Iz php-fpm-5.3.29 would install php-fpm-5.3.29p2.

If the version specified is higher than any of the available, the latest one would be installed. Ie pkg_add -Iz php-fpm-5.6 would install php-fpm-5.5.18p1.

In any case, using -z would provide better matching than what we have now, where we need to specifiy exactly the version on the mirror.

[dentarg]

Yes, this would be useful addition to the openbsd_pkg module in Ansible.

I have a task that fails now:

- name: Be sure nginx is installed
  openbsd_pkg: name=nginx-1.5.7 state=latest

Seems like -z would solve it:

openbsd% sudo pkg_add nginx
quirks-2.9 signed on 2014-07-31T22:37:55Z
Ambiguous: choose package for nginx
 a       0: <None>
         1: nginx-1.4.7p0
         2: nginx-1.4.7p0-lua
         3: nginx-1.4.7p0-naxsi
         4: nginx-1.4.7p0-passenger
         5: nginx-1.5.7p3
         6: nginx-1.5.7p3-lua
         7: nginx-1.5.7p3-naxsi
         8: nginx-1.5.7p3-passenger
Your choice: ^CFatal error: Caught SIGINT
 at /usr/libdata/perl5/OpenBSD/AddDelete.pm line 40.

openbsd% sudo pkg_add nginx-1.5.7
quirks-2.9 signed on 2014-07-31T22:37:55Z
Error from https://stable.mtier.org/updates/5.6/amd64/nginx-1.5.7.tgz
ftp: Error retrieving file: 404 Not Found
Error from http://ftp.su.se/pub/OpenBSD/5.6/packages/amd64/nginx-1.5.7.tgz
ftp: Error retrieving file: 404 Not Found
Can't find nginx-1.5.7
--- nginx-1.5.7 -------------------
Can't install nginx-1.5.7: not found

openbsd% sudo pkg_add -z nginx-1.5.7
quirks-2.9 signed on 2014-07-31T22:37:55Z
nginx-1.5.7p3: ok
The following new rcscripts were installed: /etc/rc.d/enginx
See rc.d(8) for details.
Look in /usr/local/share/doc/pkg-readmes for extra documentation.

[robynbergeron]

@landryb Thanks for submitting this feature idea -- and thanks to @dentarg for adding on with additional info. Sorry it's taken us a while to get to triaging this. :)

@eest Pinging you to bring this feature request for your attention. Please note that you are under no particular obligation to add this feature.

We encourage all Ansible community members to submit pull requests to satisfy feature requests. Consult the module guidelines (http://docs.ansible.com/ansible/developing_modules.html) for more information.

[gregdek]

[module: packaging/os/openbsd_pkg.py]

[eest]

I am against this change for the same reasons I was against it when we discussed it in ansible/ansible#8990:

• pkg_delete has no support for -z. This means we would be installing packages that we can not delete using the same name.
• the pkg_add tool is pretty hard to interact with as is, because return codes and such differ depending on if the package contains version numbers etc. This can be seen in the code paths currently in the module where I have tried to document this. Throwing -z in the mix scares me from a complexity standpoint.

There is also another point I did not make last time:

• You can no longer be completely sure what package version is installed on your potentially large number of hosts.

I agree that it is unfortunate to sometimes need to be more specific regarding package names than one would like since it means more upkeep work is needed for playbooks. However, I feel that it is more important to have a strictly controlled environment when managing hosts in bulk.

I hope this makes sense to others.

[bcoca]

If we do add this feature it should have a big warning stating what Patrik notes above.

[eest]

This request has been in the back of my mind for a while. While I am still reluctant to add -z by default for the above reasons, I would probably be OK with adding a new argument to the module which activates it, something like "fuzzy=yes" perhaps. This would make it possible to have the module be as strict as it currently is by default, yet allow people to write less cumbersome playbooks if they want to.

What do people think about this?

[landryb]

That would just make our lifes easier imo. As for the previous points you make, we dont care if pkg_delete doesnt have -z, we can always get the exact pkgstem from another source (command pkg_info etc..) or gather the installed pkg list via facts and delete the correct version.

As for pkg_add being hard to interact with, we're always open to feedback on the pkg_tools, especially in improving return codes.

[dentarg]

This request has been in the back of my mind for a while. While I am still reluctant to add -z by default for the above reasons, I would probably be OK with adding a new argument to the module which activates it, something like "fuzzy=yes" perhaps. This would make it possible to have the module be as strict as it currently is by default, yet allow people to write less cumbersome playbooks if they want to.

Sounds good to me!

[eest]

So I have finally had some time to look into this and it is getting pretty complex right of the bat. The initial problem is this: There is currently a function called get_package_state() which is used to decide if the requested package is installed or not. This is the basis for making further decisions on what needs to be changed if anything. It depends on "pkg_info -e" to check for packages.

I am not sure what would be the correct way to convert a name given to "pkg_add -z" to "pkg_info -e", because the name handling of -z is pretty different. One of the examples given by @landryb is that installing a fuzzy version of "php-fpm-5.6" would pick up "php-fpm-5.5.18p1".

Any input on how this should be handled with pkg_info -e? It might be possible to just use the 'stem' part of the name with some sort of greater-than/less-than check for the version string, but it seems any such decision creates corner cases where the module will wrongly think nothing needs to be done.

[dentarg]

@eest a quick thought, if possible, skip the pkg_info -e step and just execute pkg_add -z?

[eest]

It might be a possible way forward but there are a few drawbacks:

• We would be hammering mirrors every time we are running a playbook which manages packages even if nothing needs to be done.
• The playbook will be slower because of this. I am not sure this is "worth it" for an operation that generally does not change anything except the very first time the playbook is run.

Just from quick testing: it took me about 3-10 seconds to run pkg_add -Iz when the package was already installed, and pkg_info of course returns instantly. This adds up when a playbook handles many packages.

If this is acceptable I guess we could build a package_present_fuzzy() function which does not care about the installed_state and just calls pkg_add -Iz(n) and parses the output from that command to decide if anything changed (or would change).

[dentarg]

That would be okay for me, I think my use case so far only is "nginx", and I only manage a few OpenBSD servers, the situation might be different for others

[landryb]

Fwiw, an option is coming to pkg_add to select package names by 'branch', allowing one to do 'pkg_add autoconf%2.13 or php-fpm%5.6 ... which would solve this issue.

[eest]

@landryb: Thank you for this information, while it might seem like this is a forgotten request I do mull it over from time to time. The combination of sharp corners of the implementation and what I perceive as a generally low request rate of the feature has stopped me from going forward with it currently.

If the pkg_add tool will be gaining functionality that solves the problem then I am more than happy about that. I did a quick glance of the cvs commits but found nothing related to pkg_add branches. I guess it is not yet available in -current?

[landryb]

Soon in -current, will be in 6.0.

[jody-frankowski]

@esst I have just found this issue, and I would love to have this resolved in ansible.
If you used the new pkg_add feature how would it provide support for current/older versions of openbsd?

[eest]

@jody-frankowski: it would not. I am not very interested in adding a bunch of complexity only to cater to old systems. Basically I would rather not have the functionality at all rather than trying to deprecate it in the future. You will have to use specific package names for playbooks aimed at such older systems until replacing them instead.

[landryb]

It's in now in -current, cf http://anoncvs.spacehopper.org/openbsd-src/commit/?id=5cad1d0771d5798c01cac4962dae7c44f10b633b

[jody-frankowski]

@eest Well, I guess some versions could stay in production for long, but that sounds sensible to me.

[landryb]

@jody-frankowski - 'could stay in production' - remember that releases are only supported for one year, so the problem will fix itself once 6.1 is released and 5.9 gets out of support... :)

[jody-frankowski]

@landryb I know, but it doesn't mean some wouldn't keep them in production ;)
To me, one of the problems with this is that some might not be encouraged to use Ansible, because a portion of their infrastructure can't be managed with Ansible (Even if It's their own fault).
Fortunately in our case as @eest said, they could still use specific package names.