chef-boneyard/tar

Deprecated: Chef cookbook for tar packages

tar Cookbook

Installs tar and includes resources for managing remote tar files. tar_package handles remote source package compilation. tar_extract handles retrieving remote tar files and extracting them locally.

Deprecated

Chef 15 now ships with a built-in archive_file resource which handles many archive formats including tar. This new built-in resource should be used instead of this cookbook, which will no longer receive additional updates.

Requirements

Platforms

• Debian / Ubuntu derivatives
• RHEL and derivatives
• openSUSE / SUSE Linux Enterprises
• FreeBSD

Chef

• Chef 12.7+

Resources

tar_package

The tar_package resource provides an easy way to download remote files and compile and install them. This only works for the most basic Autoconf programs that can do ./configure && make && make install.

Actions

• install Installs the package

Properties

• source: name attribute. The source remote URL.
• prefix: Directory to be used as the --prefix configure flag.
• source_directory: Directory to which source files are downloaded.
• creates: prevent the command from running when the specified file already exists.
• configure_flags: Array of additional flags to be passed to ./configure.
• archive_name: Specify a different name for the downloaded archive. Use it if the directory name inside the tar file is different than the name defined in the URL. Additionally, tar_package supports most remote_file attributes.
• tar_binary: Specify the path to the tar binary, if "tar" is insufficient.

Example

tar_package 'http://pgfoundry.org/frs/download.php/1446/pgpool-3.4.1.tar.gz' do
  prefix '/usr/local'
  creates '/usr/local/bin/pgpool'
end

This will download, compile, and install the package from the given URL and install it into /usr/local.

tar_extract

The tar_extract resource provides an easy way to extract tar files from downloaded or local files.

Actions

• extract Extracts the tar file from a url
• extract_local Extracts the tar file from a local file path

Properties

• source: name attribute. The source remote URL.
• target_dir: Directory to extract into, e.g. tar xzf -C (target_dir)
• download_dir: Directory to which tarball is downloaded (defaults to chef cache which requires root group and user).
• creates: prevent the command from running when the specified file already exists.
• compress_char: Flag for compression type, such as z for gzip. man tar for options.
• tar_flags: Array of additional flags to be passed to tar xzf command.
• group: Group name or group ID to extract the archive under. If set to non-root group, point to a download_dir the group has permission to access.
• user: User name or user ID to extract the archive under. If set to non-root user, point to a download_dir the user has permission to access. Additionally, tar_extract supports most remote_file attributes.
• tar_binary: Specify the path to the tar binary, if "tar" is insufficient.

Example

tar_extract 'http://dev.mycoderepo.com/artifacts/mycode-1.2.3.tar.gz' do
  target_dir '/opt/myapp/mycode'
  creates '/opt/myapp/mycode/lib'
  tar_flags [ '-P', '--strip-components 1' ]
end

This will download the tarball to cache, extract the contents to /opt/myapp/mycode, use the file '/opt/myapp/mycode/lib' to determine idempotency, and pass both '-P' and '--strip-components 1' flags to the tar xzf command.

tar_extract '/tmp/mycode-1.2.3.tar.gz' do
  action :extract_local
  target_dir '/opt/myapp/mycode'
  creates '/opt/myapp/mycode/lib'
end

This will extract the contents of /tmp/mycode-1.2.3.tar.gz to /opt/myapp/mycode and use the file '/opt/myapp/mycode/lib' to determine idempotency.

LICENSE AND AUTHOR

• Author: Nathan L Smith ([email protected])
• Author: George Miranda ([email protected])
• Author: Mark Van de Vyver (mark@@taqtiqa.com)

Copyright: 2011, Cramer Development, Inc.
Copyright: 2013, TAQTIQA LLC.
Copyright: 2011-2017, Chef Software, Inc

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

tar's Issues

kitchen converge is stuck

Cookbook version

2.2.0

Chef-client version

14.5.33

Platform Details

ubuntu 16.04 on aws

Scenario:

this is my recipe.rb contents
`
#Install build and test dependencies
package 'build-essential'
package 'tcl'

#Download and extract source code
tar_extract 'http://download.redis.io/redis-stable.tar.gz' do
download_dir '/tmp'
target_dir '/tmp'
creates '/tmp/redis-stable/README.md'

notifies :run,"execute[build and install]",:immediately

end

#Build and install redis
execute 'build and install' do
cwd '/tmp/redis-stable'
command 'make && make test'
action :nothing
subscribes :run,"tar_extract[http://download.redis.io/redis-stable.tar.gz]", :immediately
end

execute 'install build' do
cwd '/tmp/redis-stable'
command 'sudo make install'
creates '/usr/local/bin/redis-server'
end

`
I was writing cookbook for install redis on ubuntu 16.04 for unziping contents download from http://download.redis.io/redis-stable.tar.gz]

this is where till kitchen will converge
Creating a new client identity for redis-test-ubuntu-1604 using the validator key.
resolving cookbooks for run list: ["redis-cookbook::default", "redis-cookbook::redis"]
Synchronizing Cookbooks:
- redis-cookbook (0.1.0)
- tar (2.2.0)
- line (2.1.1)
Installing Cookbook Gems:
Compiling Cookbooks...
Converging 8 resources
Recipe: redis-cookbook::default
* apt_update[daily] action periodic
- update new lists of packages
* directory[/var/lib/apt/periodic] action create (up to date)
* directory[/etc/apt/apt.conf.d] action create (up to date)
* file[/etc/apt/apt.conf.d/15update-stamp] action create_if_missing (up to date)
* execute[apt-get -q update] action run
- execute ["apt-get", "-q", "update"]

   Recipe: redis-cookbook::redis
     * apt_package[build-essential] action install
       - install version 12.1ubuntu2 of package build-essential
     * apt_package[tcl] action install
       - install version 8.6.0+9 of package tcl
     * tar_extract[http://download.redis.io/redis-stable.tar.gz] action extract
       * directory[/tmp] action create (up to date)
       * remote_file[redis-stable.tar.gz] action create
         - create new file /tmp/redis-stable.tar.gz
         - update content in file /tmp/redis-stable.tar.gz from none to 87a2f1
         (new content is binary, diff output suppressed)
         - change mode from '' to '0755'
         - change owner from '' to 'root'
         - change group from '' to 'root'
       * directory[/tmp] action create (up to date)
       * execute[extract /tmp/redis-stable.tar.gz] action run
         - execute tar xfz /tmp/redis-stable.tar.gz
       * execute[extract /tmp/redis-stable.tar.gz] action run (up to date)

from here its stuck

Steps to Reproduce:

[If you are filing an issue what are the things we need to do in order to repro your problem? How are you using this cookbook or any resources it includes?]

Expected Result:

[What are you expecting to happen as the consequence of above reproduction steps?]

Actual Result:

[What actually happens after the reproduction steps? Include the error output or a link to a gist if possible.]

extract with Chef v13.2.21+ fails due to missing environment opt (13.2.21+ `#execute` is backwards incompatible)

tar_extract returns an empty status from /bin/tar not found in PATH, since the newer versions of chef client removed defaults for environment variables when defining execute.

Cookbook version

2.1.1

Chef-client version

13.3.42

Platform Details

OSX 10.12.6 with kitchen docker on a debian armhf6 image

Scenario:

Using the tar_extract recipe with chef-client versions 13.2.21+ normally

Steps to Reproduce:

• Install chef-client version 13.2.21+ (includes the backwards incompatible change for extract to use shell_out_with_systems_locale)
• Launch chef-shell in recipe_mode
• execute the following

> tar_extract 'http://mirrors.peers.community/mirrors/gnu/tar/tar-latest.tar.gz' do
  target_dir '/opt'
  creates '/opt/tar-1.29' # feel free to change this example
end
> resources('tar_extract[http://mirrors.peers.community/mirrors/gnu/tar/tar-latest.tar.gz]').run_action 'extract'

Expected Result:

• an extracted tarball without errors

Actual Result:

chef shell_out is invoked without the env option passed in, and no PATH defaults are set, causing the tar command to not be found [gist with breakpoints illustrating this].

Missing inspec tests

Cookbook version

2.1.1

Chef-client version

N/A

Platform Details

All.

Scenario:

Verify that the cookbook is in a good state.

Actual Result:

The only test is whether tar is available on the system right now which doesn't exercise either of the resources in this cookbook.

Package resource is hardcoded to gzipped tar files

Cookbook version

2.1.1
Latest commit b420a8f on Sep 1

Chef-client version

latest

Platform Details

Any

Scenario:

Package resource is hardcoded to gzipped tar files.

Steps to Reproduce:

use tar_package resource

Expected Result:

I'd expect it to work with other file extensions like tar_extract such as .xz

Actual Result:

Resource fails with no such directory.

See:
https://github.com/chef-cookbooks/tar/blob/master/resources/package.rb#L39
https://github.com/chef-cookbooks/tar/blob/master/resources/package.rb#L56

tar -xf should automatically extract gz, bz2, and xz files without supplying the compression character z, j, or J. These are only required during compression.

Please update the community repo

The version in the community repo is still version 0.2.0.

Add support for FreeBSD

Cookbook version

V1.1.0

Chef-client version

12.17.44

Platform Details

FreeBSD 11.0

Scenario:

include recipe

Make tar command available in the system.

use resource tar_extract

Unpack remote tar.gz archive using resource tar_extract.

Steps to Reproduce:

1. clone this repo to machine with kitchen using vagrant: https://github.com/Katoga/test_tar
2. run kitchen converge
3. FreeBSD box will fail
4. Debian box will converge successfuly

Expected Result:

include recipe

Command tar is ready to use.

use resource tar_extract

Source archive is downloaded and unpacked in #{temp_dir}/tar-1.1.0

Actual Result:

include recipe

Chef run on FreeBSD fails because the system does not have package named tar:

       Converging 1 resources
       Recipe: tar::default
         * freebsd_package[tar] action install

           ================================================================================
           Error executing action `install` on resource 'freebsd_package[tar]'
           ================================================================================

           Mixlib::ShellOut::ShellCommandFailed
           ------------------------------------
           Expected process to exit with [0], but received '69'
           ---- Begin output of pkg rquery '%v' tar ----
           STDOUT:
           STDERR:
           ---- End output of pkg rquery '%v' tar ----
           Ran pkg rquery '%v' tar returned 69

use resource tar_extract

Chef run on FreeBSD fails because the system does not have group named root:

       Converging 1 resources
       Recipe: test_tar::resource
         * tar_extract[https://github.com/chef-cookbooks/tar/archive/v1.1.0.tar.gz] action extract
           * directory[/tmp/kitchen/cache] action create (up to date)
           * remote_file[v1.1.0.tar.gz] action create
             * cannot determine group id for 'root', does the group exist on this system?
             ================================================================================
             Error executing action `create` on resource 'remote_file[v1.1.0.tar.gz]'
             ================================================================================

             Chef::Exceptions::GroupIDNotFound
             ---------------------------------
             cannot determine group id for 'root', does the group exist on this system?

Feature: Add download_name parameter to tar_extract

Request:
I would love a feature in which I could specify the final name of the downloaded Tarball, as the URL to the tarball isn't always the best name.

For example, I pull the tarball from a Sonatype Nexus Maven Repository by hitting a URL like:
http://nexus.mycorp.com/service/local/artifact/maven/redirect?r=releases&g=com.mycompany.server&p=tar.gz&a=MyTarball&v=1.0.0

This results in a very ugly-named tarball in download_dir (The file name ends up being "redirect?r=releases&g=com.mycompany.server&p=tar.gz&a=MyTarball&v=1.0.0"), and I'd like to override that behavior and call it something sensible (e.g. "MyTarball-1.0.0.tar.gz)

Research:
I took a look at providers/extract.rb
The issue I'm having seems to be that source=r.name and local_archive="#{r.download_dir}/#{::File.basename(r.name)}". In short, both the URL and the file name are intertwined on the r.name variable. Essentially, I would like another optional variable to be made available, such that if it's set, it overrides the final name of the downloaded tarball.

Final Comments:
Thank you for this cookbook. It is fantastic, and is critical in my workflow. Even if this feature request isn't completed, I can probably soldier on, but adding this would be yet another boon for our workflow. I'd make the fork and change myself, but I'm not very familiar with LWRP coding (I really only know enough to identity/invoke them), and I'm guessing you'd be able to hook something like this up quite rapidly. If you'd be so kind, I'd greatly appreciate it.

Supermarket link problem

I don't really know how to report this, but the link to this source is broken in the supermarket.chef.io portal, as the "View Source"/"View Issues" redirects to https://github.com/chef-cookbooks/tomcat

tar_extract not working with tar 2.1.1

Cookbook version

[2.1.1]

Chef-client version

[12.5.1]

Platform Details

[CentOS Linux release 7.2.1511]

Scenario:

[Attempting to untar a local file using tar_extract resource]

Steps to Reproduce:

1. include resource in metadata

2. add something to foo::recipe:

tar_extract '/sometar.tar.gz' do
  action :extract_local
  target_dir '/some/location'
  creates '/some/location/file'
end

Expected Result:

tarball unpacked to target directory.

Actual Result:

ERROR: tar_extract[/sometar.tar.gz] (foo::recipe line 90) had an error: NoMethodError: No resource or method named extract_tar' for LWRP resource tar_extract from cookbook tar action provider "/sometar.tar.gz"'

Appears that "tar_extract" switches to "extract_tar" somewhere in the code.

To resolve, I've pinned 'tar', '= 2.0.0' which works fine for my environment. Could my Chef Client version be too old for tar version 2.1.1, or is a fix required?

Please release a new version

Would like to load the newest version from the opscode community site using berkshelf. Thanks.

remote tar file always downloaded

I'm getting the remote tar file always being downloaded on Chef 11.6.0.

Here is the snippet I'm using:

tar_extract "http://download-ln.jetbrains.com/idea/ideaIC-12.1.3.tar.gz" do
  target_dir "/home/vagrant"
  creates "/home/vagrant/idea-IC-129.451"
  download_dir "/home/vagrant"
end

Is there anything wrong with that snippet? Am I misunderstanding something? Is it because the file /home/vagrant/ideaIC-12.1.3.tar.gz already exists?

Improvement - Make execute["extract #{local_archive}"] idempotent by checking all content extracted and not single file

Current logic uses execute resource's create attribute to decide if execute["extract #{local_archive}"] should be run. I'd rather add logic before resource that checks all content already exists in target_dir location. Then check that value through a guard to decide if execute["extract #{local_archive}"] can be skipped or not.

Target dir created by tar_extract is not created with owner and group of the resource

The tar_extract should create the target_dir with the user and group of the resource.

Cookbook version

2.2.0

Chef-client version

14.10

Platform Details

Cloud, Debian, Ubuntu, Centos, etc

Feature: Add checksum support

Request

I'd like to add a optional property named checksum to tar_package and tar_extract to avoid downloading broken file.

Allow directory name inside archive to be different

Cookbook version

2.2.0

Chef-client version

14.8.12

Platform Details

AWS Amazon Linux 2

Scenario:

Install duo_unix

Steps to Reproduce:

tar_package 'https://dl.duosecurity.com/duo_unix-latest.tar.gz' do
  configure_flags [ '--with-pam' ]
  prefix          '/usr'
  creates         '/lib64/security/pam_duo.so'
end

Expected Result:

duo_unix is installed

Actual Result:

The path can't be found

The reason is that the archive is called duo_unix-latest.tar.gz, but the folder inside it the version 'duo_unix-1.11.1'

There is currently no way to specify a different folder name in the module.

I fixed this in my local copy by adding a new variable: dir_name

property :dir_name,              String

action :install do
  r = new_resource
  basename = r.archive_name || ::File.basename(r.name)
  dirname = r.dir_name || basename.chomp('.tar.gz') # Assuming .tar.gz
  src_dir = r.source_directory

Then called:

tar_package 'https://dl.duosecurity.com/duo_unix-latest.tar.gz' do
  configure_flags [ '--with-pam' ]
  prefix          '/usr'
  creates         '/lib64/security/pam_duo.so'
  dir_name        'duo_unix-1.11.1'
end

tar_extract can't use a GID, only a group name.

Cookbook version

2.2.0

Chef-client version

Chef Infra Client: 15.7.32

Platform Details

Fedora release 31 (Thirty One)

Scenario:

The documentation says I can set 'group' to either a group name or a GID. If I set a GID, I get a type error because the property type is set to 'String', rather than ['String', 'Integer'].

Steps to Reproduce:

tar_extract tarball do
target_dir '/home/user'
download_dir '/tmp'
user 'user'
group 1001
creates '/home/user/dir/file'
end

Expected Result:

I'd expect the tar ball to be extracted with files owned by user 'user' and group id 1001.

Actual Result:

A type error is shown because the GID is an integer, not a string.

Fix:

I was able to resolve this issue by changing resources/extract.rb:
Change line 29 from:
property :group, String, default: node['root_group']
to:
property :group, [String, Integer], default: node['root_group']

extract with option strip

Hi
Is there any way to extract with options?
code:
--strip-components=2
Thanks

The prefix tar_extract attribute was removed but README still shows

You still have the prefix attribute in the how-to for the tar_extract resource even though it is no longer defined/included.

Following your example this happens.

   NoMethodError
   -------------
   undefined method `prefix' for Custom resource tar_extract from cookbook tar

Support packages that work with just "make && make install"

Could we please have support for packages that build and install without a configure step (such as atop (http://atoptool.nl/))?

This could either be via an if based on the existence of the configure script, or on an argument of some sort that means no_configure_step. I fairly new to chef, so I don't know what way to handle this properly.

extract_local without creates does nothing

Cookbook version

2.2.0

Chef-client version

12.14.60

Platform Details

Amazon Linux 2018.03 - AWS

Scenario:

I would like to use extract_local to extract a tarball without the "creates" property. Since the tar cookbook does not yet support true idempotence for extracted resources, In my particular case, I wish the extract to occur with every run. This way if the source tarball is changed, the new resources will be unpacked. However, omitting the 'creates' property causes the extract_local action to do nothing.

Steps to Reproduce:

Have a tar.gz file on the local filesystem, i.e. /tmp/test/tar/gz
use the tar_extract resource with the extract_local action, without a creates property

Expected Result:

tar_extract executed the extraction on every chef run

Actual Result:

tar_extract does nothing