suse/machinery

A systems management toolkit for Linux

Machinery

Machinery is a systems management toolkit for Linux. It supports configuration discovery, system validation, and service migration. It's based on the idea of a universal system description.

A spin-off project of Machinery is Pennyworth, which is used to manage the integration test environment.

For more information, visit our website.

Contents

• Installation
• Usage
• Documentation
• Development
• Contact

Installation

Machinery runs on most Linux distributions. Install it by following one of these methods:

• via the one click installer on Machinery's homepage (for openSUSE systems)
• on the command line with zypper on all SUSE distributions
• as a Ruby gem on all distributions which have the gem tool
• from sources

Usage

Machinery is a command-line tool. You can invoke it using the bin/machinery command. It accepts subcommands (similarly to git or bundle).

To display a short overview of available commands and their descriptions, use the help command:

$ machinery help

For more information about the commands, see Machinery Documentation.

Documentation

• User Documentation
• Developer Documentation

Development

The following steps are only recommended if you want to build Machinery from sources, work on the codebase or test the latest development changes. For other distributions than SUSE look also here.

1. Install Git

   $ sudo zypper in git
   

2. Install basic Ruby environment

   $ sudo zypper in ruby rubygem-bundler
   

   After the installation, make sure that your ruby version is at least 2.0.0.p247-3.11.1:

   $ ruby -v
   

   With lower versions, bundle install won't work because of a bug.

3. Install Machinery's dependencies

   Install packages needed to compile Gems with native extensions:

   $ sudo zypper in gcc-c++ make patch ruby-devel libxslt-devel libxml2-devel libvirt-devel
   

   Install Go in order to compile the machinery-helper:

   $ sudo zypper in go
   

4. Clone Machinery repository and install Gem dependencies

   $ git clone [email protected]:SUSE/machinery.git
   $ cd machinery
   $ bundle config build.nokogiri --use-system-libraries
   $ bundle install
   

5. Done!

   You can now start using Machinery by running bin/machinery.

6. Contribute

   Now that you have Machinery running from git on your machine you are ready to hack. If you would like to get some overview of architecture and design of Machinery have a look at our Developer Documentation.

   We are happy if you share your changes with us as pull requests. Read the Contribution Guidelines for details how to do that.

Contact

If you have any question, feel free to open an issue on our GitHub page.

machinery's Issues

Better document the services scope format

The format of the services scope should describe the two different init systems and show how they look like. The description should also contain a list of possible states of services.

`machinery build` of SLES 12 description fails on openSUSE 13.1

When trying to build a SLES 12 image on openSUSE 13.1, the build fails with the Kiwi error:

Aug-19 13:31:25 <3> : Could not find valid boot image description for'vmxboot/suse-SLES12'.

There should either be an error message that building SLES 12 is not supported on openSUSE 13.1, or there should be a message telling what packages to install to make it work.

Broken system description leads to crash

A broken system description (empty file) made machinery crash:

I, [2014-09-30T10:30:06.521755 #30803]  INFO -- : Executing './bin/machinery upgrade-format --all'
E, [2014-09-30T10:30:06.612202 #30803] ERROR -- : Machinery experienced an unexpected error:
E, [2014-09-30T10:30:06.612304 #30803] ERROR -- : undefined method `[]' for nil:NilClass
E, [2014-09-30T10:30:06.612362 #30803] ERROR -- : /data/machinery/lib/migration.rb:72:in `migrate_description'
/data/machinery/lib/upgrade_format_task.rb:34:in `block in upgrade'
/data/machinery/lib/upgrade_format_task.rb:33:in `each'
/data/machinery/lib/upgrade_format_task.rb:33:in `upgrade'
/data/machinery/lib/cli.rb:532:in `block (2 levels) in <class:Cli>'
/usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:in `call'
/usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:in `execute'
/usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:284:in `block in call_command'
/usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:in `call'
/usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:in `call_command'
/usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:79:in `run'
./bin/machinery:25:in `<main>'

HTML diffs don't work

The html view works without the diffs but not after creating them. Most likely special charters are not escaped and the javascript creates an error.

Uncaught SyntaxError: Unexpected number description.js:2
getDescription description.js:2
(anonymous function) machinery.js:3
j jquery-2.1.1.min.js:2
k.fireWith jquery-2.1.1.min.js:2
n.extend.ready jquery-2.1.1.min.js:2
I jquery-2.1.1.min.js:2
´´´

Machinery should show a hint how to upgrade to the current format version

There is no hint how to upgrade to the current format version.

machinery validate no_format_version
The system description no_format_version has an incompatible data format and can not be read.

Installing Machinery RPM installs too many dependencies

When installing the Machinery RPM, it pulls in kiwi and the glance-client, which in turn pull in some perl, python, and Java dependencies. Many users don't need them, and if they are needed Machinery tells the user what to install.

A default installation of Machinery should not install additional programming languages or related dependencies. It should only require Ruby.

This is the output of installing the RPM on an openSUSE 13.1 system, which didn't have Kiwi or the Glance client before:

> sudo zypper install ~/Downloads/machinery-0.20.0.20140915T111429Zgitc09dd58-1.x86_64.rpm 
root's password:
Loading repository data...
Reading installed packages...
Resolving package dependencies...

The following 47 NEW packages are going to be installed:
  bea-stax bsdtar build build-mkbaselibs build-mkdrpms clicfs db45-utils jing kiwi kiwi-tools libcap-progs libdb-4_5 libseccomp2 lxc machinery 
  perl-Class-Singleton perl-common-sense perl-Config-IniFiles perl-File-Slurp perl-JSON perl-JSON-XS perl-List-MoreUtils perl-Readonly perl-satsolver 
  python-argparse python-Babel python-d2to1 python-glanceclient python-iso8601 python-jsonpatch python-jsonpointer python-jsonschema python-keystoneclient 
  python-netaddr python-oslo.config python-pbr python-pip python-PrettyTable python-pyOpenSSL python-requests python-simplejson python-warlock qemu-linux-user 
  saxon9 saxon9-scripts squashfs xerces-j2-xml-resolver 

The following 13 recommended packages were automatically selected:
  bsdtar build build-mkdrpms db45-utils jing kiwi lxc perl-JSON-XS perl-satsolver python-glanceclient qemu-linux-user saxon9-scripts xerces-j2-xml-resolver 

47 new packages to install.
Overall download size: 38.3 MiB. After the operation, additional 143.3 MiB will be used.

Documentation of description of file changes

The documentation of the format of file changes (https://github.com/SUSE/machinery/wiki/System-Description-Format#config-files) can be improved:

• The status should list the possible values in the type column
• The table for modified files includes mode, user, and group, but in the example mode and md5 are shown. The table should list the possible attributes of the changes array.
• The text should be explicit about what is the difference in terms of which attributes have which values between modified files, deleted files, and errors
• Maybe it would be good to have some example snippets for the three cases (modified, deleted, error)
• Make changed managed files section consistent with config files section

Man page should link to documentation of system description format

The man page should link to https://github.com/SUSE/machinery/wiki/System-Description-Format somewhere.

content of cmd variable passed to Cheetah as stdin is not escaped

sudo ./bin/machinery inspect localhost -x -s unmanaged-files

...

Details:

Execution of "/bin/bash" failed with status 2: /bin/bash: line 1: unexpected EOF while looking for matching `'' (...).

Standard output: (none)
Error output:
/bin/bash: line 1: unexpected EOF while looking for matching `''
/bin/bash: line 2: syntax error: unexpected end of file

after some search in my own changes it turned out this actually is caused by:

/usr/src/chuller/machinery/plugins/inspect/unmanaged_files_inspector.rb
line 306:

# compute command line
cmd = "find #{dir} -xdev -maxdepth 1 -maxdepth #{depth} "
cmd += '-printf "%y\0%P\0%l\0"'

# Cheetah seems to be unable to handle binary zeroes "\0" in parameters
# misuse stdin for command
puts cmd + "\n\n\n"
out = system.run_command(
  "/bin/bash",
  {
    :stdin           => cmd,
    :stdout          => :capture,
    :disable_logging => true
  }
)

when looking at the cmd variable, turns out there is this line in it - triggering the error:

find /opt/vagrant/embedded/gems/gems/rb-fsevent-0.9.4/spec/fixtures/folder1 -xdev -maxdepth 1 -maxdepth 6 -printf "%y\0%P\0%l\0"

find /opt/vagrant/embedded/gems/gems/rb-fsevent-0.9.4/spec/fixtures/custom 'path -xdev -maxdepth 1 -maxdepth 6 -printf "%y\0%P\0%l\0"

which comes from this directory:

chuller@skaro:/usr/src/chuller/machinery(review_140731_dont_cut_error_messages)> ls =l/opt/vagrant/embedded/gems/gems/rb-fsevent-0.9.4/spec/fixtures/

/opt/vagrant/embedded/gems/gems/rb-fsevent-0.9.4/spec/fixtures/:
total 8
drwxr-xr-x 2 root root 4096 Jul 31 16:15 custom 'path
drwxr-xr-x 3 root root 4096 Jul 31 16:15 folder1

I am unsure how the command passing to cheetah can be handled differently, however the way we do it right now, the error message is quite useless as one does not see what actually failed and only bash is reported to have run into an issue.

Beside that we need to escape the command sequence put into 'cmd' to avoid such problems in the future. Machinery otherwise will always break here when something in the filesystem contains characters interpreted by bash.

Missing hint when showing help

One point of the description for hints in machinery is
"After showing the help the user gets a hint how to inspect a system to get an initial description"

however, when running
./bin/machinery help
no hints are displayed.

When i run
./bin/machinery --help
I do see a hint though.

Canceling extraction of unmanaged files not handled properly

machinery inspect unmanaged-files doesn't quit with a nice error message when hitting Ctrl + C.

The log looks like this:

I, [2014-09-19T14:15:04.602288 #10421]  INFO -- : Executing '/usr/bin/machinery inspect 127.0.0.1 -s unmanaged-files -n dummy -x'
I, [2014-09-19T14:15:04.605742 #10421]  INFO -- : Running 'ssh -q -o BatchMode=yes [email protected]'
I, [2014-09-19T14:15:04.804219 #10421]  INFO -- : Running 'ssh [email protected] LC_ALL=C rpm --version'
I, [2014-09-19T14:15:05.034766 #10421]  INFO -- : Running 'ssh [email protected] LC_ALL=C sed --version'
I, [2014-09-19T14:15:05.243986 #10421]  INFO -- : Running 'ssh [email protected] LC_ALL=C cat --version'
I, [2014-09-19T14:15:05.435853 #10421]  INFO -- : Running 'ssh [email protected] LC_ALL=C find --version'
I, [2014-09-19T14:15:05.673688 #10421]  INFO -- : Running 'ssh [email protected] LC_ALL=C tar --version'
I, [2014-09-19T14:15:05.913143 #10421]  INFO -- : Running 'ssh [email protected] LC_ALL=C gzip --version'
I, [2014-09-19T14:15:06.168489 #10421]  INFO -- : Running 'ssh [email protected] LC_ALL=C rpm -qlav | sed s/\^\\\(.\\\)\[\^/\]\*\ /\\1\ /'
I, [2014-09-19T14:15:08.894762 #10421]  INFO -- : Running 'ssh [email protected] LC_ALL=C cat /proc/mounts'
I, [2014-09-19T14:16:24.263020 #10421]  INFO -- : Running 'ssh [email protected] LC_ALL=C tar --directory\=/ --create --gzip --null --files-from\=-'
E, [2014-09-19T14:16:28.439795 #10421] ERROR -- : Execution of "ssh [email protected] LC_ALL\=C tar --directory\\\=/ --create --gzip --null --files-from\\\=-" failed with status 255: tar: Removing leading `/' from member names (...).

Error output:
tar: Removing leading `/' from member names
Killed by signal 2.^M

Backtrace:
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/bundle/ruby/2.1.0/gems/cheetah-0.4.0/lib/cheetah.rb:555:in `check_errors'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/bundle/ruby/2.1.0/gems/cheetah-0.4.0/lib/cheetah.rb:364:in `run'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/lib/logged_cheetah.rb:23:in `run'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/lib/remote_system.rb:73:in `run_command'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/lib/system.rb:56:in `create_archive'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/plugins/inspect/unmanaged_files_inspector.rb:88:in `extract_unmanaged_files'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/plugins/inspect/unmanaged_files_inspector.rb:329:in `inspect'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/lib/inspect_task.rb:80:in `block in build_description'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/lib/inspect_task.rb:77:in `each'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/lib/inspect_task.rb:77:in `build_description'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/lib/inspect_task.rb:23:in `inspect_system'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/lib/cli.rb:408:in `block (2 levels) in <class:Cli>'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/bundle/ruby/2.1.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:in `call'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/bundle/ruby/2.1.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:in `execute'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/bundle/ruby/2.1.0/gems/gli-2.11.0/lib/gli/app_support.rb:284:in `block in call_command'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/bundle/ruby/2.1.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:in `call'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/bundle/ruby/2.1.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:in `call_command'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/bundle/ruby/2.1.0/gems/gli-2.11.0/lib/gli/app_support.rb:79:in `run'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140919T115829Zgita6b08ff/bin/machinery:25:in `<top (required)>'
/usr/bin/machinery:26:in `load'
/usr/bin/machinery:26:in `<main>'

Comparison of lists shows too many elements

How to reproduce

1. Inspect a system to create a description: machinery inspect <ip> -n old
2. Install an additional package foo on inspected system
3. Create a new system description with the new state: machinery inspect <ip> -n new
4. Compare descriptions: machinery compare old new --scope packages

The output of the compare command shows all packages of system old and all packages of system new instead of only what has changed:

# Packages

Only in 'old':

* a
* b
* ...

Only in 'new':

* a
* b
* foo
* ...

Expected result

Output of compare command:

# Packages

Only in 'new':
  * foo

Tests

To cover these cases we should add some more tests:

• Generic unit test of Scope#compare_with for scope which contains a list
• Specific unit test for PackagesScope#compare_with
• Unit test for Renderer#show_comparison for the case, when one partial description contains an empty list
• Integration test for compare command

Validation error in changed managed files

The entry

  "changed_managed_files": [
    {
      "name": "/etc/libvirt",
      "package_name": "libvirt-client",
      "package_version": "1.1.2",
      "status": "changed",
      "changes": [
        "replaced"
      ],
      "mode": "700",
      "user": "root",
      "group": "root"
    }
]

gives an error, because replaced is not considered a legal value.

Building openSUSE 13.1 fails because package `opensuse-release-dvd` is missing

Trying to build an openSUSE 13.1 system which includes the package opensuse-release-dvd is failing, because the package is not available in the online repository, which is used for building. This only contains the package opensuse-release-ftp.

Files with broken UTF8 names get extracted during inspection

When inspecting a system with broken UTF8 names files which reside inside a tree get extracted even though during inspection a 'skipped' warning is displayed.

$ ssh [email protected]
$ ls /etc/iscsi
ifaces  iscsid.conf  �test  �testfile test�test
$ ls /etc/iscsi/�test
�testfile2


$ machinery inspect 192.168.121.163 -n utf8-test -s unmanaged-files -x
Inspecting 192.168.121.163 for unmanaged-files...
Inspecting unmanaged-files...
Warning: Filename 'etc/iscsi/�testfile' contains invalid UTF-8 characters. Skipping.
Warning: Filename 'etc/iscsi/�test' contains invalid UTF-8 characters. Skipping.
Warning: Filename 'etc/iscsi/test�test' contains invalid UTF-8 characters. Skipping.
 -> Extracted 1270 unmanaged files and trees.

$ tar tvf ~/.machinery/utf8-test/unmanaged_files/trees/etc/iscsi.tgz 
drwxr-xr-x root/root         0 2014-08-15 10:25 etc/iscsi/
-rw-r--r-- root/root         0 2014-08-15 10:25 etc/iscsi/�testfile
drwxr-xr-x root/root         0 2014-08-15 10:26 etc/iscsi/�test/
-rw-r--r-- root/root         0 2014-08-15 10:26 etc/iscsi/�test/�testfile2
lrwxrwxrwx root/root         0 2014-07-28 14:08 etc/iscsi/ifaces -> /mounts/mp_0001/etc/iscsi/ifaces
drwxr-xr-x root/root         0 2014-08-15 10:30 etc/iscsi/test�test/
-rw-r--r-- root/root     12053 2013-10-22 10:52 etc/iscsi/iscsid.conf

Unexpected error after pressing 'q' during the 'show --show-diffs' on pager.

I tested machinery on sles12 rc1 and upgraded new machinery package.
(machinery-0.17.3.20140804T145530Zgitde36b4f-1.x86_64.rpm)

This problem was happened when I pressed 'q' during the 'pager'

ycho@ix64ph1010:~> machinery show --show-diffs 10.122.2.22
Machinery experienced an unexpected error. Please file a bug report at https://github.com/SUSE/machinery/issues/new.

Details:
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.17.3.20140804T145530Zgitde36b4f/lib/output.rb:35:in write': Broken pipe (Errno::EPIPE) from /usr/lib64/ruby/gems/2.1.0/gems/machinery-0.17.3.20140804T145530Zgitde36b4f/lib/output.rb:35:inputs'
from /usr/lib64/ruby/gems/2.1.0/gems/machinery-0.17.3.20140804T145530Zgitde36b4f/lib/output.rb:35:in block in print_output' from /usr/lib64/ruby/gems/2.1.0/gems/machinery-0.17.3.20140804T145530Zgitde36b4f/lib/output.rb:35:inpopen'
from /usr/lib64/ruby/gems/2.1.0/gems/machinery-0.17.3.20140804T145530Zgitde36b4f/lib/output.rb:35:in print_output' from /usr/lib64/ruby/gems/2.1.0/gems/machinery-0.17.3.20140804T145530Zgitde36b4f/lib/show_task.rb:62:inshow'
from /usr/lib64/ruby/gems/2.1.0/gems/machinery-0.17.3.20140804T145530Zgitde36b4f/lib/cli.rb:443:in block (2 levels) in <class:Cli>' from /usr/lib64/ruby/gems/2.1.0/gems/machinery-0.17.3.20140804T145530Zgitde36b4f/bundle/ruby/2.1.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:incall'
from /usr/lib64/ruby/gems/2.1.0/gems/machinery-0.17.3.20140804T145530Zgitde36b4f/bundle/ruby/2.1.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:in execute' from /usr/lib64/ruby/gems/2.1.0/gems/machinery-0.17.3.20140804T145530Zgitde36b4f/bundle/ruby/2.1.0/gems/gli-2.11.0/lib/gli/app_support.rb:284:inblock in call_command'
from /usr/lib64/ruby/gems/2.1.0/gems/machinery-0.17.3.20140804T145530Zgitde36b4f/bundle/ruby/2.1.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:in call' from /usr/lib64/ruby/gems/2.1.0/gems/machinery-0.17.3.20140804T145530Zgitde36b4f/bundle/ruby/2.1.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:incall_command'
from /usr/lib64/ruby/gems/2.1.0/gems/machinery-0.17.3.20140804T145530Zgitde36b4f/bundle/ruby/2.1.0/gems/gli-2.11.0/lib/gli/app_support.rb:79:in run' from /usr/lib64/ruby/gems/2.1.0/gems/machinery-0.17.3.20140804T145530Zgitde36b4f/bin/machinery:25:in<top (required)>'
from /usr/bin/machinery:26:in load' from /usr/bin/machinery:26:in

'

Show format version in output of `machinery --version`

To be able to tell what format version the currently installed machinery binary supports it would be helpful if machinery --version would put out the format version.

Can't install machinery package on openSUSE 13.1

Steps to reproduce:

1. Obtain a machine/VM with clean openSUSE13.1 installed.

2. Download the current machinery package built by CI (I tested with this one).

3. Try to install the downloaded RPM using rpm:

   tux@dhcp143:~> sudo rpm -i ~/Downloads/machinery-0.20.0.20141007T145542Zgit0b3ed0d-1.x86_64.rpm 
   

4. Observe the following failure:

   error: unpacking of archive failed on file /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.20.0.20141007T145542Zgit0b3ed0d/bundle/ruby/2.0.0/cache/nokogiri-1.6.0.gem;5434f239: cpio: read failed - No such file or directory
   error: machinery-0.20.0.20141007T145542Zgit0b3ed0d-1.x86_64: install failed
   

Expected results:

Package is installed.

machinery inspect -x localhost packaged nfs mounted /suse directory

I ran
machinery inspect -x localhost
over night and when i came back my local harddrive was filled up and the machinery job had exited with an error mesage.
Turns out that
~/.machinery/localhost/unmanaged-files/trees
contains a file called suse.tgz which is 51GB in size and contains everything from nfs mounted /suse that fitted into it before the free diskspace in / ran out, which in effect triggered the error message.

We should not include nfs mounted shares or find a way to switch the inclusion/exclusion of such filesystems.

Consistently use the term "schema"

At the moment the text sometimes says "schema" and sometimes "scheme", especially when referring to the format of a scope. We should only use one term here.

I think when referring to the scope, we should also say "scope", e.g. "group information is stored in the groups scope.".

Only when referring to the schema itself, i.e. the formal description of the format, we should say "schema".

Invalid UTF-8 in /etc/passwd breaks Machinery

Steps to reproduce:

1. Embed an invalid UTF-8 sequence (such as 0xC0 byte) into /etc/passwd on some machine.
2. Inspect that machine using machinery inspect --scope users <host>
3. Watch Machinery fail with an exception.

Machinery experienced an unexpected error. Please file a bug report at https://github.com/SUSE/machinery/issues/new.
/home/dmajda/Alfred/machinery/plugins/inspect/users_inspector.rb:32:in `split': invalid byte sequence in UTF-8 (ArgumentError)
        from /home/dmajda/Alfred/machinery/plugins/inspect/users_inspector.rb:32:in `block in parse_users'
        from /home/dmajda/Alfred/machinery/plugins/inspect/users_inspector.rb:32:in `map'
        from /home/dmajda/Alfred/machinery/plugins/inspect/users_inspector.rb:32:in `parse_users'
        from /home/dmajda/Alfred/machinery/plugins/inspect/users_inspector.rb:23:in `inspect'
        from /home/dmajda/Alfred/machinery/lib/inspect_task.rb:101:in `block in build_description'
        from /home/dmajda/Alfred/machinery/lib/inspect_task.rb:98:in `each'
        from /home/dmajda/Alfred/machinery/lib/inspect_task.rb:98:in `build_description'
        from /home/dmajda/Alfred/machinery/lib/inspect_task.rb:23:in `inspect_system'
        from /home/dmajda/Alfred/machinery/lib/cli.rb:386:in `block (2 levels) in <class:Cli>'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:in `call'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:in `execute'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:284:in `block in call_command'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:in `call'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:in `call_command'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:79:in `run'
        from bin/machinery:25:in `<main>'

Add missing description to format documentation

There are a couple of missing descriptions in the format documentation:

• users
• repositories
• services

Validation error for groups on NIS system

When the /etc/passwd contains an entry +::::::, which happens when the system is using NIS, the inspected system description can't be used anymore, because it always throws an validation error because of missing fields.

File extraction doesn't respect network file systems

When running an extraction of unmanaged files, Machinery doesn't respect network file systems and just extracts all data there. This can be prohibitively expensive, when it's a large file system, e.g. user home directories on an NFS directory.

The file extraction should stop at NFS mount points.

The general problem behind this is a bit broader, because there are many ways, how large file systems can make the unmanaged file extraction impractical, so we will need some more ways how to handle this properly. Stopping at NFS mount points probably is a safe bet, though. Samba mounts should be treated the same way.

machinery exits with zero exit code although a warning occurred

When a warning happens machinery should exit with a non-zero exit code in order to allow using machinery in automated scripts.

Inspecting 192.168.121.166 for unmanaged-files...
Inspecting unmanaged-files...
Warning: Filename 'usr/local/broken�filename' contains invalid UTF-8 characters. Skipping.
 -> Found 1271 unmanaged files and trees.
# echo $?
0

machinery upgrade-format --all should write to log which description is upgraded

machinery upgrade-format --all should write to the log which description it's going to be upgrade. Otherwise it's hard to find out with description caused an error.

Invalid UTF-8 in /etc/os-release breaks Machinery

Steps to reproduce:

1. Embed an invalid UTF-8 sequence (such as 0xC0 byte) into /etc/os-release on some machine.
2. Inspect that machine using machinery inspect --scope os <host>
3. Watch Machinery fail with an exception.

Machinery experienced an unexpected error. Please file a bug report at https://github.com/SUSE/machinery/issues/new.
/home/dmajda/Alfred/machinery/plugins/inspect/os_inspector.rb:49:in `split': invalid byte sequence in UTF-8 (ArgumentError)
        from /home/dmajda/Alfred/machinery/plugins/inspect/os_inspector.rb:49:in `analyze_os_release_file'
        from /home/dmajda/Alfred/machinery/plugins/inspect/os_inspector.rb:100:in `inspect'
        from /home/dmajda/Alfred/machinery/lib/inspect_task.rb:101:in `block in build_description'
        from /home/dmajda/Alfred/machinery/lib/inspect_task.rb:98:in `each'
        from /home/dmajda/Alfred/machinery/lib/inspect_task.rb:98:in `build_description'
        from /home/dmajda/Alfred/machinery/lib/inspect_task.rb:23:in `inspect_system'
        from /home/dmajda/Alfred/machinery/lib/cli.rb:386:in `block (2 levels) in <class:Cli>'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:in `call'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:in `execute'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:284:in `block in call_command'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:in `call'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:in `call_command'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:79:in `run'
        from bin/machinery:25:in `<main>'

Inspecting a System using NIS creates a broken manifest

Inspecting an System with an enabled NIS creates an manifest with a broken group section.

    {
      "name": "+",
      "password": "",
      "users": [

      ]     
    },    

The following error is provided:

In scope groups: The property #0 did not contain a required property of 'gid'.

Fakefs has issues with newer Ruby versions

I ran the unit tests on openSUSE Factory with ruby-2.1.2-1.2.x86_64 and got 10 failures with the message "uninitialized constant FakeFS::Dir::Tmpname".

An at least similar issue is already reported against Fakefs: fakefs/fakefs#243

Integration test for overlay file with ` '` in its name

We need to extend our integration test to cover the case fixed in pull request #33.

Handle concurrent access

Data corruption can be caused, when multiple machinery processes operate on the system description at the same time.

This could be prevented by implementing locking. At least it should be documented that parallel access must be avoided.

Document structure of extracted changed managed files

We need to document the structure of the content of the changed-managed-files sub directory.

Aborting inspection causes confusing error message

Pressing Ctrl + C while changed-managed-files inspection leads to confusing error message.

vagrant@linux-oewt:~> machinery inspect 127.0.0.1 -n dummy -x
Inspecting 127.0.0.1 for repositories,packages,users,changed-managed-files,config-files,os,patterns,services,groups,unmanaged-files...
Inspecting repositories...
 -> Found 1 repositories.
Inspecting packages...
 -> Found 376 packages.
Inspecting users...
 -> Found 18 users.
Inspecting changed-managed-files...
^CMachinery experienced an unexpected error. Please file a bug report at https://github.com/SUSE/machinery/issues/new.
Execution of "ssh [email protected] LC_ALL\=C bash -c \\\#\\\!/bin/bash\''
'\'\\\#\\\ Copyright\\\ \\\(c\\\)\\\ 2013-2014\\\ SUSE\\\ LLC\''
'\'\\\#\''
'\'\\\#\\\ This\\\ program\\\ is\\\ free\\\ software\\\;\\\ you\\\ can\\\ redistribute\\\ it\\\ and/or\''
'\'\\\#\\\ modify\\\ it\\\ under\\\ the\\\ terms\\\ of\\\ version\\\ 3\\\ of\\\ the\\\ GNU\\\ General\\\ Public\\\ License\\\ as\''
'\'\\\#\\\ published\\\ by\\\ the\\\ Free\\\ Software\\\ Foundation.\''
'\'\\\#\''
'\'\\\#\\\ This\\\ program\\\ is\\\ distributed\\\ in\\\ the\\\ hope\\\ that\\\ it\\\ will\\\ be\\\ useful,\''
'\'\\\#\\\ but\\\ WITHOUT\\\ ANY\\\ WARRANTY\\\;\\\ without\\\ even\\\ the\\\ implied\\\ warranty\\\ of\''
'\'\\\#\\\ MERCHANTABILITY\\\ or\\\ FITNESS\\\ FOR\\\ A\\\ PARTICULAR\\\ PURPOSE.\\\ \\\ \\\ See\\\ the\''
'\'\\\#\\\ GNU\\\ General\\\ Public\\\ License\\\ for\\\ more\\\ details.\''
'\'\\\#\''
'\'\\\#\\\ You\\\ should\\\ have\\\ received\\\ a\\\ copy\\\ of\\\ the\\\ GNU\\\ General\\\ Public\\\ License\''
'\'\\\#\\\ along\\\ with\\\ this\\\ program\\\;\\\ if\\\ not,\\\ contact\\\ SUSE\\\ LLC.\''
'\'\\\#\''
'\'\\\#\\\ To\\\ contact\\\ SUSE\\\ about\\\ this\\\ file\\\ by\\\ physical\\\ or\\\ electronic\\\ mail,\''
'\'\\\#\\\ you\\\ may\\\ find\\\ current\\\ contact\\\ information\\\ at\\\ www.suse.com\''
'\'\''
'\'\''
'\'\\\#\\\ Print\\\ a\\\ list\\\ of\\\ each\\\ package\\\ with\\\ changed\\\ managed\\\ files\\\ followed\\\ by\\\ a\\\ list\\\ of\''
'\'\\\#\\\ the\\\ changed\\\ files,\\\ e.g.\''
'\'\\\#\''
'\'\\\#\\\ \\\ \\\ libpulse0-4.0.git.270.g9490a:\''
'\'\\\#\\\ \\\ \\\ S.5......\\\ \\\ c\\\ /etc/pulse/client.conf\''
'\'\\\#\\\ \\\ \\\ ntp-4.2.6p5:\''
'\'\\\#\\\ \\\ \\\ S.5......\\\ \\\ c\\\ /etc/ntp.conf\''
'\'for\\\ package\\\ in\\\ \\\`rpm\\\ -qa\\\ --queryformat\\\ \\\"\\\%\\\{NAME\\\}-\\\%\\\{VERSION\\\}\\\\\\\\n\\\"\\\`\\\;\\\ do\''
'\'\\\ \\\ CHANGES\\\=\\\`rpm\\\ -V\\\ --nodeps\\\ --nodigest\\\ --nosignature\\\ --nomtime\\\ --nolinkto\\\ \\\$package\\\`\\\;\''
'\'\\\ \\\ if\\\ \\\[\\\ -n\\\ \\\"\\\$CHANGES\\\"\\\ \\\]\\\;\\\ then\''
'\'\\\ \\\ \\\ \\\ echo\\\ -e\\\ \\\"\\\$package:\\\\\\\\n\\\$CHANGES\\\"\\\;\''
'\'\\\ \\\ fi\\\;\''
'\'done\''
.\'" failed with status 255: Killed by signal 2.

Error output:
Killed by signal 2.

Standard output:
openssh-6.6p1:
S.5......  c /etc/ssh/sshd_config


Backtrace:
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/bundle/ruby/2.1.0/gems/cheetah-0.4.0/lib/cheetah.rb:555:in `check_errors'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/bundle/ruby/2.1.0/gems/cheetah-0.4.0/lib/cheetah.rb:364:in `run'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/lib/logged_cheetah.rb:23:in `run'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/lib/remote_system.rb:73:in `run_command'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/lib/system.rb:78:in `run_script'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/plugins/inspect/changed_managed_files_inspector.rb:55:in `changed_files'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/plugins/inspect/changed_managed_files_inspector.rb:25:in `inspect'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/lib/inspect_task.rb:80:in `block in build_description'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/lib/inspect_task.rb:77:in `each'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/lib/inspect_task.rb:77:in `build_description'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/lib/inspect_task.rb:23:in `inspect_system'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/lib/cli.rb:408:in `block (2 levels) in <class:Cli>'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/bundle/ruby/2.1.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:in `call'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/bundle/ruby/2.1.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:in `execute'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/bundle/ruby/2.1.0/gems/gli-2.11.0/lib/gli/app_support.rb:284:in `block in call_command'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/bundle/ruby/2.1.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:in `call'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/bundle/ruby/2.1.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:in `call_command'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/bundle/ruby/2.1.0/gems/gli-2.11.0/lib/gli/app_support.rb:79:in `run'
/usr/lib64/ruby/gems/2.1.0/gems/machinery-0.20.0.20140916T181052Zgit44ef7d2/bin/machinery:25:in `<top (required)>'
/usr/bin/machinery:26:in `load'
/usr/bin/machinery:26:in `<main>'

/usr/bin/machinery: Interrupt

vagrant@linux-oewt:~> 

`machinery list` throws exception

When calling machinery list from the 0.19.0 package on an openSUSE 13.1 machine which already has some descriptions created with previous version, there is an exception (names :

The system description x has an incompatible data format and can not be read.
The system description y has an incompatible data format and can not be read.
The system description z has an incompatible data format and can not be read.
Machinery experienced an unexpected error. Please file a bug report at https://github.com/SUSE/machinery/issues/new.
/usr/lib64/ruby/gems/2.0.0/gems/machinery-0.19.0/lib/system_description.rb:64:in `const_get': wrong constant name Changed-managed-filesScope (NameError)
        from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.19.0/lib/system_description.rb:64:in `block in create_attrs'
        from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.19.0/lib/system_description.rb:60:in `each'
        from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.19.0/lib/system_description.rb:60:in `map'
        from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.19.0/lib/system_description.rb:60:in `create_attrs'
        from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.19.0/lib/system_description.rb:51:in `from_json'
        from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.19.0/lib/system_description_store.rb:47:in `load'
        from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.19.0/lib/list_task.rb:25:in `block in list'
        from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.19.0/lib/list_task.rb:22:in `each'
        from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.19.0/lib/list_task.rb:22:in `list'
        from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.19.0/lib/cli.rb:429:in `block (2 levels) in <class:Cli>'
        from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.19.0/bundle/ruby/2.0.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:in `call'
        from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.19.0/bundle/ruby/2.0.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:in `execute'
        from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.19.0/bundle/ruby/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:284:in `block in call_command'
        from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.19.0/bundle/ruby/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:in `call'
        from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.19.0/bundle/ruby/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:in `call_command'
        from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.19.0/bundle/ruby/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:79:in `run'
        from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.19.0/bin/machinery:25:in `<top (required)>'
        from /usr/bin/machinery:26:in `load'
        from /usr/bin/machinery:26:in `<main>'

Release isn't tagged

There is release 0.17.3 in the build service (http://download.opensuse.org/repositories/systemsmanagement:/machinery/openSUSE_13.1/x86_64/), but it isn't tagged in git (https://github.com/SUSE/machinery/releases).

Not sure, if this a one-off because of the move of the repository, or if this is a bug in our release scripts.

All releases should be tagged in git and available on the releases page.

Add quotes around system description name in output of compatibility check.

I'd propose to add quotes around the description name.
The system description perelman.suse.de has an incompatible data format and can not be read.

Document structure of extracted config-files

We need to document the cpntent of the config-files directory

Cancelling extraction of unmanaged files leaves inconsistent state

When cancelling the extraction of unmanaged files with Ctrl-C, the unmanaged_files directory is left in an incomplete state. machinery list shows the unmanaged files to be extracted, but they actually are missing part of the data.

Keeping the files there can be useful in some cases, e.g. when diagnosing problems with the extraction. The user should get a hint, though, that there is incomplete data left, if we keep it around.

The state of an incomplete extraction should not be reported as extracted.

Broken UTF8 warnings don't honor capitalization

Warnings for broken UTF8 filenames remove all capitalization from the filename and the path to the file.

$ ls /TEST/
�TeSt


$ machinery inspect 192.168.121.163 -n utf8-test2 -s unmanaged-files -x
Inspecting 192.168.121.163 for unmanaged-files...
Inspecting unmanaged-files...
Warning: Filename 'test/�test' contains invalid UTF-8 characters. Skipping.
 -> Extracted 1271 unmanaged files and trees.

Document structure of extracted unmanaged files

We need to document the content of the unmanaged-files directory, how the files are structured there.

Invalid UTF-8 in /etc/group breaks Machinery

Steps to reproduce:

1. Embed an invalid UTF-8 sequence (such as 0xC0 byte) into /etc/group on some machine.
2. Inspect that machine using machinery inspect --scope groups <host>
3. Watch Machinery fail with an exception.

Machinery experienced an unexpected error. Please file a bug report at https://github.com/SUSE/machinery/issues/new.
/home/dmajda/Alfred/machinery/plugins/inspect/groups_inspector.rb:32:in `split': invalid byte sequence in UTF-8 (ArgumentError)
        from /home/dmajda/Alfred/machinery/plugins/inspect/groups_inspector.rb:32:in `block in parse_groups'
        from /home/dmajda/Alfred/machinery/plugins/inspect/groups_inspector.rb:31:in `map'
        from /home/dmajda/Alfred/machinery/plugins/inspect/groups_inspector.rb:31:in `parse_groups'
        from /home/dmajda/Alfred/machinery/plugins/inspect/groups_inspector.rb:22:in `inspect'
        from /home/dmajda/Alfred/machinery/lib/inspect_task.rb:101:in `block in build_description'
        from /home/dmajda/Alfred/machinery/lib/inspect_task.rb:98:in `each'
        from /home/dmajda/Alfred/machinery/lib/inspect_task.rb:98:in `build_description'
        from /home/dmajda/Alfred/machinery/lib/inspect_task.rb:23:in `inspect_system'
        from /home/dmajda/Alfred/machinery/lib/cli.rb:386:in `block (2 levels) in <class:Cli>'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:in `call'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:in `execute'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:284:in `block in call_command'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:in `call'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:in `call_command'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:79:in `run'
        from bin/machinery:25:in `<main>'

Broken UTF8 files in directories in unmanaged trees aren't detected

When inspecting a system broken UTF8-files which are not on the top level of an unmanaged tree tarball do not get detected, and as such are extracted.

$ ls /etc/iscsi
testdir  ifaces  iscsid.conf  �test

$ ls /etc/iscsi/testdir
�test


$ machinery inspect 192.168.121.209 -s unmanaged-files -x
Inspecting 192.168.121.209 for unmanaged-files...
Inspecting unmanaged-files...
Warning: Filename 'etc/iscsi/�test' contains invalid UTF-8 characters. Skipping.
 -> Extracted 1150 unmanaged files and trees.

$ tar tvf ~/.machinery/192.168.121.209/unmanaged_files/trees/etc/iscsi.tgz 
drwxr-xr-x root/root         0 2014-08-19 15:58 etc/iscsi/
-rw-r--r-- root/root     11802 2014-07-09 06:06 etc/iscsi/iscsid.conf
drwxr-xr-x root/root         0 2014-08-19 15:58 etc/iscsi/asd/
-rw-r--r-- root/root         0 2014-08-19 15:58 etc/iscsi/asd/�test
drwxr-xr-x root/root         0 2014-07-15 13:46 etc/iscsi/ifaces/
-rw-r--r-- root/root      5983 2014-07-09 06:06 etc/iscsi/ifaces/iface.example

Invalid UTF-8 in a repository alias breaks Machinery

Steps to reproduce:

1. Add a repository with an invalid UTF-8 sequence (such as 0xC0 byte) in the alias on some machine. This can be done for example using this command:

   $ ruby -e '`sudo zypper ar http://download.opensuse.org/repositories/devel:/languages:/ruby/openSUSE_13.1/ \xc0`'
   

2. Inspect that machine using machinery inspect --scope repositories <host>

3. Watch Machinery fail with an exception.

Machinery experienced an unexpected error. Please file a bug report at https://github.com/SUSE/machinery/issues/new.
/home/dmajda/Alfred/machinery/plugins/inspect/repositories_inspector.rb:29:in `split': invalid byte sequence in UTF-8 (ArgumentError)
        from /home/dmajda/Alfred/machinery/plugins/inspect/repositories_inspector.rb:29:in `inspect'
        from /home/dmajda/Alfred/machinery/lib/inspect_task.rb:101:in `block in build_description'
        from /home/dmajda/Alfred/machinery/lib/inspect_task.rb:98:in `each'
        from /home/dmajda/Alfred/machinery/lib/inspect_task.rb:98:in `build_description'
        from /home/dmajda/Alfred/machinery/lib/inspect_task.rb:23:in `inspect_system'
        from /home/dmajda/Alfred/machinery/lib/cli.rb:386:in `block (2 levels) in <class:Cli>'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:in `call'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:in `execute'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:284:in `block in call_command'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:in `call'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:in `call_command'
        from /usr/lib64/ruby/gems/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:79:in `run'
        from bin/machinery:25:in `<main>'

Tests shouldn't generally stub standard output

In the spec_helper.rb STDOUT#puts and STDERR#puts are stubbed for all tests. This makes debugging tests harder than necessary. Statements added to the code printing debug information don't work anymore and for example using pry as debugger is broken as well.

Stubbing them generally also hides all problems where the code is printing something where it shouldn't.

We should not stub these functions generally, but only in those tests where we explicitly want to test output.

/usr/lib64/ruby/gems/2.0.0/gems/machinery-0.18.0/plugins/inspect/unmanaged_files_inspector.rb:173:in `split': invalid byte sequence in UTF-8 (ArgumentError

I ran "sudo machinery inspect localhost" on my 13.1 system and got the following error:

/usr/lib64/ruby/gems/2.0.0/gems/machinery-0.18.0/plugins/inspect/unmanaged_files_inspector.rb:173:in split': invalid byte sequence in UTF-8 (ArgumentError) from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.18.0/plugins/inspect/unmanaged_files_inspector.rb:173:inget_find_data'
from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.18.0/plugins/inspect/unmanaged_files_inspector.rb:239:in inspect' from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.18.0/lib/inspect_task.rb:97:inblock in build_description'
from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.18.0/lib/inspect_task.rb:94:in each' from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.18.0/lib/inspect_task.rb:94:inbuild_description'
from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.18.0/lib/inspect_task.rb:23:in inspect_system' from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.18.0/lib/cli.rb:375:inblock (2 levels) in class:Cli'
from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.18.0/bundle/ruby/2.0.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:in call' from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.18.0/bundle/ruby/2.0.0/gems/gli-2.11.0/lib/gli/command_support.rb:126:inexecute'
from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.18.0/bundle/ruby/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:284:in block in call_command' from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.18.0/bundle/ruby/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:incall'
from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.18.0/bundle/ruby/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:297:in call_command' from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.18.0/bundle/ruby/2.0.0/gems/gli-2.11.0/lib/gli/app_support.rb:79:inrun'
from /usr/lib64/ruby/gems/2.0.0/gems/machinery-0.18.0/bin/machinery:25:in <top (required)>' from /usr/bin/machinery:26:inload'
from /usr/bin/machinery:26:in `

'

Use consistent capitalization in tables of format documentation

The description field should use consistent capitalization. At the moment it's mixed. I would suggest to use lower case everywhere.

Show which systems can be built in output of command line tool

Currently the command line tool refers to the build matrix in the man page. This has the problem that the output is not related to the context and that the man page has to manually be synchronized with the code, which defines the support matrix.

As we know on which platform we are running, we can show which platforms are supported to build. We could also show, on platform the requested build would be supported.

As output of machinery build --help we could show the complete build matrix, generated from the definition of build support in the code.

Machinery extracts unchanged config-files

Machinery does also extract config-files/changed-managed-files where only the meta data was changed like user permissions. This doesn't really make sense.

machinery upgrade-format --all exits when upgrading one descriptions fails

machinery upgrade-format --all exits when upgrading one descriptions fails. I'd expect it to continue on the next description.

machinery doesn't show if file scopes were extracted

machinery list and machinery show don't show if file scopes were extracted