chef / chef-analyze Goto Github PK

View Code? Open in Web Editor NEW

6.0 41.0 6.0 15.14 MB

A CLI to analyze artifacts from a Chef Infra Server

License: Apache License 2.0

Shell 3.40% Go 95.90% PowerShell 0.46% Makefile 0.24%

chef-analyze chef-infra-server cli reporting cookbooks nodes

chef-analyze's Introduction

chef-analyze

A CLI to analyze artifacts from a Chef Infra Server.

Umbrella Project: Chef Workstation

Project State: Prototyping

Issues Response Time Maximum: Not yet defined

Pull Request Response Time Maximum: Not yet defined

Installation

We highly recommend using Chef Workstation, which includes chef-analyze out of the box. If for some reason you can't use Chef Workstation you can manually install the Habitat package chef/chef-analyze.

hab pkg install -b chef/chef-analyze

Development Documentation

The development of this CLI is being done inside a Chef Habitat Studio, you will need to have Chef Habitat installed on your local workstation to proceed with any development task.

Building

From within a Chef Habitat Studio, you can build the chef-analyze CLI by:

Building a native Habitat package.

$ hab studio enter
[1][default:/src:0]# build

Helper method to build cross-platform binaries.
```
$ hab studio enter
[1][default:/src:0]# build_cross_platform
```
NOTE: The generated binaries will be located inside the bin/ directoy.

Testing

From within a Chef Habitat Studio, you can run both, unit and integration tests:

Unit tests. (Go-based)

$ hab studio enter
[1][default:/src:0]# unit_tests

Integration tests. (Go-based)
```
$ hab studio enter
[1][default:/src:0]# integration_tests
```
NOTE: The integration tests require a binary to test against, this helper automatically triggers a cross-platform build and uses the generated binary for the running platform.

Code coverage

This repository requires any change to always increase, or at least, maintain the percentage of code coverage, to execute the current coverage run:

$ hab studio enter
[1][default:/src:0]# code_coverage

For details about the code coverage open the generated HTML report located at coverage/coverage.html.

Patching a local Chef Workstation Install

You can override the chef-analyze binary that comes inside your local Chef Workstation install by running make patch_local_workstation at the top level folder of this repository. Then just simply run chef-analyze or chef analyze to use the top-level chef wrapper.

Contributing

For information on contributing to this project please see our Contributing Documentation

License & Copyright

Copyright:: Copyright (c) 2019 Chef Software, Inc.
License:: Apache License, Version 2.0

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

chef-analyze's People

Contributors

Stargazers

Watchers

Forkers

msystechnologiesllc tarcinil isabella232 ekmixon ghas-results

chef-analyze's Issues

Add buildkite test for verification

Motivation

Add Tests to verify chef-analyze functionality.

Possible Solution

TBA but for sure in Buildkite.

Use Temporal directories on Unit Tests

Describe the problem

Placeholder for updating the unit tests to use Temporal Directories.

Possible Solution

Try to mock the PWDenvironment variable.

[SPIKE] Inaccurate reports when nodes are using Policyfiles

Description

Currently, chef-analyze presents two reports to the users, a nodes report and
a cookbooks report. Both of these reports focus only on environments that are
running a very old version of Chef Client. The reports do not cover newer deployment
patterns like Policyfiles causing users to misunderstand the reports and actually,
causing confusion with the current state of their infrastructure.

Users that are in transition to migrate to the newer patterns, that is, users that have
both, old and new patterns will generate partial reports with inaccurate data.

Additionally, users on the latest patterns like Policyfiles and Effortless won't have any
benefit from using chef analyze.

Chef Analyze Version

The latest version from master.

Acceptance Criteria

What is missing or inaccurate from the chef analyze report data?
Why is that data inaccurate?
File a new card that address the above bullet point and ensures chef analyze report works for nodes using Policyfiles
Get backup from the Operations team that we can use to troubleshoot and reproduce this issue

Dependabot can't parse your go.mod

Dependabot couldn't parse the go.mod found at /go.mod.

The error Dependabot encountered was:

go: github.com/ory/[email protected] requires
	cloud.google.com/[email protected] requires
	honnef.co/go/[email protected]: unrecognized import path "honnef.co/go/tools" (https fetch: Get https://honnef.co/go/tools?go-get=1: EOF)

Move Go libraries out from chef-analyze repo

Describe

We need to move all the Go libraries like, featflag, config, credentials, etc.
to a more centralized and generic repo so that other Go projects can depend on.

Proposed Repo: https://github.com/chef/go-libs

In reporting/nodes, use pointer for interfaces and structs

This will prevent a lot of value copying at runtime.

Have appropriate environments to test this tool

As a chef-analyze developer,
I need to have appropriate environments to test and develop chef-analyze,
So I can ensure that this tool will help real customers with real data and to cover all possible scenarios.

Acceptance Criteria

Ability to create ephemeral environments (able to be created and destroyed at will by engineers) to test against
Environments should be able to be created from a customer backup
By default, only the engineer who created an environment will have access to it
Customer backups should be secured from access by any non-chef employee

Implementation notes

Modify https://github.com/chef/chef-server/tree/master/terraform so others can consume these enhancements

Add node filtering for chef-analyze reports

As an administrator upgrading my organization's Chef-powered infrastructure
I want to filter the cookbooks and nodes that chef-analyze reports on
so that I can slice up my upgrade work in a way that best suits my environment.

Specification

When using chef-analyze to report on nodes and cookbook usage, accept a flag --node-filter that is compatible with knife search syntax.

Example usage:

chef analyze report cookbooks --node-filter 'platform:ubuntu AND chef_environment:development'

For nodes reports, the filter is straight-forward - we will only report on nodes that match the filter.

For cookbooks reports, the filter is applied when determining which nodes use a given discovered cookbooks. For example, you could filter out out all nodes that uses cookbook 'apache2'. Assuming that apache2 is an the role run list for role webservers, a filter like NOT roles::webservers, your list of in-use cookbooks may not include apache2.

When the operator is aware of this behavior, it can be very helpful - for example, they may only want to operate on isolated nodes identified by roles, tags, environments, etc. But if they are unaware of the behavior it could be misleading.

Making clear that the resulting data is incomplete both in the on-screen output and in the name of the generated report should help mitigate this risk.

Acceptance Criteria

Existing POC has test coverage
Rebased and updated with current master

Downstream Impact

None

Add --only-unused flag to cookbooks report

Describe the Enhancement:

Update the default behavior of the reports to only include cookbooks that are assigned to nodes. Remove the --skip-unused flag because that becomes the default behavior.

Create an --only-unused flag so that users can have a report that displays
cookbooks that are NOT used by any node. This is the nemesis of the flag
--skip-unused that do not include unused cookbooks that are not applied
to any nodes.

Describe the Need:

As a user, I would benefit from this report to know and understand how many
and, which cookbooks are not being used. Probably, even remove them.

Current Alternative

Use https://github.com/chef/knife-tidy

[chef_employee] Create node oriented infrastructure report

Motivation

As a Chef employee,
I want to know the current state of customer infrastructure,
so I can build better reports about how to upgrade their infrastructure.

Specification

Create a report that looks like this:

Node Name	Chef Client Infra Version	Operating System	OS Version	Cookbooks (...)
node1	12.13.47	windows	6.3.9600	chef_client(11.3.4), iis(7.2.0)
node2	12.13.47	ubuntu	18.10
node3	14.10.9	mac_os_x	10.14.6	ntp(3.7.0)

Downstream Impact

None

Acceptance Criteria

The report is formated so UX and Product can analyze it. (Human and CSV formats)

Build hab packages for Windows

The previous limitation to build a Hab package for Windows systems was that
there was no Core plan for Golang, well, once this PR gets merged
habitat-sh/core-plans#3085 we should be able to create a habitat/plan.ps1
to build the chef-analyze hab package for Windows.

Verbage cleanup in reports

Describe the Enhancement:

We need to take feedback from our pilot sessions and incorporate it into the reports. Users identified many improvements we could make and this card covers those.

Describe the Need:

TBD

Current Alternative

N/A

Can We Help You Implement This?:

N/A

(bug-pagination) Number of nodes caped to 1000

Description

Currently, the number of nodes is caped to 1000 since we need to fix/handle
pagination right here:

🔢

chef-analyze/pkg/reporting/cookbooks.go

Line 387 in b678332

// TODO add pagination

This means that the maximum number of nodes that we will report is 1000,
even though there could be more nodes using each cookbook.

Acceptance Criteria

When a customer has a cookbook assigned to more than 1000 nodes, the cookbooks report should accurately display that.

In the summary, display the correct total number of nodes.
In the .txt and .csv reports list the name of all nodes the cookbook is assigned to.

It is possible that this generates massive .txt and .csv reports for very large enterprises.

Chef Analyze Version

Lastest on master.

Platform Version

All.

Some unit tests fail if credentials exist on filesystem

The boundary of our unit tests should not extend iinto external filesystem access. Instead, let's mock out the higher-level calls that are being performed; or if our code is directly accessing the file system, consider something like https://github.com/spf13/afero to handle mocking for test.

Failures when ~/.chef/credentials exists:

chef-analyze/$ go test ./pkg/*
--- FAIL: TestNewConfigNotFoundError (0.00s)
    config_test.go:31:
                Error Trace:    config_test.go:31
                Error:          Expected value not to be nil.
                Test:           TestNewConfigNotFoundError
--- FAIL: TestFindConfigFileNotFoundErr (0.00s)
    finder_test.go:32:
                Error Trace:    finder_test.go:32
                Error:          Expected value not to be nil.
                Test:           TestFindConfigFileNotFoundErr
FAIL
FAIL    github.com/chef/chef-analyze/pkg/config 0.005s
--- FAIL: TestFindCredentialsFileNotFound (0.00s)
    credentials_test.go:32:
                Error Trace:    credentials_test.go:32
                Error:          Expected value not to be nil.
                Test:           TestFindCredentialsFileNotFound
--- FAIL: TestNewErrorCredsNotFound (0.00s)
    credentials_test.go:76:
                Error Trace:    credentials_test.go:76
                Error:          "
                                  profile not found in credentials file.

                                  verify the format of the credentials file by following this documentation:
                                    - https://docs.chef.io/knife_setup.html#knife-profiles
                                " does not contain "credentials file not found"
                Test:           TestNewErrorCredsNotFound
    credentials_test.go:77:
                Error Trace:    credentials_test.go:77
                Error:          "
                                  profile not found in credentials file.

                                  verify the format of the credentials file by following this documentation:
                                    - https://docs.chef.io/knife_setup.html#knife-profiles
                                " does not contain "default: $HOME/.chef/credentials"
                Test:           TestNewErrorCredsNotFound
    credentials_test.go:78:
                Error Trace:    credentials_test.go:78
                Error:          "
                                  profile not found in credentials file.

                                  verify the format of the credentials file by following this documentation:
                                    - https://docs.chef.io/knife_setup.html#knife-profiles
                                " does not contain "setup your local credentials config by following this documentation"
                Test:           TestNewErrorCredsNotFound
    credentials_test.go:80:
                Error Trace:    credentials_test.go:80
                Error:          Not equal:
                                expected: credentials.Credentials{Profiles:credentials.Profiles(nil), profile:"", CredsDetail:credentials.CredsDetail{ChefServerUrl:"", ClientName:"", ClientKey:""}}
                                actual  : credentials.Credentials{Profiles:credentials.Profiles{"default":credentials.CredsDetail{ChefServerUrl:"https://api.chef.io/organizations/DELETED", ClientName:"marcparadise", ClientKey:"/home/DELETED"}}, profile:"", CredsDetail:credentials.CredsDetail{ChefServerUrl:"", ClientName:"", ClientKey:""}}

                                Diff:
                                --- Expected
                                +++ Actual
                                @@ -1,3 +1,9 @@
                                 (credentials.Credentials) {
                                - Profiles: (credentials.Profiles) <nil>,
                                + Profiles: (credentials.Profiles) (len=1) {
                                +  (string) (len=7) "default": (credentials.CredsDetail) {
                                +   ChefServerUrl: (string) (len=49) "https://api.chef.io/organizations/jiminywarehouse",
                                +   ClientName: (string) (len=12) "marcparadise",
                                +   ClientKey: (string) (len=36) "/home/marc/env/keys/marcparadise.pem"
                                +  }
                                + },
                                  profile: (string) "",
                Test:           TestNewErrorCredsNotFound
FAIL
FAIL    github.com/chef/chef-analyze/pkg/credentials    0.006s
ok      github.com/chef/chef-analyze/pkg/featflag       (cached)
--- FAIL: TestReportingNewDefaultErrorWithoutCredentials (0.00s)
    reporting_test.go:73:
                Error Trace:    reporting_test.go:73
                Error:          Expected value not to be nil.
                Test:           TestReportingNewDefaultErrorWithoutCredentials
FAIL
FAIL    github.com/chef/chef-analyze/pkg/reporting      0.004s
FAIL

Related issue: #57

Prevent panics and/or raw error text from rendering to the human operator

Given that ...

software can ship with bugs
bugs in golang can sometimes terminate in a panic that gets rendered to the terminal; and third party libraries can give us errors that are not helpful or meaningful to the end user.
most human operators of chef-analyze will find this output unhelpful at best and confusing at worst

... we should provide more user-friendly handling when the unexpected occurs.

I propose that when a panic occurs, we capture the same data that is normally dumped to terminal into an error report. We can then display a message indicating how the user can proceed, for example:

CHEFAN001 

chef-analyze encountered an unexpected error.  If this issue 
keeps occurring, please [do whatever the support process is] 
and include the file below: 

  /home/you/.chef-workstation/logs/chef-analyze-crash-report.log

This also applies to third party errors out of libraries - often the error messages included in there are developer-oriented, and don't provide guidance to the user on how to proceed or fix. Routing all errors to a component that can interpret them and provide more meaningful messaging to the operator would be preferable.

A starting point: https://blog.golang.org/defer-panic-and-recover

Intermittent unit tests

Description

There are one or more intermittent tests: https://buildkite.com/chef/chef-chef-analyze-master-verify/builds/238#d8342483-81de-462a-9896-9a153ccd8fc2 introduced with PR #55

FAIL: TestNodes (0.00s)
    nodes_test.go:91:
        	Error Trace:	nodes_test.go:91
        	            				nodes_test.go:45
        	Error:      	Not equal:
        	            	expected: []reporting.NodeReportItem{reporting.NodeReportItem{Name:"node1", ChefVersion:"12.22", OS:"windows", OSVersion:"10.1", CookbookVersions:[]reporting.CookbookVersion{reporting.CookbookVersion{Name:"mycookbook", Version:"1.0"}}}, reporting.NodeReportItem{Name:"node2", ChefVersion:"13.11", OS:"", OSVersion:"", CookbookVersions:[]reporting.CookbookVersion{reporting.CookbookVersion{Name:"test", Version:"9.9"}, reporting.CookbookVersion{Name:"mycookbook", Version:"1.0"}}}, reporting.NodeReportItem{Name:"node3", ChefVersion:"15.00", OS:"ubuntu", OSVersion:"16.04", CookbookVersions:[]reporting.CookbookVersion(nil)}}
        	            	actual  : []reporting.NodeReportItem{reporting.NodeReportItem{Name:"node1", ChefVersion:"12.22", OS:"windows", OSVersion:"10.1", CookbookVersions:[]reporting.CookbookVersion{reporting.CookbookVersion{Name:"mycookbook", Version:"1.0"}}}, reporting.NodeReportItem{Name:"node2", ChefVersion:"13.11", OS:"", OSVersion:"", CookbookVersions:[]reporting.CookbookVersion{reporting.CookbookVersion{Name:"mycookbook", Version:"1.0"}, reporting.CookbookVersion{Name:"test", Version:"9.9"}}}, reporting.NodeReportItem{Name:"node3", ChefVersion:"15.00", OS:"ubuntu", OSVersion:"16.04", CookbookVersions:[]reporting.CookbookVersion(nil)}}
 
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -16,4 +16,4 @@
        	            	   CookbookVersions: ([]reporting.CookbookVersion) (len=2) {
        	            	-   (reporting.CookbookVersion) test(9.9),
        	            	-   (reporting.CookbookVersion) mycookbook(1.0)
        	            	+   (reporting.CookbookVersion) mycookbook(1.0),
        	            	+   (reporting.CookbookVersion) test(9.9)
        	            	   }
        	Test:       	TestNodes
FAIL
coverage: 73.2% of statements
FAIL	github.com/chef/chef-analyze/pkg/reporting	0.005s
FAIL
🚨 Error: The command exited with status 1

Chef Analyze Version

master

Platform Version

Any + Pipeline

Dependabot can't parse your go.mod

Dependabot couldn't parse the go.mod found at /go.mod.

The error Dependabot encountered was:

go: gopkg.in/[email protected] requires
	gopkg.in/[email protected]: invalid version: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /opt/go/gopath/pkg/mod/cache/vcs/9241c28341fcedca6a799ab7a465dd6924dc5d94044cbfabb75778817250adfc: exit status 128:
	fatal: The remote end hung up unexpectedly

View the update logs.

Detect when there are no nodes in the Chef Organization

Describe the Enhancement:

For the chef-analyze report nodes command, could we add a message that
says that there are no nodes to analyze when the organization is empty?

Currently, the report we show on an empty org is this:

➜  chef-analyze git:(mp/analyze-83) chef-analyze report nodes -p empty

               Node Name                Chef Version   Operating System    Cookbooks
--------------------------------------+--------------+-------------------+------------

Note:  If the report is not formatted correctly, please
       please expand your terminal window to be at least
       120 characters wide.

I rather display something similar like:

➜  chef-analyze git:(mp/analyze-83) chef-analyze report nodes -p empty
No nodes found to analyze.

Note that we ignore the message and the table headers since there is no data to present.

Describe the Need:

Better UX.

Current Alternative

N/A

Can We Help You Implement This?:

Yes! 💯

Colorize cookbook violation counts

Describe the Enhancement:

When running chef analyze report cookbooks -V, you get output like this:

       Cookbook        Version   Violations   Auto-correctable   Nodes Affected
---------------------+---------+------------+------------------+-----------------
  ohai                 4.2.3              0                  0              231
  runit                1.7.8              1                  1                3
  windows              1.44.1            23                 22              827

(this is a very abbreviated list).

Describe the Need:

It can be hard to pick out which cookbooks have violations, and (worse) which ones have violations requiring manual intervention. Anyone upgrading nodes would be interested is seeing impact at a glance.

Current Alternative

Can We Help You Implement This?:

Consider - I've placed a y for yellow and a r for red to mark colorization:

       Cookbook        Version   Violations   Auto-correctable   Nodes Affected
---------------------+---------+------------+------------------+-----------------
  ohai                 4.2.3              0                  0              231
  runit                1.7.8             y1y                y1y               3
  windows              1.44.1           r23r               r22r             827

Add support for proxy setup

Describe the Enhancement:

As a chef-analyze user,
I expect the command to respect my workstation's proxy configuration,
so I can run the command in a locked down environment that requires proxies.

Describe the Need:

Particularly for our enterprise customers we need to add proxy support. In the Ruby ecosystem we developed a gem that knows how to ready proxy settings from chef config and environment variables, merges them, and exports the merged proxy config as an environment variable in the scope of the running command. We did this because our research showed that environment variables are the most widely understood / respected proxy setup for our tools and dependencies we call out to.

A minimum implementation would be for chef-analyze to respect proxy environment variables and system level proxy settings on MacOS/Windows.

It should also have automated testing to ensure that we correctly support proxies, but that greatly increases the complexity of this card.

Current Alternative

We need to look into whether our current Golang libraries already respect reading proxy environment variables. If they do we can decide whether to attempt to parse the Ruby config (EG, ~/.chef/config.rb) to do the proxy settings merging that currently occurs in our Ruby products.

Implement a global flag to disable SSL Cert Verification

Description

As a chef-analyze user,
I would like to have a way to disable SSL Cert Verification on Chef Server API requests,
So that I can connect to development servers that are not self-signed.

Proposal Solution

Add a global flag --no-ssl-check to skip the SSL cert verification.

Design Work For: Move functionality from flags to new subcommands

Motivation

As a `chef analyze` user,
I want to understand what functionality the tool gives me,
so that I can more accurately generate reports applicable to me.

Specification

The --verify-upgrade flag is confusing to users. Instead of pushing this functionality into a flag we want to expose it as a subcommand. The --unused flag is also confusing. Update our current chef analyze report subcommands to include:

chef analyze report nodes
chef analyze report cookbooks
chef analyze report cookbook-violations
...

TODO: Product and engineering need to fill out this list of how the commands should be structured. Before we can work on this card we need to specify the exact commands we want to have and what they will do (EG, the help text associated with each command and subcommand in the tree).

TODO: The UX CLI guidelines have a "executable noun verb" format. We should condense the depth of the tree (IE, chef analyze report cookbook-violations -> chef analyze cookbook-violations). This involves deciding which 'noun' we want the various functionalities to live in.

Downstream Impact

We're going to change the structure of all the chef analyze report commands so... yeah.

Add GH issue templates and GH tags

Just like: https://github.com/chef/chef-workstation/tree/master/.github/ISSUE_TEMPLATE

[chef_employee] Create cookbook oriented infrastructure report

Motivation

As a Chef employee,
I want to know the current state of customer infrastructure,
so I can build better reports about how to upgrade their infrastructure.

Specification

Create a report that looks like this:

Cookbook Name	Version	Violations	Automatically Correctable	Error Output
tacobell_users	1.2.0	ChefCorrectness/UnnecessaryNameProperty	Y	license '', ^^ Cookbook metadata.rb contains an field with an empty string.
		ChefStyle/SimplifyPlatformMajorVersionCheck	N
tacobell_users	1.3.0	ChefCorrectness/UnnecessaryNameProperty	Y
tacobell_hardening	0.10.0	ChefCorrectness/InvalidVersionMetadata	N
		ChefDeprecations/ChefSpecLegacyRunner	N

Downstream Impact

None

Acceptance Criteria

The report is formated so UX and Product can analyze it. (Human and CSV formats)

Implement an Integration test framework.

Describe the problem

We need to have an integration test framework so that contributors can
start writing integration tests for the chef-analyze CLI. Such framework
can be any out there or a very simple one created by Chef.

Possible Solution

Create an integration/ folder that has everything we need to run and write
integration tests.
Use Go as our language to write test, just to not have to learn, yet another
language.

make chef-analyze easily distributable under a configurable name

Describe the problem

As a community member, I would like to be able to freely distribute Chef Workstation under a different, trademark-compliant package. Chef-Analyze is a component of the Workstation, and would therefor need support for configurable name.

Software Version

N/A

Replication Case

N/A

Stacktrace

N/A

Possible Solution

Something similar to what is being done in chef/chef, as described here: chef/chef#8376

It would likely materialize in the form of a dist.go package residing in the Chef-analyze repo (or somewhere else? or standalone? Where's it best to put it?). This package would define string constants that can be imported into packages with client-facing inputs/outputs. Doing it this way allows community distributions like the CinC-project to substitute/patch a single file a build time in order to be trademark compliant.

Feature Request: CMDB Functionality

I would love the ability to see more info, something similar to Ansible CMDB,
visualizing OHAI data.

a bonus would be the ability to export to CSV and HTML

Build hab packages for linux-kernel2

From #10

[pipeline] Issue with file permissions in our docker images

Describe the problem

I have a feeling that our docker images are busted, or at least it behaves very weirdly.

Try running this:

$ docker run -it --rm --init chefes/buildkite:latest /bin/bash
root@448fd6a3f210:/# echo "hola" > file
root@448fd6a3f210:/# chmod 000 file
root@448fd6a3f210:/# cat file
hola
root@448fd6a3f210:/# ls -ld file
---------- 1 root root 5 Nov  5 23:49 file

OMG! Waaaaat!?

The expected behavior (IMHO) is to have an error like permissions denied instead of being able to read the file.

Possible Solution

Respect the basic Unix laws.

TODO

Once this issue is solved, we need to uncomment some tests at 6f896da

Store reports to disk

Describe the Enhancement:

As a Chef analyze user,
I would like to be able to save the reports to disk as well as any error detail,
So that I can visualize the output of long reports as well as export and share them.

Current Alternative

Run the commands and redirect the standard out and standard error to different files.

chef-analyze report nodes > nodes-report.out 2> nodes-report.err

Can We Help You Implement This?:

N/A

Can we download cookbooks?

As a chef-analyze developer,
I need to be able to download cookbooks from a Chef Server,
So I am able to analyze them and generate reports for our customers.

Acceptance Criteria

Create code that shows that it is possible to download a foo cookbook from a Chef Server.

Switch go-chef library to Chef's Fork

As a Chef developer that users the community repo https://github.com/go-chef/chef,
I would like to be able to add and extend the functionality of the community library fast,
So I am not blocked or delayed on any internal deliverables at Chef.

Description

We are going to do a switch to use the Chef fork https://github.com/chef/go-chef of the
community library so that we can have full control over the progress, bugs, and features
coming forwards. Something to mention is that, even with this modification, we will continue
to contribute back to the community repo in a constant cadence.

Contributions: How to? How often?

This is a TBD topic, the cadence and method on how we will contribute back will be defined
on a different task (GH Issue)

Accept -h/--help flags

For consistency with other Chef Infra CLI tools and the a2 CLI style guide[1], accept a --help/-h flag to get help.

Document cached directory for cleanup

Describe the Enhancement:

Since we are starting to store local data from a Chef Infra Server, it would be good
to provide documentation about the local data cached locally.

Describe the Need:

Current Alternative

Can We Help You Implement This?:

Create a playground environment to design remediation steps

Describe the Task

Create a playground environment so that developers can identify and fully
understand customers environments, this environment will help us design
the future of chef-analyze, that is, how do we automatically remediate
cookbooks and upgrade nodes.

Acceptance Criteria

Create an initial playground environment
The environment should have:
- One or more node(s)
- Nodes should be running on an old version of Chef Infra Client (12.5.1)
- The run-list on the node(s) should have more than one cookbook
- Cookbooks should have violations and deprecations to analyze and remediate
- Cookbooks should be stored in a SCM (git)
Create a document that contains the details about the environment and
what things were found while creating such playground (questions/answers/statement)

Test existing cookbook migration to generate customer upgrade suggestions

Describe the Enhancement:

We want to test upgrading a customer cookbook(s) to support Chef Infra Client 15. Through this process, we want to generate suggestions for customers on best practices for performing their own upgrades.

Describe the Need:

Customers have ran the chef-analyze tool and identified baseline cookbooks they want to upgrade, but they are not sure the best way to proceed with that upgrade. We want to provide them some guidance.

The setup we are examing is one where a new Chef Infra Server (latest version) has been spun up, there is a CI pipeline for deploying cookbooks to it, and the plan is to re-bootstrap nodes to the new Chef Server while upgrading the client version.

We also want to look at the process for converting a cookbook over to using Policyfiles. Is this possible for a baseline cookbook?

Acceptance Criteria

Produce a process document that users could follow to prepare a cookbook for upgrading from Chef Client 12 to 15 and applying it to an existing node (but a new Chef Infra Server).

Consistency on report commands

Motivation

As a chef-analyze user that generates multiple reports,
I want to have the same behavior and output,
so that I don't have to guess the functionality of every report and
I can understand them all at once.

Specification

Have both report-commands, nodes and cookbooks to behave in the same
manner, that is, to print out a summary of the report and store a full report to disk.

The suggested output of both commands are:

Nodes

-- REPORT SUMMARY --

    Node Name      Chef Version   Operating System    Cookbooks
-----------------+--------------+-------------------+------------
  bar              -              -                   -
  chef-load-6      13.2.28        mac_os_x v10.14.6   -
  chef-load-40     12.5.54        mac_os_x v10.14.6   -
  kitchen-ubuntu   15.3.14        ubuntu v18.04       -
  chef-load-52     13.1.0         ubuntu v18.04       -
  chef-load-7      15.3.14        platform 14 v       -
  chef-load-9      15.3.14        ubuntu v18.04       -
  chef-load-8      15.3.14        windows v6.3.9600   -
  chef-load-50     15.3.14        platform 14 v       -
  chef-load-4      15.3.14        windows v6.3.9600   -
  ubuntu-node1     15.3.14        ubuntu v18.04       -
  chef-load-1      14.1.2         mac_os_x v10.14.6   -
  chef-load-10     15.3.14        oracle v8.1.0       -

Cookbooks

-- REPORT SUMMARY --

  Cookbook   Version   Nodes Affected
-----------+---------+-----------------
  my_users   3.0.10                 0
  awesome    0.1.0                  0
  awesome    0.9.0                  0
  foo        0.3.0                  0
  foo        0.2.0                  0
  foo        0.1.0                  0

Downstream Impact

None that I know of since we are still designing these commands.

Remove .ruby-version and .rubocop.yml to prevent failures

Description

Right now we have a lot of failures when a cookbook includes an outdated .ruby-version file or a .rubocop.yml file that includes legacy cop names. We need to remove these configs so that we can actually process these cookbooks.

Additionally, as time goes on users are going to start using the .rubocop.yml file to disable the Chef cops, which will mean chef-analyze will incorrectly return a cookbook as having no violations.

Chef Analyze Version

master

Platform Version

All

Move .analyze-cache/ to $HOME and make it configurable

Describe the Enhancement:

As a chef-analyze user,
I would like the cache directory to be consistent and configurable,
So that I don't generate multiple directories on my workstation and also
So that I can configure a specific path to cache analyze data.

Describe the Need:

If a user runs chef-analyze on multiple directories, we will be generating
multiple .analyze-cache/ directories on the local workstation. Instead,
we would like to be consistent and set the default cache directory to be
$HOME/.analyze-cache/

Current Alternative

None. Remember where you have ran the command and move or delete
the cache data.

Similar Issues

There is one issue to generate a cleanup command that might be a good
idea to link

[Cookbooks Report] Order cookbooks by name

Describe the Enhancement:

For the Cookbooks Report, order the displayed cookbook by alphabetical name.

Describe the Need:

For any operator that uses this report, it would be difficult to digest the list
of cookbooks if they are not ordered in a predictable way.

Acceptance Criteria

Cookbooks are displayed in alphabetical order.

Current Alternative

None. (or probably use sort unix command)

Implement code patterns to allow sub-commands

As a chef-analyze developer,
I would like to have a pattern to add sub-commands following the best practices at Chef,
so that I can easily and iteratively add new sub-commands and/or parameters.

Acceptance Criteria

Follow the project layout for sub-commands from the go-guild section at https://github.com/chef/go-guild/blob/master/PROJECT_LAYOUT.md
Have at least one sub-command pattern.

Do not save empty reports

Description

The cookbooks report is writing reports even if they are empty, let's avoid that.

Allow users to customize cache directory location

Motivation

As a `chef analyze` user,
I want to customize my cache directory,
so that I can store data where it most makes sense to me.

Specification

Allow users to specify their own chef analyze cache directory via a --cache-dir path flag

Downstream Impact

N/A

CSV output should contain data in every cell

Description

When I originally implemented the CSV output I wanted to make it look 'nice' like the text output - meaning there would be a row with the cookbook name and then information would cascade down and to the right about that cookbook.

CSV is meant to act like a database - every row that contains information should contain information in every cell. EG, every row will contain the cookbook name instead of how it is now.

Chef Analyze Version

Platform Version

[user/customer] Create cookbook oriented aggregation report

Motivation

As a Chef customer,
I want a report of all cookbook violations,
so that I can fix them in preparation for upgrading my chef infra client version.

Specification

Create a report that looks like this that can be persisted:

Cookbook Name	Version	Violations	Automatically Correctable	Nodes (...)
tacobell_users	0.1.0	59	50
tacobell_users	0.9.10	61	34	node2, node10
tacobell_users	0.10.0	49	31	node1, node3, node99
tacobell_hardening	0.1.0	27	27
tacobell_hardening	0.1.1	12	5	node4, node5, node88
tacobell_hardening	1.1.2	0	0	node6, node7, node8, node80, node90

Create a summary report that looks like this::

tacbell_users (0.1.0)
    59 violations, 50 automatically remediable, 0 nodes affected
tacobell_users (0.9.10)
    61 violations, 34 automatically remediable, 2 nodes affected
...

Downstream Impact

None

Acceptance Criteria

The report is formatted so UX and Product can analyze it. (Human and CSV formats)

Render table reports by calculating data to display

Describe the Enhancement:

This is an idea to better render the table reports by calculating the data that
will be displayed, instead of hardcoding the column size. Once we retrieve the
data, we go over every line and calculate the maximum node name, CCR version
and operating system type. Then we use that max value per column.

Describe the Need:

The data that we display for the nodes table report could be very different from each
user and therefore, hardcoding the header size is not the best way to display these reports.

Current Alternative

The same, maximize the terminal window.

Can We Help You Implement This?:

all yours @marcparadise 😉

Calculate screen size

Describe the Enhancement:

When a user runs a command on a small terminal the reports that are displayed will end up
in a very weird format since we are defaulting the screen size to XXX a static number. We
need to either calculate the screen size and display the report on that size or notify the user
so that the screen is to small to run the report.

The recommendation is to have a screen size of 80 characters.

Current Alternative

The user needs to expand the screen on their own. They sometimes get confused.

Better handle timeouts

Description

chef-analyze doesn't seem to timeout in a reasonable time or retry. I ran a report and I'm assuming that Hosted Chef is being slow to respond, but multiple minutes later I'm just stuck at 99%+. If the server hasn't responded we should fail after a short period of time and retry.

❰tsmith❙~/dev/work/chef-analyze(git✱≠tas50/specific_cops)❱✔≻ ./chef-analyze report cookbooks --detailed
Finding available cookbooks... (190 found)
Analyzing cookbooks...
189 / 190 [---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->_] 99.47% 0 p/s

Chef Analyze Version

Latest

Platform Version

macOS Catalina

[Nodes Report] Order displayed cookbooks list per node

Describe the Enhancement:

For the Nodes report, order the rows by node name in alphabetical descending order. Add information to the report that this is the ordering.

For the cookbooks on each node (each row) list the cookbooks in run list order (the order in which a Chef Client run will execute them). Change the column header to read Cookbooks (run list order)

UPDATE: Getting run_list order is prohibitively expensive and after thinking through this more alpha order is probably more useful to the user anyway. See updated acceptance criteria below.

Describe the Need:

For any operator that uses this report, it would be difficult to digest the list
of cookbooks that a node depends on if they are not ordered in a predictable
way.

Acceptance Criteria

Nodes displayed in alphabetical order
Cookbooks in nodes report on disk are written in run_list order (if possible)
Report heading matches Cookbooks (run list order)

Updated Acceptance Criteria

Nodes displayed in alphabetical order
Cookbooks in nodes report on disk are written in alphabetic order
Report heading matches Cookbooks

Progress bar is incorrect

Description

During our last customer demo the progress bar would continually stop at 93% when the analyze process was complete.

Chef Analyze Version

Chef Workstation 0.13.35 (I don't think we have a --version flag for chef-analyze yet)

Platform Version

I believe it was a linux box that was SSHed into from a Windows box