datreeio/datree

Prevent Kubernetes misconfigurations from reaching production (again 😤 )! From code to cloud, Datree provides an E2E policy enforcement solution to run automatic checks for rule violations. See our docs: https://hub.datree.io

datree's Issues

inject the account token from the pre-commit config to the docker image

Is your feature request related to a problem? Please describe.
when running Datree's pre-commit, there is no ("native") way to pass the account token to the docker image

Describe the solution you'd like
A way to inject the account token from the pre-commit config

Describe alternatives you've considered
pass the account token as env variable to the docker image

Best practices for managing policy as code in production

Report type

• Missing documentation

What can be improved?
Best practices for managing policy as code in production.

Suggested text
Discussed in this issue.

owner label check not working

Describe the bug
When 'owner' label check enabled helm datree test does not fail on chart that does not have this label

To Reproduce
Steps to reproduce the behaviour:

1. Run command '$ helm datree test hawtio'
2. output does not show any error despite my settings in

Expected behavior
i was expecting it to fail since my chart does not have that label

Screenshots
my deployment yaml starts with:

apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ $chartname }}-{{ $branchName }}
labels:
app: {{ $chartname }}
branch: {{ $branchName }}
spec:

[hawtio]$ grep -ir owner *
[hawtio]$

Desktop (please complete the following information):

• OS: Linux

Datree version (run datree verion):

• Version: 0.1.741

Additional context
(Summary)WWWW
WWWWWWWWWWWWW

• Passing YAML validation: 1/1
  WWWWWWWWWWWWW
• Passing Kubernetes (1.18.0) schema validation: 1/1
  WWWWWWWWWWWWW
• Passing policy check: 1/1
  WWWWWWWWWWWWW
  +-----------------------------------+----------------------------------------------------------+
  | Enabled rules in policy “Default” | 29 |
  | Configs tested against policy | 2 |
  | Total rules evaluated | 29 |
  | Total rules failed | 0 |
  | Total rules passed | 29 |
  | See all rules in policy | https://app.datree.io/login?cliId=Aq6F7RCReLipvMRNp9V... |
  +-----------------------------------+----------------------------------------------------------+
  [charts]$

Policy-As-Code (PaC)

What

Users should be able to set Datree’s policies via code (YAML file)

Why

• Change management of policies
• Collaboration between the policy owner and policy consumers
• GitOps - everything as code

How

1. User will Generate a PaC file (yaml) from scratch, or he will export his policies configuration from Datree's dashboard (aka UI)
2. In the UI, the user will switch to PaC mode
3. The user will make changes to his PaC file and will apply them by running $ datree publish [PaC file]
4. Every execution of Datree will use the policies as defined in the PaC file that was published in step 3'

Update Contributing.MD file to raise PR for the main branch

Describe the bug
In the CONTRIBUTING.MD file under the Submitting a Pull Request(PR) section step 11 is written to raise PR against the staging branch. Recently I raised a PR where I was told that staging branch is going to be deprecated so I should merge my PR to the main branch. Documentation must be updated in order to avoid such confusions to anyone.

"Ensure Deployment has more than one replica configured" shouldn't fail when replicas is omitted

I think the policy "Ensure Deployment has more than one replica configured" - DEPLOYMENT_INCORRECT_REPLICAS_VALUE shouldn't fail when replicas is completely omitted. According to https://hub.datree.io/ensure-minimum-two-replicas this should only occur when replicas is set to 1. An example situation where you'd want to omit "replicas" is when using a Horizontal Pod Autoscaler to control the replicas instead.

update katacoda scenario

Is your feature request related to a problem? Please describe.
Update the demo scenario to also include the policy and rules dashboard (UI)

Describe the solution you'd like
Maybe use a split layout: https://katacoda.com/scenario-examples/courses/uilayouts/uilayout-terminal-iframe-split

Additional context
https://github.com/datreeio/katacoda-scenarios

Autocompletion usage instructions in post install message

Is your feature request related to a problem? Please describe.
In order to have autocomplete for the CLI the user need to run the completion command

Describe the solution you'd like
Print a message to the user that he should run the completion command if he wants autocompletion enabled after the installation is finished

Additional context
I suggest adding a short description about the completion command here

"Policy check didn't run for this file" shown for a helm test

Describe the bug
"Policy check didn't run for this file" shown for a helm test.

To Reproduce
Steps to reproduce the behavior:

1. Create a umbrella helm chart
2. Navigate to the sub-chart and use "helm dep up && helm datree test ."
3. Observe that we encounter : [?] Policy check didn't run for this file

Expected behavior
Set Policy should be executed

Screenshots

Desktop (please complete the following information):

• OS: Debian (WSL2 Windows)

Datree version (run datree version):

• Version: [0.14.20]

check_commit_msg workflow should ignore main branch

Describe the bug
The workflow for checking commit messages should not be executed after a pr is approved and pushed to main branch

plugins/eco-systems to support

We are getting a lot of requests for different plugins to support.
This issue is a place for our community to understand which plugins Datree should support and prioritization.

• kubectl ⭐
• kustomize ⭐
• pre-commit framework
• Homebrew
• Chocolatey
• GitHub Action ⭐
• VS code
• tldr
• Junit (output) ⭐
• Tekton ⭐
• CircleCI orb ⭐
• helmfile
• arkade
• Civo
• Azure DevOps (docs) ⭐
• ArgoCD (docs) ⭐
• BuildKite
• Krew ⭐
• Dron ⭐
• SARIF (output)
• JsonSchemaStore (PaC)

Invalid namespace file passes Datree validation

apiVersion: v1
kind: Namespace
metadata:
creationTimestamp: null
name: Test2
spec: {}
status: {}

Trying to validate a namespace YAML file that has an invalid name.
If I apply the above file with the dry-run=server flag it throws an error "The Namespace "Test2" is invalid: metadata.name: Invalid value: "Test2": a lowercase RFC 1123 label must consist of lower case alphanumeric characters or '-', and must start and end with an alphanumeric character (e.g. 'my-name', or '123-abc', regex used for validation is 'a-z0-9?')"

but datree test doesn't detect this as an error.

Config file

Is your feature request related to a problem? Please describe.
Right now Datree requires that we sign up in order to be able to change policies or default configs.

Describe the solution you'd like
It would be great to be able to edit that info having a datree config file that could be comited to repos

Additional context
It would be equivalent to files like .eslintrc.yaml or analysis_options.yaml

Releases don't show complete changelog

Describe the bug
In Releases page: https://github.com/datreeio/datree/releases you don't see the full changelog between the current and previous release

To Reproduce
Go to releases page: https://github.com/datreeio/datree/releases

Expected behavior
in a release, see the full changelog

Screenshots

Desktop (please complete the following information):

• OS: not relevant

Datree version (run datree version):

• Version: not relevant

Replace "stale app" with the "stale Action"

Is your feature request related to a problem? Please describe.
Today, we are using the outdated stale app to close stale issues

Describe the solution you'd like
use the stale Action that has more capabilities

Additional context
we should also configure the stale app to close stale PR

Self hosted datree configuration dashboard.

Is your feature request related to a problem? Please describe.
We need a centralized policy configuration but we cannot go to the external environment to get those polices. It must be inside our infrastructure.

Describe the solution you'd like
A docker image of app.datree

Add developers guide for new contributers

Is your feature request related to a problem? Please describe.
First-time contributers can face difficulties setting up the development environment

Describe the solution you'd like
A small and concise guide to how to set up environment for development

Additional context
A good example can be found here - https://helm.sh/docs/community/developers/

Release workflow not waiting for travis build to end

Describe the bug
Release flow of a new version is defined as a github workflow that sends a request to travis using http request. The workflow is not taking into account the time it takes for travis to finish the deployment step and instead shows releases as finished after a few seconds

To Reproduce
As maintainer, execute a release workflow. Notice that the release finishes a lot faster than the corresponding travis build that is still in progress

Expected behavior
Workflow should wait until travis build is finished

Additional context
We probably need to add a step that waits for the specific travis build to finish and check its status

homebrew support

Discussed in #90

^{Originally posted by virajp July 8, 2021}
It will be great if datree is available via homebrew

Automate contribution workflow

Is your feature request related to a problem? Please describe.
When we receive code contributions from the community, before reviewing the code logic we have to do a manual set of checks that can be automated

Describe the solution you'd like
Add a bot to check for incompatibility with our contribution guidelines before getting a real person to review the code

Additional context
Adding automated labels can also be a nice addition - ltgm, cla-yes/no, fix-commit-message, etc.

Docs: Need to update the docker image name

Describe the bug
From Datree following the step 2
Docker image already pull from step 1
Using the Docker command:

While running the docker command it shows an error Unable to find image 'datree:latest' locally docker: Error response from daemon: pull access denied for datree, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.

To Reproduce
Steps to reproduce the behavior:

1. Run command cat ~/.datree/k8s-demo.yaml | docker run -i datree test -
2. Hit enter
3. See error Unable to find image 'datree:latest' locally docker: Error response from daemon: pull access denied for datree, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.

Expected behavior
It should run the pulled docker image i.e datree/datree
The below command works fine:
cat ~/.datree/k8s-demo.yaml | docker run -i datree/datree test -

Screenshots
Not working:

Working:

Desktop (please complete the following information):

• OS: Linux

Datree version (run datree verion):

• Version: datree/datree:latest

Additional context
Add any other context about the problem here.
If possible, include a YAML file to reproduce the bug.

Structured output improvement

Is your feature request related to a problem? Please describe.
We support JSON and YAML output but the structure of the output is not "developers/machine/pipe-line friendly" enough

Describe the solution you'd like
The output structure should be documented, clear, and consistent.

Additional context
YAML output: datree test manifest.yaml -o yaml
JSON output: datree test manifest.yaml -o json
Related to datreeio/helm-datree#5 (comment)

installation with "curl" doesn't work with mac m1 chip

description
installation with "curl" doesn't work with mac m1 chip

Steps to reproduce the behavior:

1. open a mac machine with an m1 chip
2. run the datree installation command: "curl https://get.datree.io | /bin/bash"
3. run "datree --help"
4. you will get an error: "datree: command not found"

Expected behavior
Expecting the datree cli to be installed

• OS: MacOS, m1 chip
• Version: 0.14.62

The simple output flag should print output without any colors

Describe the bug
Simple output prints color (tags)

To Reproduce
Steps to reproduce the behavior:

1. Run command 'datree test --output simple k8s-demo.yaml'
2. See the summary table

Expected behavior
The simple output flag should print output without any colors or emoji

Screenshots

| [91mTotal rules failed[0m                | [91m0[0m                                                        |
| [32mTotal rules passed[0m                | [32m0[0m                                                        |

Desktop (please complete the following information):

• OS: Windows 10

Datree version (run datree verion):

• Version: 0.7.1

How to installed all policies into our datacenter instead of GitHub on internet

Is your feature request related to a problem? Please describe.

currently, we install date binary on Unix and if we want to change policy then we need to login to datree site and do changes and also we are not allowed to use the internet in that case how we can do.
how we can do all setup on-prem hardware.

Docker image for Datree

Is your feature request related to a problem? Please describe.
Sometimes, users prefer not to install tools on their CI machines.

Describe the solution you'd like
To make it easier to use Datree in the CI, it will be nice to have a ready docker image.

plug in framwork

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. Run command '...'
2. Click on '....'
3. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. Linux]

Datree version (run datree verion):

• Version: [e.g. 0.1.443]

Additional context
Add any other context about the problem here.
If possible, include a YAML file to reproduce the bug.

Empty files pass Kubernetes validation

Describe the bug
Running datree test on empty files shows invalid memory address or nil pointer dereference exception.

To Reproduce
Steps to reproduce the behavior:

1. Run command 'test' on an empty file.

Expected behavior
Empty files should not pass the Kubernetes validation step.

Screenshots

Desktop (please complete the following information):

• OS: macOS

Datree version (run datree verion):

• Version: 0.14.4

Show Detailed Occurrences for rules failures

Is your feature request related to a problem? Please describe.
As a user, I want to be able to have a clearer indication for the mistakes in my yaml manifests.

Describe the solution you'd like
Showing manifest's metadata.name and kind upon failure.

Describe alternatives you've considered
Manually iterating through all my manifest to find where exactly my config failed.

Add semantic versioning to the project release flow

Is your feature request related to a problem? Please describe.
We used to use svu in our project. Because of some technical difficulties we removed this dependency and the versioning is now hard-coded.
We'd like to reintroduce svu to our project (or any other recommended version management solution)

Describe the solution you'd like

1. Clean unused files from the ./scripts folder (deploy.sh for example is not used anymore)

2. Both deploy_release_candidate.sh and release.sh are using the same logic to determine the version. Use define_semver_number.sh script instead to avoid code duplication

3. Remove hard-coded MAJOR_VERSION and MINOR_VERSION variables and use svu logic to determine next release version

4. Versioning should work as follows:
   for x.x.x -> major.minor.patch

• Every feat! should bump major version
• Every feat should bump minor version
• Everything else should bump patch version
• Every commit to main should bump exactly one of the above
  for example:
  two feat commits should bump one minor version
  feat and fix commits should bump one minor version

Describe alternatives you've considered
svu is not mandatory for this solution. This is a suggested package to use but we're not set on it and are open to alternatives

Add linter to the project

Is your feature request related to a problem? Please describe.
As our comminity grows, we need to setup development standards. Adding a linter is a good start

Describe the solution you'd like
A go linter (should be widely used in several big open-source projects) that corresponds with golang development best practices. Preferably not too aggressive

Additional context
Add linter to the build flow in travis as part of the deployment process

k8s schema validation error: could not find schema for Ingress

Is your feature request related to a problem? Please describe.
Datree test isn't working on my Ingress, with the following error: k8s schema validation error: could not find schema for Ingress It has an apiVersion of "networking.k8s.io/v1". Is this not supported? According to https://hub.datree.io/prevent-ingress-forwarding-traffic-to-single-container it looks like ingresses are supported in some form.

Describe the solution you'd like
Support for "networking.k8s.io/v1" ingresses.

macOS Installation failes due to missing sudo prefix

Hi,

I just come across Datree by TechWorld with Nana (https://www.youtube.com/watch?v=hgUfH9Ab258).

When i try to install Datree with the command (curl https://get.datree.io | /bin/bash) it fails, because the commands run without sudo privilege (because the platform is Darwin and not Linux):

datree/install.sh

Line 26 in 06e01f8

else

When I run the whole script with sudo in line 27 and 28 everything works just fine.

I'm running macOS Big Sur 11.4 on a MacBook Air M1.

Wolfgang

Ability to hide "loading" animation

Is your feature request related to a problem? Please describe.
I'm running datree in Azure Pipelines, and my logs are being filled up with lots of loading messages, such as:

| Loading... WWWWWWWWWWWWWW

/ Loading... WWWWWWWWWWWWWW

- Loading... WWWWWWWWWWWWWW

\ Loading... WWWWWWWWWWWWWW

| Loading... WWWWWWWWWWWWWW

/ Loading... WWWWWWWWWWWWWW

- Loading... WWWWWWWWWWWWWW

\ Loading... WWWWWWWWWWWWWW

| Loading... WWWWWWWWWWWWWW

/ Loading... WWWWWWWWWWWWWW

- Loading... WWWWWWWWWWWWWW

\ Loading... WWWWWWWWWWWWWW

| Loading... WWWWWWWWWWWWWW

/ Loading... WWWWWWWWWWWWWW

- Loading... WWWWWWWWWWWWWW

\ Loading... WWWWWWWWWWWWWW

Describe the solution you'd like
It would be good to be able to disable this with an optional argument.

error when running Datree helm plugin on a Windows machine

Describe the bug
error when running Datree helm plugin on a Windows machine

To Reproduce
Steps to reproduce the behavior:

1. Install Datree's helm plugin on a Windows machine
2. Run Datree: helm datree test [chart]
3. See error %1 is not a valid Win32 application.

Expected behavior
The plugin should work fine (like on a Linux machine)

Screenshots

Desktop (please complete the following information):

• OS: Windows 10

Datree version (run datree verion):

• Version: 0.7.1

Policy check fails but it doesn't reflect in the final summary

Describe the bug
Let's say I have a deployment file and it has replicas field set to blank for whatever reason. Now when 'datree test' is run on this file, it shows a failed policy but final summary shows 'Total rules failed' as 0.

To Reproduce
Steps to reproduce the behaviour:

1. Make a sample Kubernetes Deployment in a yaml and set the 'replicas' field to an empty value
2. Run 'datree test '
3. Observe the logs. It says 'Ensure Deployment has more than one replica configured [1 occurrences]' under 'Policy Check' but in the final summary at the bottom, you can see 'Total rules failed' as 0.

Expected behavior
'Total rules failed' should include the count for this failed check.

Screenshots

Desktop (please complete the following information):

• OS: MacOS

Additional context
None

Docker image missing tag versions

Is your feature request related to a problem? Please describe.
Datree's docker image doesn't include old versions, only the latest

Describe the solution you'd like
A dedicated image tag version for each CLI release

Additional context
Related to #225

stale action not working

Describe the bug
The "Close stale issues" workflow throw an error when running

Expected behavior
The workflow should work as expected

Screenshots

Additional context
From reading the docs, we are not using the exempt-issue-labels property correctly

Support installation for architectures other than x86_64

Is your feature request related to a problem? Please describe.
Installation script supports only x86_64 architecture. 386 and arm64 are not supported.

Describe the solution you'd like
Detect the machine's architecture and download the relevant release in the install script

Additional context
the relevant file that should be changed should be install.sh and windows_install.ps1

Non strict mode - ignore files are not k8s files

Is your feature request related to a problem? Please describe.

when a user has several files (e.g. "user_configs.yaml") in a dir that not all of them are K8s files and he runs datree on the entire dir content, he will get errors (not valid yaml/k8s file check will fail) on those files.

Describe the solution you'd like

add a flag that will ignore files that are not K8s files.
we can do that by checking for mandatory properties such as "apiVersion" or "kind".

Windows support

Is your feature request related to a problem? Please describe.
We use Windows laptops and desktops to manage our Kubernetes infrastructure. Not having a windows compatible CLI prohibits distribution to our team.

Describe the solution you'd like
Elevate Windows to a supported platform

Describe alternatives you've considered
We are using home-grown validation scripts.

Additional context
Maybe it is already on the roadmap, given that the public facing documentation makes a reference to the Windows environment.

Installation script continues even if download fails

Describe the bug
The install script continues execution even if there are errors in the download process and the download fails

To Reproduce
Steps to reproduce the behavior:

1. curl https://get.datree.io | /bin/bash
2. interrupt internet(disable wifi during download)
3. See error

Expected behavior
Error message and non-zero exit code

Screenshots

Desktop (please complete the following information):

• OS: Any

Datree version (run datree verion):

• Version: Any

Datree fails when installing with brew

Describe the bug
Any datree command fails

fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0xb01dfacedebac1e pc=0x7fff20458cbe]

runtime stack:
runtime: unexpected return pc for runtime.sigpanic called from 0x7fff20458cbe
stack: frame={sp:0x7ffeefbff528, fp:0x7ffeefbff578} stack=[0x7ffeefb805c8,0x7ffeefbff630)
0x00007ffeefbff428:  0x01007ffeefbff448  0x0000000000000004 
0x00007ffeefbff438:  0x000000000000001f  0x00007fff20458cbe
...

To Reproduce
Steps to reproduce the behavior:

1. Run command brew install datree
2. Run datree
3. Error is displayed

Expected behavior
Expecting datree to work

Desktop (please complete the following information):

• OS: MacOS

Datree version (run datree verion):

• Version: 0.9.0

Additional context
I suspect this is because of go versioning issues during go build flow in homebrew-core. datree is not building well with version 1.17 and it looks like the default build is using 1.17 version

https://github.com/Homebrew/homebrew-core/blob/master/Formula/datree.rb

General datree question (pre-commit hook)

Is there any provided examples of how datree could be implemented as a pre-commit hook?

create a plugin for the pre-commit framework

Is your feature request related to a problem? Please describe.
Sharing Datree's pre-commit hooks across projects can be painful because it requires copying and pasting unwieldy bash scripts.

Describe the solution you'd like
The pre-commit framework was created to solve those hook issues.

Additional context
Creating new hooks.
Related to #125

Unable to view results in an Azure DevOps pipeline

Is your feature request related to a problem? Please describe.
There are limited output options for Datree. When used with Azure DevOps, it would be nice to able to pulish the results of these checks

Describe the solution you'd like
A JUnit output format would be extremely welcome to make it easier to export the results. This can then be viewed in the Azure DevOps pipeline.

Add auto completion to terminal

Is your feature request related to a problem? Please describe.
When using datree cli, there's no support for auto complete for next possible command

Describe the solution you'd like
Add auto-completion support for terminal (zsh, bashrc, etc..)
Can be part of the installation flow for datree

CLI doesn't show Usage message with unrecognized flag

Describe the bug
When running datree test --foobar with an unrecognized flag --foobar, the cli doesn't show Usage message (whereas datree version --foobar does return Usage mesage

To Reproduce
Steps to reproduce the behavior:

1. Run command datree test ~/.datree/k8s-demo.yaml --foobar
2. nothing happens (only exit with nonzero)
3. Run command datree version --foobar
4. I can see Usage message

Expected behavior
should show Usage message
Screenshots

Desktop (please complete the following information):

• OS: macOS

Datree version (run datree verion):

• Version: 0.14.17

Digest policy as code from a targeted github repository

Is your feature request related to a problem? Please describe.
Handling policy as code in datree's current state requires datree to be ran anytime a change to a policy is added or implemented.

Describe the solution you'd like
Instead of running datree as a process, it would be nice if datree could be configured to just read policies.yaml from a repo.

Describe alternatives you've considered
The alternative would be manually defining our own CI pipeline to run datree publish on main whenever changes are pulled in. The thing I don't like about this solution is that a pipeline needs to be defined and then maintained. Reading from source would be a much more elegant and preferred solution.

Can't install DaTree on windows11(x64)

Hi there I'm having problem in installing datree on win11(x64) when I ran the command
curl https://get.datree.io | /bin/bash
Result :

[V] Downloaded Datree
unzip:  cannot find or open datree-latest.zip, datree-latest.zip.zip or datree-latest.zip.ZIP.
cp: cannot stat 'datree-latest/datree': No such file or directory
/bin/bash: line 29: sudo: command not found
rm: cannot remove 'datree-latest.zip': No such file or directory
[V] Finished Installation

 Usage: $ datree test ~/.datree/k8s-demo.yaml
 Using Helm? => https://hub.datree.io/helm-plugin

then I tried datree verison but It says bash: datree: command not found

then i tried $ iwr -useb https://get.datree.io/windows_install.ps1 | iex
Output

bash: iwr: command not found
bash: iex: command not found