argoproj-labs/argocd-bot

Bot to automate Kubernetes deployment via Github PRs

argocd-bot's Issues

Important Features

Some of the important features we need:

• argo sync to sync all the diffs found on the branch, feels a bit dangerous to me
• fix argo rollback
• argo list, to list all apps tracked in repo
• print message about locking
• timeout for sync
• some config on requiring approval on a PR before sync-ing

Project status

Is this project still being actively contributed to? It appears that commits have not been made for a long period of time. Is there an alternate approach that is suggested now over this?

GHE_HOST left empty as the doc said bot should use github.com but it does not do that

If we left GHE_HOST empty we hitting the issue as below

rgocd-bot-597b9df857-slcpw argocd-bot > probot run ./lib/index.js
argocd-bot-597b9df857-slcpw argocd-bot [before-after-hook]: "Hook()" repurposing warning, use "Hook.Collection()". Read more: https://git.io/upgrade-before-after-hook-to-1.4
argocd-bot-597b9df857-slcpw argocd-bot 07:25:21.079Z  INFO probot: Listening on http://localhost:8080
argocd-bot-597b9df857-slcpw argocd-bot 07:25:21.097Z DEBUG github: GitHub request: GET /app/installations - 500 Internal Server Error (installation=undefined)
argocd-bot-597b9df857-slcpw argocd-bot   params: {
argocd-bot-597b9df857-slcpw argocd-bot     "per_page": 100,
argocd-bot-597b9df857-slcpw argocd-bot     "baseUrl": "",
argocd-bot-597b9df857-slcpw argocd-bot     "request": {
argocd-bot-597b9df857-slcpw argocd-bot       "timeout": 0
argocd-bot-597b9df857-slcpw argocd-bot     }
argocd-bot-597b9df857-slcpw argocd-bot   }
argocd-bot-597b9df857-slcpw argocd-bot 07:25:21.098Z ERROR probot: Only absolute URLs are supported
argocd-bot-597b9df857-slcpw argocd-bot   HttpError: Only absolute URLs are supported
argocd-bot-597b9df857-slcpw argocd-bot       at fetch.then.then.catch.error (/home/argocd/argocd-bot/node_modules/@octokit/rest/lib/request/request.js:105:13)

Support GitHub App instead of GitHub PAT

WHY

Because GitHub App has full-grained access control and the owner can be a GitHub org.
PAT is associated with the user that created the PAT thus if the user leaves the company, the PAT will have no longer access to the org.

tests stub on exec are not working

The JS code execs a few helper bash scripts. I'm using child_process.exec. When trying to mock this out using sinon.stub, the stub is not working (no data is being returned). Seems that the JS code is not using the mock.

const child_process = require('child_process')
const execStub = sinon.stub(child_process, 'exec')
execStub.returns({'stdout': 'test'})

Should work with pull requests from different user

Currently this only works for PRs from within the same repository.

This can easily be fixed by using the special github git references for pull requests e.g. refs/pull/42/head

Code re-factoring checklist

Some useful code refactoring:

• put all constants, like messages and such in a separate JS class/file

Add SECURITY.md

The Argo maintainers recently agreed to require all Argoproj Labs project repositories to contain a SECURITY.md file which documents:

• Contact information for reporting security vulnerabilities
• Some minimal information about policies, practices, with possibly links to further documentation with more details

This will help direct vulnerability reporting to the right parties which can fix the issue.

You are free to use the following as examples/templates:

• Argoproj
• Workflows
• CD

Also, please note that in the future we are exploring a requirement that argoproj-labs projects perform a CII self-assessment to better inform its users about which security best practices are being followed.

Can this work without using a GitHub App?

We run an internal GitHub Enterprise instance. We have github build users for which we can easily generate tokens.

It's not as simple for us to use GitHub Apps. Can this be configured entirely using PAT rather than requiring a GitHub App? The README.md only mentions the GitHub App workflow.

Support Bitbucket PRs

We use bitbucket so I'd like for this to support BB PRs.

Clean up new lines from bot comment

kubectl logs argocd-bot-6cbc5bbc84-9g5np
Received diff, processing; command: [ 'argo', 'diff', '-d', 'projects/go-test\r\n\r\n' ]

collapse multiple spaces in comment into one

handle weird comments like this:
example comment:

argo     diff  -d  dir .    

Support GitLab PRs

It would be awesome if the argocd-bot would also support GitLab PRs. But it seems this can not be easily achieved due the usage of GitHub-specific probot library.

I am opening the issue just for tracking how many would like this feature. Similar feature requests:

• #10
• #30

Docker build fails on npm run build

The docker image in Dockerhub was built with an older version of the ArgoCD image, so it pulls the Helm2 binary instead of Helm3. To get around that, I attempted to build my own version of the image to get the latest ArgoCD image and binaries.

However, I get a build failure running the npm commands in a docker build:

Step 18/18 : RUN npm install && npm run build && npm run test
 ---> Running in 4629bb1e0cbd
npm WARN deprecated [email protected]: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.
npm WARN deprecated [email protected]: use String.prototype.padStart()
npm WARN deprecated [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated [email protected]: request-promise-native has been deprecated because it extends the now deprecated request package, see https://github.com/request/request/issues/3142
npm WARN deprecated [email protected]: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated [email protected]: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated [email protected]: this library is no longer supported

> [email protected] install /home/argocd/argocd-bot/node_modules/dtrace-provider
> node-gyp rebuild || node suppress-error.js

make: Entering directory '/home/argocd/argocd-bot/node_modules/dtrace-provider/build'
  TOUCH Release/obj.target/DTraceProviderStub.stamp
make: Leaving directory '/home/argocd/argocd-bot/node_modules/dtrace-provider/build'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@^1.2.7 (node_modules/jest-haste-map/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN [email protected] No license field.

added 672 packages from 869 contributors and audited 673 packages in 23.718s
found 1 low severity vulnerability
  run `npm audit fix` to fix them, or `npm audit` for details

> [email protected] build /home/argocd/argocd-bot
> tslint src/**/*.ts{,x} && tsc && cp -r ./src/sh ./lib/

node_modules/probot/lib/application.d.ts(34,20): error TS2314: Generic type 'WebhookEvent<T>' requires 1 type argument(s).
node_modules/probot/lib/context.d.ts(1,24): error TS2724: Module '"../../@octokit/webhooks"' has no exported member 'WebhookPayloadWithRepository'. Did you mean 'WebhookPayloadRepositoryImport'?
node_modules/probot/lib/context.d.ts(20,41): error TS2314: Generic type 'WebhookEvent<T>' requires 1 type argument(s).
node_modules/probot/lib/context.d.ts(29,24): error TS2314: Generic type 'WebhookEvent<T>' requires 1 type argument(s).
node_modules/probot/lib/github/index.d.ts(1,23): error TS2688: Cannot find type definition file for 'bunyan'.
node_modules/probot/lib/github/index.d.ts(2,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@octokit/rest/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/index.d.ts(1,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@octokit/webhooks/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/index.d.ts(16,20): error TS2314: Generic type 'WebhookEvent<T>' requires 1 type argument(s).
node_modules/probot/lib/wrap-logger.d.ts(1,23): error TS2688: Cannot find type definition file for 'bunyan'.
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! [email protected] build: `tslint src/**/*.ts{,x} && tsc && cp -r ./src/sh ./lib/`
npm ERR! Exit status 2
npm ERR! 
npm ERR! Failed at the [email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/argocd/.npm/_logs/2020-08-26T20_31_08_119Z-debug.log
The command '/bin/sh -c npm install && npm run build && npm run test' returned a non-zero code: 2

In case this was an out-of-date package issue, I tried updating npm packages via the instructions here, and while that did help with deprecation warnings, it still failed for the same reason, errors related to probot.

docker build output

Step 18/18 : RUN npm install && npm run build && npm run test
 ---> Running in ae90af191043

> [email protected] install /home/argocd/argocd-bot/node_modules/dtrace-provider
> node-gyp rebuild || node suppress-error.js

make: Entering directory '/home/argocd/argocd-bot/node_modules/dtrace-provider/build'
  TOUCH Release/obj.target/DTraceProviderStub.stamp
make: Leaving directory '/home/argocd/argocd-bot/node_modules/dtrace-provider/build'
npm WARN [email protected] No license field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 1018 packages from 912 contributors and audited 1019 packages in 12.849s
found 0 vulnerabilities


> [email protected] build /home/argocd/argocd-bot
> tslint src/**/*.ts{,x} && tsc && cp -r ./src/sh ./lib/

node_modules/probot/lib/application.d.ts(3,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@octokit/webhooks/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/application.d.ts(4,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@types/express/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/context.d.ts(1,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@octokit/webhooks/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/context.d.ts(2,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/deepmerge/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/github/logging.d.ts(1,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@types/bunyan/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/index.d.ts(4,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@octokit/webhooks/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/index.d.ts(5,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@types/bunyan/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/index.d.ts(6,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@types/express/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/index.d.ts(7,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@types/ioredis/index"' can only be default-imported using the 'esModuleInterop' flag
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! [email protected] build: `tslint src/**/*.ts{,x} && tsc && cp -r ./src/sh ./lib/`
npm ERR! Exit status 2
npm ERR! 
npm ERR! Failed at the [email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/argocd/.npm/_logs/2020-08-26T20_34_27_005Z-debug.log
The command '/bin/sh -c npm install && npm run build && npm run test' returned a non-zero code: 2

improved support for diff command

Should support
diff --auto-sync
and diff --all

ArgoCD bot Communication with Github while posting diffs on PR

I am trying to configure ArgoCD bot to comment the diffs on PR. I have come so far to see that Argo bot is deployed in my system and pod/deployment being made. I enabled the debug flag also to see the logs in pod. The github app is installed in my repo and I have populated the .env file as well, so the pod is made without any issues.
I have loaded the secrets using the script that is provided in the repo and as instructed in the README. The .pem key is placed at my root folder.

This is my .env file (has dummy values, not real ones)

PORT=80
LOG_LEVEL=debug
KUBECTL_EXTERNAL_DIFF=
APP_ID=700908
[email protected]:user/app.git
GITHUB_TOKEN=dc78adfh28374hjshf8b061dcd1f69bc1389f4f6
WEBHOOK_SECRET=webhook-secret
PRIVATE_KEY_PATH=/home/private/key/argocd-bot/argocd-diff.private-key.pem
ARGOCD_SERVER=https://ARGO-IP/
ARGOCD_AUTH_TOKEN=adfadfadf34342sfsfsfs23aI

The following are the logs from the pod:

  params: {
    "per_page": 100,
    "baseUrl": "https://api.github.com",
    "request": {
      "timeout": 0
    }
  }
06:01:09.276Z DEBUG github: GitHub request: POST /app/installations/:installation_id/access_tokens - 201 Created (installation=198181122)
  params: {
    "installation_id": 198181122,
    "baseUrl": "https://api.github.com",
    "request": {
      "timeout": 0
    }
  }
06:01:10.314Z DEBUG github: GitHub request: GET /installation/repositories - 200 OK (installation=198181122)
  params: {
    "per_page": 100,
    "baseUrl": "https://api.github.com",

Also ARGOCD_AUTH_TOKEN it is recommended to generate an automation token using the /api/v1/projects/{project}/roles/{role}/token API but this is not found on my Argo Deployment. I am unable to find this endpoint.

When the PR is made in the repo, the events are received in the logs of the pod but the bot is unable to post the diff on the PR.
I also changed the internal ingress to the external one to see if there was any communication issue.
The slack channel doesnot have any pointers in this regard as well. I have tried it on multiple ArgoCD installations local and on cloud premises but circling back at the same issue again and again

Switch to using .ts instead of .js

• rename all files from .ts to .js
• switch to using ts-node

argocd-bot to work using Azure Repos

The bot needs to be able to support Git repos in Azure DevOps

One bot per repo?

We are running Github Enterprise and we have all of our k8s repos within a single Github organization. From the looks of the config file argo-bot is configured one bot per repo? Is there currently support to have argo-bot be available to support all repos within an org?

Comments on a closed PR should not hold lock

When a PR is closed/merged and someone posts a comment the bot should just ignore it.

Current behavior is if the PR is closed, the bot responds to the command and creates a lock

Useful Features

List of useful features to have

• argo preview to create a preview environment from PR and tear-down environment once PR closes, see argoproj/argo-cd#1157
• post github hook status
• when running argo sync app have an option to wait for the operation to complete
• don't attempt to implement rollback. just let users rollback to whatever version they want, using hstory command, need to think more about how rollbacks should work
• get clone URL from hook, instead of relying on config
• download argocd cli at runtime
• support downloading custom argocd version from a specific URL
• look into if we can integrate with github deployment API

view_app_info.sh failed due to transport is closing

We tried to get the bot up and running. Even our webhook and subscribe events is correct but the bot didn't do anything. The argocd-bot pod has no logs. And if we mimic script ran it by hand it error out as below:

argocd@argocd-bot-6776676ffb-5n24f:~/argocd-bot$ argocd app list --plaintext
FATA[0000] rpc error: code = Unavailable desc = transport is closing
argocd@argocd-bot-6776676ffb-5n24f:~/argocd-bot$ argocd app get argo-events --plaintext
FATA[0000] rpc error: code = Unavailable desc = transport is closing