argoproj-labs / argocd-bot Goto Github PK
View Code? Open in Web Editor NEWBot to automate Kubernetes deployment via Github PRs
License: Apache License 2.0
Bot to automate Kubernetes deployment via Github PRs
License: Apache License 2.0
Some of the important features we need:
argo sync
to sync all the diffs found on the branch, feels a bit dangerous to meargo rollback
argo list
, to list all apps tracked in repoIs this project still being actively contributed to? It appears that commits have not been made for a long period of time. Is there an alternate approach that is suggested now over this?
If we left GHE_HOST empty we hitting the issue as below
rgocd-bot-597b9df857-slcpw argocd-bot > probot run ./lib/index.js
argocd-bot-597b9df857-slcpw argocd-bot [before-after-hook]: "Hook()" repurposing warning, use "Hook.Collection()". Read more: https://git.io/upgrade-before-after-hook-to-1.4
argocd-bot-597b9df857-slcpw argocd-bot 07:25:21.079Z INFO probot: Listening on http://localhost:8080
argocd-bot-597b9df857-slcpw argocd-bot 07:25:21.097Z DEBUG github: GitHub request: GET /app/installations - 500 Internal Server Error (installation=undefined)
argocd-bot-597b9df857-slcpw argocd-bot params: {
argocd-bot-597b9df857-slcpw argocd-bot "per_page": 100,
argocd-bot-597b9df857-slcpw argocd-bot "baseUrl": "",
argocd-bot-597b9df857-slcpw argocd-bot "request": {
argocd-bot-597b9df857-slcpw argocd-bot "timeout": 0
argocd-bot-597b9df857-slcpw argocd-bot }
argocd-bot-597b9df857-slcpw argocd-bot }
argocd-bot-597b9df857-slcpw argocd-bot 07:25:21.098Z ERROR probot: Only absolute URLs are supported
argocd-bot-597b9df857-slcpw argocd-bot HttpError: Only absolute URLs are supported
argocd-bot-597b9df857-slcpw argocd-bot at fetch.then.then.catch.error (/home/argocd/argocd-bot/node_modules/@octokit/rest/lib/request/request.js:105:13)
Because GitHub App has full-grained access control and the owner can be a GitHub org.
PAT is associated with the user that created the PAT thus if the user leaves the company, the PAT will have no longer access to the org.
The JS code execs a few helper bash scripts. I'm using child_process.exec
. When trying to mock this out using sinon.stub
, the stub is not working (no data is being returned). Seems that the JS code is not using the mock.
const child_process = require('child_process')
const execStub = sinon.stub(child_process, 'exec')
execStub.returns({'stdout': 'test'})
Currently this only works for PRs from within the same repository.
This can easily be fixed by using the special github git references for pull requests e.g. refs/pull/42/head
Some useful code refactoring:
The Argo maintainers recently agreed to require all Argoproj Labs project repositories to contain a SECURITY.md
file which documents:
This will help direct vulnerability reporting to the right parties which can fix the issue.
You are free to use the following as examples/templates:
Also, please note that in the future we are exploring a requirement that argoproj-labs projects perform a CII self-assessment to better inform its users about which security best practices are being followed.
We run an internal GitHub Enterprise instance. We have github build users for which we can easily generate tokens.
It's not as simple for us to use GitHub Apps. Can this be configured entirely using PAT rather than requiring a GitHub App? The README.md only mentions the GitHub App workflow.
We use bitbucket so I'd like for this to support BB PRs.
kubectl logs argocd-bot-6cbc5bbc84-9g5np
Received diff, processing; command: [ 'argo', 'diff', '-d', 'projects/go-test\r\n\r\n' ]
handle weird comments like this:
example comment:
argo diff -d dir .
The docker image in Dockerhub was built with an older version of the ArgoCD image, so it pulls the Helm2 binary instead of Helm3. To get around that, I attempted to build my own version of the image to get the latest ArgoCD image and binaries.
However, I get a build failure running the npm commands in a docker build:
Step 18/18 : RUN npm install && npm run build && npm run test
---> Running in 4629bb1e0cbd
npm WARN deprecated [email protected]: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.
npm WARN deprecated [email protected]: use String.prototype.padStart()
npm WARN deprecated [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated [email protected]: request-promise-native has been deprecated because it extends the now deprecated request package, see https://github.com/request/request/issues/3142
npm WARN deprecated [email protected]: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated [email protected]: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated [email protected]: this library is no longer supported
> [email protected] install /home/argocd/argocd-bot/node_modules/dtrace-provider
> node-gyp rebuild || node suppress-error.js
make: Entering directory '/home/argocd/argocd-bot/node_modules/dtrace-provider/build'
TOUCH Release/obj.target/DTraceProviderStub.stamp
make: Leaving directory '/home/argocd/argocd-bot/node_modules/dtrace-provider/build'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@^1.2.7 (node_modules/jest-haste-map/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN [email protected] No license field.
added 672 packages from 869 contributors and audited 673 packages in 23.718s
found 1 low severity vulnerability
run `npm audit fix` to fix them, or `npm audit` for details
> [email protected] build /home/argocd/argocd-bot
> tslint src/**/*.ts{,x} && tsc && cp -r ./src/sh ./lib/
node_modules/probot/lib/application.d.ts(34,20): error TS2314: Generic type 'WebhookEvent<T>' requires 1 type argument(s).
node_modules/probot/lib/context.d.ts(1,24): error TS2724: Module '"../../@octokit/webhooks"' has no exported member 'WebhookPayloadWithRepository'. Did you mean 'WebhookPayloadRepositoryImport'?
node_modules/probot/lib/context.d.ts(20,41): error TS2314: Generic type 'WebhookEvent<T>' requires 1 type argument(s).
node_modules/probot/lib/context.d.ts(29,24): error TS2314: Generic type 'WebhookEvent<T>' requires 1 type argument(s).
node_modules/probot/lib/github/index.d.ts(1,23): error TS2688: Cannot find type definition file for 'bunyan'.
node_modules/probot/lib/github/index.d.ts(2,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@octokit/rest/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/index.d.ts(1,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@octokit/webhooks/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/index.d.ts(16,20): error TS2314: Generic type 'WebhookEvent<T>' requires 1 type argument(s).
node_modules/probot/lib/wrap-logger.d.ts(1,23): error TS2688: Cannot find type definition file for 'bunyan'.
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! [email protected] build: `tslint src/**/*.ts{,x} && tsc && cp -r ./src/sh ./lib/`
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the [email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /home/argocd/.npm/_logs/2020-08-26T20_31_08_119Z-debug.log
The command '/bin/sh -c npm install && npm run build && npm run test' returned a non-zero code: 2
In case this was an out-of-date package issue, I tried updating npm packages via the instructions here, and while that did help with deprecation warnings, it still failed for the same reason, errors related to probot.
Step 18/18 : RUN npm install && npm run build && npm run test
---> Running in ae90af191043
> [email protected] install /home/argocd/argocd-bot/node_modules/dtrace-provider
> node-gyp rebuild || node suppress-error.js
make: Entering directory '/home/argocd/argocd-bot/node_modules/dtrace-provider/build'
TOUCH Release/obj.target/DTraceProviderStub.stamp
make: Leaving directory '/home/argocd/argocd-bot/node_modules/dtrace-provider/build'
npm WARN [email protected] No license field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
added 1018 packages from 912 contributors and audited 1019 packages in 12.849s
found 0 vulnerabilities
> [email protected] build /home/argocd/argocd-bot
> tslint src/**/*.ts{,x} && tsc && cp -r ./src/sh ./lib/
node_modules/probot/lib/application.d.ts(3,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@octokit/webhooks/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/application.d.ts(4,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@types/express/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/context.d.ts(1,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@octokit/webhooks/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/context.d.ts(2,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/deepmerge/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/github/logging.d.ts(1,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@types/bunyan/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/index.d.ts(4,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@octokit/webhooks/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/index.d.ts(5,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@types/bunyan/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/index.d.ts(6,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@types/express/index"' can only be default-imported using the 'esModuleInterop' flag
node_modules/probot/lib/index.d.ts(7,8): error TS1259: Module '"/home/argocd/argocd-bot/node_modules/@types/ioredis/index"' can only be default-imported using the 'esModuleInterop' flag
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! [email protected] build: `tslint src/**/*.ts{,x} && tsc && cp -r ./src/sh ./lib/`
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the [email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /home/argocd/.npm/_logs/2020-08-26T20_34_27_005Z-debug.log
The command '/bin/sh -c npm install && npm run build && npm run test' returned a non-zero code: 2
Should support
diff --auto-sync
and diff --all
I am trying to configure ArgoCD bot to comment the diffs on PR. I have come so far to see that Argo bot is deployed in my system and pod/deployment being made. I enabled the debug flag also to see the logs in pod. The github app is installed in my repo and I have populated the .env
file as well, so the pod is made without any issues.
I have loaded the secrets using the script that is provided in the repo and as instructed in the README. The .pem key is placed at my root folder.
This is my .env file (has dummy values, not real ones)
PORT=80
LOG_LEVEL=debug
KUBECTL_EXTERNAL_DIFF=
APP_ID=700908
[email protected]:user/app.git
GITHUB_TOKEN=dc78adfh28374hjshf8b061dcd1f69bc1389f4f6
WEBHOOK_SECRET=webhook-secret
PRIVATE_KEY_PATH=/home/private/key/argocd-bot/argocd-diff.private-key.pem
ARGOCD_SERVER=https://ARGO-IP/
ARGOCD_AUTH_TOKEN=adfadfadf34342sfsfsfs23aI
The following are the logs from the pod:
params: {
"per_page": 100,
"baseUrl": "https://api.github.com",
"request": {
"timeout": 0
}
}
06:01:09.276Z DEBUG github: GitHub request: POST /app/installations/:installation_id/access_tokens - 201 Created (installation=198181122)
params: {
"installation_id": 198181122,
"baseUrl": "https://api.github.com",
"request": {
"timeout": 0
}
}
06:01:10.314Z DEBUG github: GitHub request: GET /installation/repositories - 200 OK (installation=198181122)
params: {
"per_page": 100,
"baseUrl": "https://api.github.com",
Also ARGOCD_AUTH_TOKEN it is recommended to generate an automation token using the /api/v1/projects/{project}/roles/{role}/token API but this is not found on my Argo Deployment. I am unable to find this endpoint.
When the PR is made in the repo, the events are received in the logs of the pod but the bot is unable to post the diff on the PR.
I also changed the internal ingress to the external one to see if there was any communication issue.
The slack channel doesnot have any pointers in this regard as well. I have tried it on multiple ArgoCD installations local and on cloud premises but circling back at the same issue again and again
.ts
to .js
ts-node
The bot needs to be able to support Git repos in Azure DevOps
We are running Github Enterprise and we have all of our k8s repos within a single Github organization. From the looks of the config file argo-bot is configured one bot per repo? Is there currently support to have argo-bot be available to support all repos within an org?
When a PR is closed/merged and someone posts a comment the bot should just ignore it.
Current behavior is if the PR is closed, the bot responds to the command and creates a lock
List of useful features to have
argo preview
to create a preview environment from PR and tear-down environment once PR closes, see argoproj/argo-cd#1157argo sync app
have an option to wait for the operation to completehstory
command, need to think more about how rollbacks should workargocd
cli at runtimeargocd
version from a specific URLWe tried to get the bot up and running. Even our webhook and subscribe events is correct but the bot didn't do anything. The argocd-bot pod has no logs. And if we mimic script ran it by hand it error out as below:
argocd@argocd-bot-6776676ffb-5n24f:~/argocd-bot$ argocd app list --plaintext
FATA[0000] rpc error: code = Unavailable desc = transport is closing
argocd@argocd-bot-6776676ffb-5n24f:~/argocd-bot$ argocd app get argo-events --plaintext
FATA[0000] rpc error: code = Unavailable desc = transport is closing
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.