kiwicom/crane

⬆ A GitLab CI ready image to upgrade services in Rancher

crane's Issues

Integrate with PagerDuty

Upon release, set someone to be on-call for 2 hours on a given schedule.

Default to gitlab project name as stack

Currently, setting --stack is required. We could just use the project name variable for this by default.

---

🍫This issue has a bounty of €10! Check open.kiwi.com/bounties for details on how to claim the bounty by solving the issue.

Consider supporting upgrading from and to arbitrary tags

This feature would require a major rewrite as crane assumes the tags to be git refs pointing to the old and new versions in the repo; this is done throughout the code.

There are no current plans for removing this assumption, as doing so would make the code much more complicated. But if anyone's aware of a pressing reason to allow arbitrary tags: consider this issue a place to present use cases that would make it worth it to complicate the code to add support for this.

Make sure the init of integrations cannot break deployments

Either wrapping them in try...except, or fixing all the inits to be totally safe.

Traceback (most recent call last):
  File "/usr/local/bin/crane", line 11, in <module>
    load_entry_point('crane==3.0.0', 'console_scripts', 'crane')()
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/crane-3.0.0-py3.6.egg/crane/cli.py", line 52, in main
  File "/usr/local/lib/python3.6/site-packages/crane-3.0.0-py3.6.egg/crane/hooks/__init__.py", line 10, in dispatch
  File "/usr/local/lib/python3.6/site-packages/crane-3.0.0-py3.6.egg/crane/hooks/__init__.py", line 10, in <genexpr>
  File "/usr/local/lib/python3.6/site-packages/crane-3.0.0-py3.6.egg/crane/hooks/slack.py", line 74, in __init__
KeyError: 'members'

Bring back --new-image

Greetings!

Changed
⚠️ Breaking change: Instead of the whole URL passed via --new-image, Crane now only needs the new commit hash you want to deploy (via --new-commit). The value of this option defaults to the commit SHA the GitLab CI pipeline is running for, so in most cases you can just remove the --new-image option entirely.

Unless I am missing something your new method assumes you never want to deploy via Git Tags and only via Commit Refs.

For example, I want to deploy a new version of my app: dtr.my-registry.com/app/app:1.0.11

How is this possible with the --new-commit flag?

Modify environment variables

Hi, it would be really handy if not only image tags could be updated but also environment variables.

CI_ENVIRONMENT_NAME env variable not detected

It seems that the Datadog hook it's triggered even if it's not needed. The CI_ENVIRONMENT_NAME is a GitLab CI variable and it should be detected independently of the datadog hook trigger, but the error just appears there.

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/crane-3.2.0-py3.7.egg/crane/hooks/base.py", line 25, in dispatch
    handler()
  File "/usr/local/lib/python3.7/site-packages/crane-3.2.0-py3.7.egg/crane/hooks/datadog.py", line 32, in create_event
    "environment:{0}".format(environ["CI_ENVIRONMENT_NAME"]),
  File "/usr/local/lib/python3.7/os.py", line 678, in __getitem__
    raise KeyError(key) from None

--no-supervision option

There are some cases when upgrading takes time (for example, some worker-containers have to complete their tasks before stopping gracefully), such that waiting for the upgrade to complete is time-consuming (and unnecessary, sometimes it might even interfere with the runner timeout). There should be a "--no-supervision" option where crane exits without waiting for the upgrade to complete.

Exit with status code 1 on gracefully handled errors

Such as the

Please see if the service is upgradeable

one.

KeyError: 'CI_PROJECT_DIR'

crane fails with an uknown error https://circleci.com/gh/brainhubeu/issue-label-bot/86

#!/bin/sh -eo pipefail
crane --stack issue-label-bot --service issue-label-bot-main --url $RANCHER_URL --access-key $RANCHER_ACCESS_KEY --secret-key $RANCHER_SECRET_KEY --env 1a5
Traceback (most recent call last):
  File "/usr/local/bin/crane", line 11, in <module>
    load_entry_point('crane==3.2.1', 'console_scripts', 'crane')()
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/crane-3.2.1-py3.7.egg/crane/cli.py", line 56, in main
  File "/usr/local/lib/python3.7/site-packages/crane-3.2.1-py3.7.egg/crane/models.py", line 61, in load_from_settings
  File "/usr/local/lib/python3.7/site-packages/crane-3.2.1-py3.7.egg/crane/models.py", line 112, in check_preconditions
  File "/usr/local/lib/python3.7/os.py", line 678, in __getitem__
    raise KeyError(key) from None
KeyError: 'CI_PROJECT_DIR'

Exited with code exit status 1

Maybe I've configured something in an invalid way but crane should tell me exactly what's wrong.

support multiple slack channels

Since we are pushing all company releases to #releases slack channel, it started to be a bit spammy. That is pretty useful on outages, to find out what was released throughout company within specific time range.
But then developers don't follow releases. I think developers should be aware of all releases related to their projects, that is why I'd like to request option to be able to set CRANE_SLACK_CHANNEL as list of channels. So we can use one for general releases and one separate channel for just our releases.

---

🍫This issue has a bounty of €10! Check open.kiwi.com/bounties for details on how to claim the bounty by solving the issue.

Don't restart service if sidekick is upgraded

Hi again!

Here's my use case:
We have a prometheus container deployed with a sidekick (prometheus-conf) volume for the configuration. When we deploy a new sidekick for the prometheus container we don't need it to restart the prometheus container. Instead we would have another job to reload the prometheus configuration via a curl request as described here.

I hope this makes sense. Thanks.

Create services

Greetings,

The project looks interesting but I have one question, will this also create a new service in Rancher or just update an already existing one?

And if it only updates, is there any intention to add this features?

Thanks for the feedback!

Support for Rancher 2

I am definitely late to docker and started learning by using the newly released rancher 2.
Problem is that there is no stacks in rancher 2 anymore and replaced with projects, etc.

Can this be used with Rancher 2?

---

🍫This issue has a bounty of €50! Check open.kiwi.com/bounties for details on how to claim the bounty by solving the issue.

slack: Edit Slack message with percentage of upgraded containers

Show something like '67% (4/6)' Instead of just the spinner emoji.

---

🍫This issue has a bounty of €10! Check open.kiwi.com/bounties for details on how to claim the bounty by solving the issue.

Fail with nice errors if stack or service is not found

Now we just crash with an IndexError.

---

🍫This issue has a bounty of €10! Check open.kiwi.com/bounties for details on how to claim the bounty by solving the issue.

Adopt Black code formatting

https://github.com/ambv/black/

`--clone-from` option to create new stacks automatically

gitdb.exc.BadName: Ref 'master-93b091e2' did not resolve to an object

Hi again,

I have tried to use the 3.1.0 version of the image like I was doing in previous 2.x versions but I am getting an error:

Checking out 93b091e2 as master...
Skipping Git submodules setup
$ crane --env 1a4481 --stack mystack --service myservice --new-image "$ARTIFACTORY_REG:$CI_BUILD_REF_SLUG-${CI_COMMIT_SHA:0:8}"
Traceback (most recent call last):
  File "/usr/local/bin/crane", line 11, in <module>
    load_entry_point('crane==3.1.0', 'console_scripts', 'crane')()
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/crane-3.1.0-py3.6.egg/crane/cli.py", line 49, in main
  File "/usr/local/lib/python3.6/site-packages/crane-3.1.0-py3.6.egg/crane/models.py", line 59, in load_from_settings
  File "/usr/local/lib/python3.6/site-packages/crane-3.1.0-py3.6.egg/crane/models.py", line 104, in check_preconditions
  File "/usr/local/lib/python3.6/site-packages/crane-3.1.0-py3.6.egg/crane/models.py", line 81, in new_commit
  File "/usr/local/lib/python3.6/site-packages/git/repo/base.py", line 460, in commit
    return self.rev_parse(text_type(rev) + "^0")
  File "/usr/local/lib/python3.6/site-packages/git/repo/fun.py", line 213, in rev_parse
    obj = name_to_object(repo, rev[:start])
  File "/usr/local/lib/python3.6/site-packages/git/repo/fun.py", line 147, in name_to_object
    raise BadName(name)
gitdb.exc.BadName: Ref 'master-93b091e2' did not resolve to an object

This looks like a issue using $CI_BUILD_REF_SLUG-${CI_COMMIT_SHA:0:8}. We have a requirement to include the branch name and git commit hash in the tag.

Console output contains invalid Rancher URL address

Crane returns these lines in the output:

Alrighty, let's deploy! ᕕ( ᐛ )ᕗ
(But please supervise me at https://url.com//env/123/apps/stacks/456)
Upgrading graphql…

But the URL address doesn't work because of the com//env/ part (Rancher is not happy and you'll get only an empty page).

Abstract crane components

Right now all the workflow depends on the GitLab -> Rancher deployment itself. If we would abstract the different components and we make them available directly from the CI we could use the crane integrations without having to do the deployment with crane itself, like in the case of Kubernetes.

GitLab integrates nicely with Kubernetes and can deploy directly to clusters, so having the different integrations easily callable from the image could allow us keep the same notifications to slack and integrations with datadog while using the GitLab integration for deployments.

We could also integrate the autodevops logic into crane, but that is something else that needs to be maintained and kept in sync. Moreover, having those components abstracted and easily callable can help also with developing different deployment workflows.