spinnaker/keel

Spinnaker's declarative service

Welcome to the Spinnaker Project

Spinnaker is an open-source continuous delivery platform for releasing software changes with high velocity and confidence. Spinnaker provides a single platform that allows you to safely deploy and manage your application in multiple cloud environments (AWS, GCP, Kubernetes and more!). With automated canary analysis and multiple ways to initiate deployments, you can safely promote your code from a git commit into a dev environment all the way through multiple production environments with confidence in the results. Spinnaker is the most mature and widely productionalized continuous delivery platform built upon the work of Netflix, Google, Microsoft, Amazon and numerous other companies to allow fast, safe deploys. Used by numerous organizations including Target, Salesforce, Airbnb, Cerner, Adobe, JPMorgan Chase and more!

Project status and updates

Note that this repo is only used for issue tracking across the various services at this time. Releases are not managed here nor are updates regularly made to this repository. Please see the following for more information about spinnaker and it’s state:

• Current available versions

• The spinnaker slack for help and discussions around spinnaker

• Issues page for bug reports, enhancements and similar

• Governance for the project as a whole There is a lot of activity and we’d welcome participation!

More about spinnaker

Manage your SDLC in Spinnaker using the GUI (graphical user interface), or config-as-code. View, manage, and construct application workflows involving one or all of these resources:

• Virtual machine (VM) deployments to a public cloud provider, "baked" as immutable infrastructure

• Container deployments to a cloud

• Container deployments to Kubernetes

• Load balancers

• Security groups

• Server groups

• Clusters

• Firewalls

• Functions

Facilitate GitOps, and grant developers control of provisioning infrastructure for apps. Use Spinnaker’s Clouddriver to deploy to all of the major public cloud providers and Kubernetes. You may even orchestrate configuration and firmware changes as part of Spinnaker delivery pipelines.

Why Do I Need Spinnaker?

Use Spinnaker to create a “paved road” for application delivery, with guardrails that ensure only valid infrastructure and configuration reach production.

Free development teams from burdensome ops provisioning while automating reinforcement of business and regulatory requirements. Delivery automation strategies such as canary deployments provide the safety necessary to capture value from quick innovation, while protecting against business and end-user impact.

Tech Specs

Spinnaker consists of a number of independent microservices, with the Halyard CLI tool or the Kubernetes Operator (Beta) managing the lifecycle of the other services. A variety of SDLC tools integrate with Spinnaker, and its plugin framework makes Spinnaker more easily customizable to your needs. Plugins allow us to add system integrations without updating Spinnaker, broadening its potential to easily leverage the entire software delivery toolchain. With this in place, Spinnaker is evolving towards a smaller threat surface, enabling performance and operational advantages. Meanwhile, managed delivery, a newer Spinnaker featureset, provides declarative definitions of common infrastructure and other requirements; users can declare requirements using those prebuilt definitions and move changes through environments via a visual interface.

---

This repository centralizes issue tracking across Spinnaker for each microservice. The core code making up Spinnaker’s microservices is found in the other Spinnaker repositories.

Using Spinnaker

Spinnaker users should refer to the main Spinnaker site and Installation guide.

For more information on how Spinnaker is designed, see the Concepts topic in the documentation Overview.

If you’re interested in a detailed walkthrough of the Spinnaker systems, check the Spinnaker How-to guides, videos, and codelabs.

Setting Up Spinnaker For Development

To pull Spinnaker from source and set it up to run locally against any of the Cloud Providers, follow the Developer Setup Guide

keel's Issues

Prevent duplicate resource checks when > 1 Keel instance

Bake a new AMI when a new artifact version is detected

Core: Asset storage engine

Content-agnostic storage for desired asset state

Titus: Asset plugin

List detected resource diffs and resulting actions

EC2: ENIs

EC2: application load balancers

Resource deletion

Do we want the ability for users to define a desired state of “this resource MUST NOT exist” and delete it if it ever does?

Use SQL rather than Redis

Core: Rule processing

Periodic validation of current vs desired asset state and convergence via appropriate plugin.

Removing security group name from a cluster desired state does not delete it

Should red/black the cluster resulting in a new server group without the security group attached.

Veto: Execution windows

Core: prevent re-requesting convergence

Core: Initial architectural changes

EC2: Scheduled actions

Core: Version history for asset desired state

EC2: scaling policies

Declarative management of scaling policies.

Associate artifact versions with environments

Fix SQL / SSL

Don't attempt to anneal if dependent resources don't exist

e.g. security groups referenced by a cluster

Maintain a log of detected state changes

Veto: Application level enable/disable

Allow disabling / enabling of declarative management on a per-app basis.

EC2: classic load balancers

Description / convergence of load balancers. Should be at least equivalent functionality to earlier Keel implementation.

Deactivate diff/actuation if instance is disabled

EC2: clusters / server groups

Mocking framework

Hi,

hope find you well with this cold call.

I am an author of mocking framework for Kotlin

I see you are using mockito-kotlin.

I just want you to be aware that there is solution that fully supports Kotlin and ask to try it in your new/current projects.

I can help you if you answer to this issue.

Thanks and please star it

API: Protobuf APIs for core & asset plugins

Clients need to be able to pull in protos for e.g. Core and EC2 without any Java dependencies (crucially any dependencies of the asset plugins themselves) coming along for the ride.

Core: Plugin storage engine

API: Exportable protobuf API for EC2 plugin

Cannot include a self-ref ingress rule on a new security group

Actuation fails because the referenced security group does not exist during the operation that is creating it.

EC2: Route53

EC2: Aurora

API: Registering assets via REST

REST -> Protobuf -> Keel. This should include defining any gRPC API necessary to get desired asset descriptions from the REST "plugin" to the core application.

Core: health/status endpoints that list registered plugins

Cannot upsert security group description

Description is a required field when creating a security group. However the upsertSecurityGroup stage does not allow it to be changed. This can cause Keel to spin forever trying to actuate the desired state.

Event based diff/merge loop

Raise an event to go check resource state vs desired. That way we can trigger the same event on resource upsert for instant action.

Refactor: Rename AWS->EC2

When an artifact is registered get latest version from Igor

Prevent creation of duplicate resources

If a resource already exists with the name we generate for a new one we should respond with an error.

EC2: security groups

Description / convergence of security groups. Should be at least equivalent functionality to earlier Keel implementation.

EC2: Network load balancers

Prevent repeated attempts to converge the same diff

There are 2 aspects to this:

1. don't re-converge if the submitted task is still in progress
2. don't keep trying to re-converge a diff when the task fails

EC2: Elastic IPs

Core: Enforce / generate resource names

When a resource is created the metadata/name should be auto-generated using an appropriate mechanism for each resource type that creates a unique name/id.

REST API: List available resource kinds

Core: Authz for asset registration / updates

REST API: list resources for an application

Horizontal scaling of diff/actuation

Currently we're using Spring events to trigger diff/actuation of a resource but that ties processing to a single instance. We should:

1. ensure only one instance is triggering diff/actuation of a given resource.
2. allow diff/actuation to be performed on a different instance to the one requesting it.

SQS? Keiko?

Swagger

Separate resource validation / normalization from annealing

Instead of the validate method being part of the ResourceHandler interface we should create a ResourceValidator, allowing multiple to be registered for each resource kind. This has a couple of benefits:

1. Separation of concerns.
2. Ability to include Netflix-specific validation in keel-nflx.

We'll need to ensure all validators for a given resource kind can run in any order.