Giter Site home page Giter Site logo

console-operator's Introduction

Console Operator

An operator for OpenShift Console.

The console-operator installs and maintains the web console on a cluster.

Run on a 4.0.0 Cluster

The console operator is installed by default and will automatically maintain a console.

Development Setup

Clone the Repo & Build Locally

To avoid some of the standard quirks of gopath, the recommended way to clone and work with this repository is to do the following:

Cloning the Repo

Rather than ~/go for everything, provide separate gopaths. For this example we will use $HOME/gopaths as the root of our projects and consoleoperator as our project directory.

mkdir -p $HOME/gopaths/consoleoperator

Now get the repository from github.

cd $HOME/gopaths/consoleoperator
export GOPATH=`pwd`
go get github.com/openshift/console-operator

You may see a message like package github.com/openshift/console-operator: no Go files in $HOME/gopaths/consoleoperator/src/github.com/openshift/console-operator and you can safely ignore it.

You may now add your fork to the repo as a remote.

cd src/github.com/openshift/console-operator/
git remote rename origin upstream
git remote add origin {Your fork url}

Gopath

Note that we created $HOME/gopaths. This implies that each project will have its own gopath, so you will need to set that while working:

export GOPATH=$HOME/gopaths/consoleoperator

If you have multiple goprojects and don't want to fuss with maintaining this when you cd to different projects, give this script a try. It will add a command called calc_gopath to your prompt_command and set your gopath appropriately depending on the current working directory.

(At some point golang will fix $GOPATH and this won't be necessary)

Development & Building the Binary

Running the make command will build the binary:

make 

The binary output will be:

./_output/local/bin/<os>/<arch>/console

You may want to add this to your path or symlink it:

# if your ~/bin is in your path:
ln -s ./_output/local/bin/<os>/<arch>/console ~/bin/console 

However, it is no longer recommended to run the operator locally. Instead, you should be building a docker image and deploying it into a development cluster. Continue below for instructions to do this with a reasonable feedback loop.

Verify Source Code

Test gofmt and other verification tools:

make verify

Let gofmt automatically update your source:

gofmt -w ./pkg
gofmt -w ./cmd 

Run Unit Tests

make test-unit

It is suggested to run integration and e2e tests with CI. This is automatic when opening a PR.

Development Against a 4.0 Dev Cluster

The approach here is to build & deploy your code in a new container on a development cluster. Don't be put off by the need to redeploy your container, this is intended to provide a quick feedback loop.

Create a Cluster

To develop features for the console-operator, you will need to run your code against a dev cluster. The console-operator expects to be running in a container. It is difficult to fake a local environment, and the debuggin experience is not like debugging a real container. Instead, do the following to set yourself up to build your binary & deploy a new container quickly and frequently.

Visit https://try.openshift.com/, download the installer and create a cluster. Instructions (including pull secret) are maintained here.

# create a directory for your install config
# aws is recommended
mkdir ~/openshift/aws/us-east
cd ~/openshift/aws/us-east
# generate configs using the wizard
openshift-install create install-config
# then run the installer to get a cluster 
openshift-install create cluster --dir ~/openshift/aws/us-east --log-level debug

If successful, you should have gotten instructions to set KUBECONFIG, login to the console, etc.

Shut down CVO & the Default Console Operator

We don't want the default console-operator to run if we are going to test our own. Therefore, do the following:

# Instruct CVO to stop managing the console operator
# CVO's job is to ensure all of the operators are functioning correctly
# if we want to make changes to the operator, we need to tell CVO to stop caring.
oc apply -f examples/cvo-unmanage-operator.yaml
# Then, scale down the default console-operator 
oc scale --replicas 0 deployment console-operator --namespace openshift-console-operator

Note that you can also simply delete the CVO namespace if you want to turn it off completely (for all operators).

Now we should be ready to build & deploy the operator with our changes.

Preparation to Deploy Operator Changes Quickly

Typically to build your binary you will use the make command:

# this will build for your platform:
make
# if you are running OSX, you will need to build for linux doing something like:
OS_DEBUG=true OS_BUILD_PLATFORMS=linux/amd64 make
# note that you can build for mulitiple platforms with:
make build-cross

But the make step is included in the Dockerfile, so this does not need to be done manually. You can instead simply build the container image and push the it to your own registry:

# the pattern is:
docker build -t <registry>/<your-username>/console-operator:<version> .
# following: docker.io/openshift/origin-console-operator:latest
# for development, you are going to push to an alternate registry.
# specifically it can look something like this:
docker build -f Dockerfile.rhel7 -t quay.io/benjaminapetersen/console-operator:latest .

You can optionally build a specific version.

Then, push your image:

docker push <registry>/<your-username>/console-operator:<version>
# Be sure your repository is public else the image will not be able to be pulled later
docker push quay.io/benjaminapetersen/console-operator:latest

Then, you will want to deploy your new container. This means duplicating the manifests/07-operator.yaml and updating the line image: docker.io/openshift/origin-console-operator:latest to instead use the image you just pushed.

# duplicate the operator manifest to /examples or your ~/ home dir
cp manifests/07-operator.yaml ~/07-operator-alt-image.yaml

Then, update the image & replicas in your 07-operator-alt-image.yaml file:

# before
replicas: 2
image: docker.io/openshift/origin-console-operator:latest
# after 
# image: <registry>/<your-username>/console-operator:<version>
replicas: 1
image: quay.io/benjaminapetersen/console-operator:latest

And ensure that the imagePullPolicy is still Always. This will ensure a fast development feedback loop.

imagePullPolicy: Always

Deploying

At this point, your pattern will be

  • Change code
  • Build a new docker image
    • This will automatically & implicitly make build a new binary
  • Push the image to your repository
  • Delete the running console-operator pod
    • This will cause the Deployment to pull the image again before deploying a new pod

Which looks like the following:

# build binary + container
docker build -t quay.io/benjaminapetersen/console-operator:latest .
# push container
docker push quay.io/benjaminapetersen/console-operator:latest
# delete pod, trigger a new pull & deploy
oc delete pod console-operator --namespace openshift-console-operator

Docker containers are layered, so there should not be a significant time delay in between your pushes.

Manifest changes

If you are making changes to the manifests, you will need to oc apply the manifest.

Debugging

# inspect the clusteroperator object
oc describe clusteroperator console
# get all events in openshift-console-operator namespace
oc get events -n openshift-console-operator
# retrieve deployment info (including related events)
oc describe deployment console-operator -n openshift-console-operator
# retrieve pod info (including related events)
oc describe pod console-operator-<sha> -n openshift-console-operator
# watch the logs of the operator pod (scale down to 1, no need for mulitple during dev)
oc logs -f console-operator-<sha> -n openshift-console-operator
# exec into the pod
 oc exec -it console-operator-<sha> -- /bin/bash

Tips

If you don't know where your kubeconfig is due to running against multiple clusters this can be handy:

# just a high number
oc whoami --loglevel=100
# likely output will be $HOME/.kube/config 

If you need to know information about your cluster:

# this will list all images, associated github repo, and the commit # currently running.
# very useful to see if the image is running current code...or not.
oc adm release info --commits
# get just the list of images & sha256 digest
oc adm release info
# coming soon...
oc adm release extract 

Quick Starts

See the Quick Starts README for contributing console quick starts.

console-operator's People

Contributors

alebedev87 avatar benjaminapetersen avatar chenz4027 avatar csrwng avatar deads2k avatar debsmita1 avatar dperaza4dustbit avatar eggfoobar avatar enj avatar florkbr avatar jerolimov avatar jhadvig avatar kartikey-star avatar liouk avatar marun avatar ncdc avatar openshift-ci[bot] avatar openshift-merge-bot[bot] avatar openshift-merge-robot avatar pedjak avatar ravisantoshgudimetla avatar serenamarie125 avatar smarterclayton avatar spadgett avatar stlaz avatar therealjon avatar wking avatar yapei avatar yselkowitz avatar zherman0 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

console-operator's Issues

Create Event objects at meaningful points

If it makes sense, we should generate events for meaningful changes as these can be seen via the API.

Debugging initially is done via stdout, but we should improve this experience as much as possible.

Create oauth client via /deploy manifests

It may be wise to have OLM/SLO stamp out our oauth client via a manifest in our /deploy dir. This could eliminate some responsibility from our operator. We still would need to be able to update it with our route host, as well as random secret.

Cleanup is also an outstanding question in this case, would our operator be responsible for removing the oauth client if it isn't responsible for creating it?

@spadgett @enj

Update namespaces

See aos-devel thread. At the very least you need to split out the namespace for operator and operand.

Add cr.spec.hostname

Currently hostname is defaulted, but the user should be able to add a hostname.

If a hostname is provided, then a cert for the hostname should also be provided.

console should have multiple replicas ootb

right now the console deployment is only creating a single console replica, historically we always went with the same replicas as the number of masters by default, so 3, bare minimum would be 2, we must be HA by default

Add cr.spec.deleted

Deleted as a sibling to Paused.
This should allow the operator to exist, see a deleted state & not recreate a resource, even though oc delete -f cr.yaml also deletes the custom resource.

Assign a priority class to pods

Priority classes docs:
https://docs.openshift.com/container-platform/3.11/admin_guide/scheduling/priority_preemption.html#admin-guide-priority-preemption-priority-class

Example: https://github.com/openshift/cluster-monitoring-operator/search?q=priority&unscoped_q=priority

Notes: The pre-configured system priority classes (system-node-critical and system-cluster-critical) can only be assigned to pods in kube-system or openshift-* namespaces. Most likely, core operators and their pods should be assigned system-cluster-critical. Please do not assign system-node-critical (the highest priority) unless you are really sure about it.

Adoption (upgrade)

Certain things from a previous version will need to be honored. For example:

  • 3.11 to 4.0, will need to read the replicas off the existing console Deployment and set that on the cr.yaml, so that this can be honored.

Note that the method for doing adoption is yet to be determined.

Add CustomRole.Spec.Deleted

  • oc delete -f deploy/cr.yaml should delete the custom role & all of its sub-resources, including its cluster scoped oauth client.
  • cr.Spec.Deleted should be set to true

Migrate /deploy to /manifests

Prev /deploy is now /manfests/4.0.0. Creating & closing just for documentation purposes. There is no change to the files within (yet).

Setup CI

Initially can:

  • check builds
  • passes gofmt
  • etc

And then eventually add our tests (once written).

Log ONLY when changes actually update the config map

Currently operator logs look like:

time="2018-11-19T23:14:24Z" level=info msg="reconciling console..."
time="2018-11-19T23:14:25Z" level=info msg="created console service 'console'"
management state is - Managed 
time="2018-11-19T23:14:25Z" level=info msg="reconciling console..."
time="2018-11-19T23:14:25Z" level=info msg="created console service 'console'"
time="2018-11-19T23:14:25Z" level=info msg="created console service 'console'"
time="2018-11-19T23:14:25Z" level=info msg="console configmap base addr set to https://console-openshift-console.router.default.svc.cluster.local"
management state is - Managed 
time="2018-11-19T23:14:25Z" level=info msg="reconciling console..."
time="2018-11-19T23:14:25Z" level=info msg="console configmap base addr set to https://console-openshift-console.router.default.svc.cluster.local"
time="2018-11-19T23:14:25Z" level=info msg="created console configmap 'console-config'"
time="2018-11-19T23:14:25Z" level=info msg="console configmap base addr set to https://console-openshift-console.router.default.svc.cluster.local"
management state is - Managed 
time="2018-11-19T23:14:25Z" level=info msg="reconciling console..."
time="2018-11-19T23:14:25Z" level=info msg="created console configmap 'console-config'"
time="2018-11-19T23:14:25Z" level=info msg="created console deployment 'openshift-console'"
time="2018-11-19T23:14:25Z" level=info msg="created console deployment 'openshift-console'"
time="2018-11-19T23:14:25Z" level=info msg="console configmap base addr set to https://console-openshift-console.router.default.svc.cluster.local"
management state is - Managed 
time="2018-11-19T23:14:25Z" level=info msg="reconciling console..."
time="2018-11-19T23:14:25Z" level=info msg="route host is not yet available for oauth callback"
time="2018-11-19T23:14:25Z" level=info msg="oauth redirect URI set to https://console-openshift-console.router.default.svc.cluster.local"
time="2018-11-19T23:14:25Z" level=info msg="console configmap base addr set to https://console-openshift-console.router.default.svc.cluster.local"
management state is - Managed 
time="2018-11-19T23:14:25Z" level=info msg="oauth redirect URI set to https://console-openshift-console.router.default.svc.cluster.local"

But the "console configmap base addr set to" messages are hopefully no-ops, with the exception of the first. It would be nice to log only changes made by the operator (e.g. with a diff between the old and new configmap). I looked around briefly, but didn't see a way to get this. Maybe it's a generic issue with the SDK's Update function? Which is gone with SDK 0.1.0 (migration notes). Are there plans for porting to a more recent SDK release?

Tech Debt: update Sync funcs to be methods instead of funcs

// this is a little silly since the `co` is the first arg
actualDeployment, depChanged, depErr := SyncDeployment(co, recorder, operatorConfig, cm, serviceCAConfigMap, sec)

// instead 
actualDeployment, depChanged, depErr := co.SyncDeployment(recorder, operatorConfig, cm, serviceCAConfigMap, sec)

For all of the Sync funcs.

Fails to create deployement: `cannot list oauthclientauthorizations.oauth.openshift.io at the cluster scope`

$ oc desribe po/console-operator-7f48d87777-7fqhp -n openshift-console
...
Image:         registry.svc.ci.openshift.org/openshift/origin-v4.0-20181128050751@sha256:21fb0b92ad78b89c31083bfe7d42b7650174eebdc9d9e04d43fae502d28eb83a
...

$ oc logs -f po/console-operator-7f48d87777-7fqhp -n openshift-console
...
I1129 11:27:58.718708       1 reflector.go:240] Listing and watching *v1.OAuthClientAuthorization from github.com/openshift/console-operator/vendor/github.com/openshift/client-go/oauth/informers/externalversions/factory.go:101
E1129 11:27:58.720525       1 reflector.go:205] github.com/openshift/console-operator/vendor/github.com/openshift/client-go/oauth/informers/externalversions/factory.go:101: Failed to list *v1.OAuthClientAuthorization: oauthclientauthorizations.oauth.openshift.io is forbidden: User "system:serviceaccount:openshift-console:console-operator" cannot list oauthclientauthorizations.oauth.openshift.io at the cluster scope: no RBAC policy matched

Write integration / e2e tests

After #11 (ci) and #4 (initial test setup)

Tests

  • ManagementState changed from Managed, Unmananged, Removed, and should prove that the operator responds appropriately to each state.

Add cr.spec.oauthCert

If an auth server like keycloak is being used, then a cert file for keycloak will be necessary.

Add CustomRole.Spec.Paused

  • Deleted will be used to tell the operator to go delete the custom resource & clean up after itself.
  • Paused should basically cause the Operator to skip the Reconcile loop.

Tech Debt: avoid mutations

We should be following the pattern:

        // get the original
	original, err := c.Informers.Foo().V1().Bar().Lister().Get(api.TheThing)
	if err != nil {
		return nil, "", err
	}
        // make a copy
	copy := original.DeepCopy()
        // update the copy
        copy.Foo = foo
        copy.Bar = bar
        // send it off
	ret, err := c.Client.Foo().Update(copy)
	if err != nil {
		return nil, err
	}

@jhadvig @zherman0 for when we have time to tidy and polish we can go back and weed out the mutations.

Create ClusterOperator resource to report back status

example:

apiVersion: operatorstatus.openshift.io/v1
kind: ClusterOperator
metadata:
 creationTimestamp: '2018-10-17T19:11:16Z'
 generation: 1
 name: openshift-apiserver
 namespace: openshift-apiserver
 resourceVersion: '1356410'
 selfLink: >-
   /apis/operatorstatus.openshift.io/v1/namespaces/openshift-apiserver/clusteroperators/openshift-apiserver
 uid: 6be8a3f4-d240-11e8-80db-02ab100ffb3a
status:
 conditions:
   - lastTransitionTime: null
     message: replicas ready
     status: 'True'
     type: Available
   - lastTransitionTime: '2018-10-17T19:11:23Z'
     message: no errors found
     status: 'False'
     type: Failing
   - lastTransitionTime: null
     message: available and not waiting for a change
     status: 'False'
     type: Progressing
 version: 3.11.0

Add deployment probes

Some items are left off, such as the probes. Circle back around and fill out the last few bits.

API review on CRD

The CRD is an API. We will need an API review & ensure it is consistent with what other Operators are doing.

console is not being deployed to masters

the deployment for the console needs to have a toleration so that it can be scheduled to masters and a node selector so that it only targets masters, otherwise when the cluster is auto-scaling and trying to move things around it can take down the console

Set various cr.Status. properties

As things happen throughout the operator, the cr.Status should be filled out with meaningful properties. It may also be a good idea to update events at the same time.

Update `oauth.yaml` to the appropriate annotations once ready

Possibly will match something like this:

apiVersion: oauth.openshift.io/v1
kind: OAuthClient
metadata:
  name: openshift-console
  annotations:
    # match the route name
    # alpha.mo.will.let.us.define.route: console
    # alpha.mo.will.let.us.define.namespace: openshift-console
grantMethod: auto
respondWithChallenges: false

if @enj makes it work

/assign @enj

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.