fabric8-services / fabric8-tenant Goto Github PK

View Code? Open in Web Editor NEW

12.0 12.0 29.0 1.17 MB

Service responsible for provisioning and updating the tenant scoped services

License: Apache License 2.0

Ruby 0.10% Makefile 6.29% Shell 1.18% Go 92.33% PLSQL 0.01% TSQL 0.09%

fabric8-tenant's People

Contributors

Stargazers

Watchers

fabric8-tenant's Issues

Support for Early access / non updateble accounts

Certain OSiO accounts need early access to tenant deployments for 'pre live' verification, e.g. as part of EE tests, while other dev accounts would only want manual updates e.g. devs testing in prod.

Certain dev accounts want/need more access as well, e.g. admin of jenkins/che namspaces:

ibuziuk@redhat@com
amisevsk
mloriedo
sutan-1

Need brief description for the repo

See

Tenant response does not allow for the `user` type of namespace

See https://github.com/fabric8-services/fabric8-tenant/blob/master/design/tenant.go#L54

@aslakknutsen is there any reason for not allowing this?

Replace `glide` with `dep` to manage dependencies

add a REST API to test the users openshift + github tokens

it'd be good to have a REST API that tried to use the openshift + github tokens to check they are still valid.

e.g. its quite easy to recreate an openshift cluster (via say minishift) at the same URL and open the console and it thinks we're logged in as we have a cached token

A simple REST API that say, lists projects on openshift (or namespaces on kubernetes) or queries the user details on github could then report accurately if the toke is valid so that the UI could check this and if there's a failure redirect to the login page

Improve versioning of template/quotas files

The current versioning system of template files works in a way that every template has a version equal to the short SHA of the latest commit changing the file.
However, there is still one thing that should be improved. In the case of Che and Jenkins templates, there are two related files containing resource quotas and limits. They have the label version as well, however, it is hard to find it in the OS console and (from my point of view) everyone checks the label that is visible on the page of deployments or services and no-one tries to find the label of resource quotas nor limits. In addition, if the resource quotas file is changed, then it affects the whole namespace, however, this information is not easily visible.

My proposal is adding another label next to the version one - it could be called for example version-quotas and set the commit SHA of the quotas file there. With this, it will be transparent which version of quotas as well as template the namespace is running. It will be also easily trackable which commit affects the changes.

Add reorder of items in Template/List

e.g. ResourceQuote/LimitRange objects needs to be inserted before most other things.

Update local built oc with latest release v3.6.0

Remove HG from build requirements

Not needed to build the project nor to run the tests

Implement Cache Warm up and validation of Cluster tokens

Fetch all clusters and tokens on startup
Refresh cluster on a timer?
Token Validation be8f393

Allow different levels of logging

Currently it's a bit difficult to debug/develop with f8-tenant as the output logs get filled really quickly with info messages like this :

It would be nice to have some sort of filtering for some of the debug messages,

support tenant profiles (free, paid, huge, ...)

for running OSIO on OSIO we're gonna need a slightly different set of YAMLs for the tenant services (e.g. including a Nexus, Hubot etc). So different users may be configured/associated with a different level of tenant services.

Maybe for now we just have a free tier + prod tier flag/setting somewhere so we can have the default free YAML and a paid YAML that comes with a bit more quota/resources. Over time we could swizzle things to have more flexible mappings of users <-> templates to use for tenant services - but for now the free tier/prod tier flag might be enough

provide a REST API to return all tenant namespaces created by fabric8-tenant

so that we can implement a gofabric8 uninstall command to remove all resources installed into a kubernetes/openshift cluster by fabric8

lets have a `Delete Tenant Apps` REST API

when running the system tests we want to be able to easily delete all the BuildConfigs, Jenkins jobs and running apps/services/configmaps/routes so that we can clear down the system so that we can run tests again.

So a REST API we could then add into the Profile web UI (with sufficiently nice Red warning lights around it) so that the automated end to end tests can clear down a tenants apps and jenkins jobs.

e.g. some script along these lines:

oc delete bc --all -n ${ID}
oc delete build --all -n ${ID}
oc delete all --all -n ${ID}-test
oc delete all --all -n ${ID}-stage
oc delete all --all -n ${ID}-run

Then we also need to zap the Jenkins jobs (will figure out some code for that bit)

cannot create a Codebase on fabric8 on minishift (with openshift 3.6.0)

I just tried to create a codebase from the UI, it posted to http://wit-fabric8.192.168.64.82.nip.io/api/codebases/ae25a12e-44f7-4313-b46c-c5b28c17eca8/create and got this error:

{
    "errors":
    [
        {
            "code":"unknown_error",
            "detail":"panic: interface conversion: interface 
            {
            }
             is nil,
             not string\npanic(0xca77c0,
             0xc42094fdc0)\n\t/usr/local/go/src/runtime/panic.go:489 +0x2cf\ngithub.com/fabric8io/almighty-core/controller.getNamespace(0x12650a0,
             0xc4209a6b40,
             0x1a,
             0x0)\n\t/home/jenkins/go/src/github.com/fabric8io/almighty-core/controller/codebase.go:299 +0x19e\ngithub.com/fabric8io/almighty-core/controller.(*CodebaseController).Create(0xc4207383f0,
             0xc4209a6b40,
             0xc42071ef00,
             0xc420155ec0)\n\t/home/jenkins/go/src/github.com/fabric8io/almighty-core/controller/codebase.go:137 +0x1ac\ngithub.com/fabric8io/almighty-core/app.MountCodebaseController.func1(0x1264de0,
             0xc4209a6b10,
             0x1263be0,
             0xc42094f840,
             0xc42071ef00,
             0x0,
             0x0)\n\t/home/jenkins/go/src/github.com/fabric8io/almighty-core/app/controllers.go:232 +0xdd\ngithub.com/fabric8io/almighty-core/vendor/github.com/goadesign/goa/middleware/security/jwt.New.func1.1(0x1264de0,
             0xc4209a6b10,
             0x1263be0,
             0xc42094f840,
             0xc42071ef00,
             0x1264de0,
             0xc4209a6810)\n\t/home/jenkins/go/src/github.com/fabric8io/almighty-core/vendor/github.com/goadesign/goa/middleware/security/jwt/jwt.go:123 +0x563\ngithub.com/fabric8io/almighty-core/app.handleSecurity.func1(0x1264de0,
             0xc4209a6810,
             0x1263be0,
             0xc42094f840,
             0xc42071ef00,
             0x1264de0,
             0xc4209a67e0)\n\t/home/jenkins/go/src/github.com/fabric8io/almighty-core/app/security.go:49 +0x17a\ngithub.com/fabric8io/almighty-core/app.handleCodebaseOrigin.func1(0x1264de0,
             0xc4209a67e0,
             0x1263be0,
             0xc42094f840,
             0xc42071ef00,
             0x1264de0,
             0xc4209a6720)\n\t/home/jenkins/go/src/github.com/fabric8io/almighty-core/app/controllers.go:311 +0x389\ngithub.com/fabric8io/almighty-core/vendor/github.com/goadesign/goa.(*Controller).MuxHandler.func1.1(0x1264de0,
             0xc4209a6720,
             0x1263be0,
             0xc42094f840,
             0xc42071ef00,
             0xc4209a6720,
             0xc4fd00)\n\t/home/jenkins/go/src/github.com/fabric8io/almighty-core/vendor/github.com/goadesign/goa/service.go:254 +0xaa\ngithub.com/fabric8io/almighty-core/space/authz.InjectAuthzService.func1.1(0x1264de0,
             0xc4209a6240,
             0x1263be0,
             0xc42094f840,
             0xc42071ef00,
             0xc4209a6240,
             0xc420cc8f18)\n\t/home/jenkins/go/src/github.com/fabric8io/almighty-core/space/authz/authz.go:180 +0x159\ngithub.com/fabric8io/almighty-core/login.InjectTokenManager.func1.1(0x1264de0,
             0xc4209a6030,
             0x1263be0,
             0xc42094f840,
             0xc42071ef00,
             0x44487e,
             0x402b85)\n\t/home/jenkins/go/src/github.com/fabric8io/almighty-core/login/service.go:921 +0x98\ngithub.com/fabric8io/almighty-core/vendor/github.com/goadesign/goa/middleware.Recover.func1.1(0x1264de0,
             0xc4209a6030,
             0x1263be0,
             0xc42094f840,
             0xc42071ef00,
             0x0,
             0x0)\n\t/home/jenkins/go/src/github.com/fabric8io/almighty-core/vendor/github.com/goadesign/goa/middleware/recover.go:37 +0xac\ngithub.com/fabric8io/almighty-core/jsonapi.ErrorHandler.func1.1(0x1264de0,
             0xc4209a6030,
             0x1263be0,
             0xc42094f840,
             0xc42071ef00,
             0xc420213bc0,
             0x24)\n\t/home/jenkins/go/src/github.com/fabric8io/almighty-core/jsonapi/error_handler.go:37 +0x90\ngithub.com/fabric8io/almighty-core/vendor/github.com/goadesign/goa/middleware/gzip.Middleware.func2.1(0x1264de0,
             0xc4209a6030,
             0x1263be0,
             0xc42094f840,
             0xc42071ef00,
             0x0,
             0x0)\n\t/home/jenkins/go/src/github.com/fabric8io/almighty-core/vendor/github.com/goadesign/goa/middleware/gzip/middleware.go:98 +0x3bb\ngithub.com/fabric8io/almighty-core/log.LogRequest.func1.1(0x1264de0,
             0xc4209a6030,
             0x1263be0,
             0xc42094f840,
             0xc42071ef00,
             0xc4202af4a0,
             0x1264de0)\n\t/home/jenkins/go/src/github.com/fabric8io/almighty-core/log/log_request.go:87 +0xdab\ngithub.com/fabric8io/almighty-core/vendor/github.com/goadesign/goa/middleware.RequestIDWithHeaderAndLengthLimit.func1.1(0x1264de0,
             0xc420213ec0,
             0x1263be0,
             0xc42094f840,
             0xc42071ef00,
             0xc420213c80,
             0x1264de0)\n\t/home/jenkins/go/src/github.com/fabric8io/almighty-core/vendor/github.com/goadesign/goa/middleware/request_id.go:63 +0x177\ngithub.com/fabric8io/almighty-core/vendor/github.com/goadesign/goa.(*Controller).MuxHandler.func1(0x12644e0,
             0xc42096e8c0,
             0xc42071ef00,
             0xc420213c80)\n\t/home/jenkins/go/src/github.com/fabric8io/almighty-core/vendor/github.com/goadesign/goa/service.go:287 +0x347\ngithub.com/fabric8io/almighty-core/vendor/github.com/goadesign/goa.(*mux).Handle.func1(0x12644e0,
             0xc42096e8c0,
             0xc42071ef00,
             0xc420213c20)\n\t/home/jenkins/go/src/github.com/fabric8io/almighty-core/vendor/github.com/goadesign/goa/mux.go:59 +0x1b3\ngithub.com/fabric8io/almighty-core/vendor/github.com/dimfeld/httptreemux.(*TreeMux).ServeHTTP(0xc4204b6960,
             0x12644e0,
             0xc42096e8c0,
             0xc42071ef00)\n\t/home/jenkins/go/src/github.com/fabric8io/almighty-core/vendor/github.com/dimfeld/httptreemux/router.go:200 +0xe6\ngithub.com/fabric8io/almighty-core/vendor/github.com/goadesign/goa.(*mux).ServeHTTP(0xc420011720,
             0x12644e0,
             0xc42096e8c0,
             0xc42071ef00)\n\t/home/jenkins/go/src/github.com/fabric8io/almighty-core/vendor/github.com/goadesign/goa/mux.go:85 +0x4c\nnet/http.(*ServeMux).ServeHTTP(0x12b6d00,
             0x12644e0,
             0xc42096e8c0,
             0xc42071ef00)\n\t/usr/local/go/src/net/http/server.go:2238 +0x130\nnet/http.serverHandler.ServeHTTP(0xc420762160,
             0x12644e0,
             0xc42096e8c0,
             0xc42071ef00)\n\t/usr/local/go/src/net/http/server.go:2568 +0x92\nnet/http.(*conn).serve(0xc4202c48c0,
             0x1264d20,
             0xc42094ec40)\n\t/usr/local/go/src/net/http/server.go:1825 +0x612\ncreated by net/http.(*Server).Serve\n\t/usr/local/go/src/net/http/server.go:2668 +0x2ce\n",
            "status":"500",
            "title":"Unknown error"
        }
    ]
}

lets separate out the quotas into different templates and make their use optional

the quotas don't work on minishift and are not really needed anyway so lets use a separate template and environment variable to disable their use

Add auto token refresh by login via developers.redhat.com

OSO does not have a service account for the master user so there is no long lived token available. Each every now and then we need to login to developers.redhat.com outside of webflow to get a fresh token on oso.

@bartoszmajsak

allow a list of user names to be upgraded to a specific version of the tenant YAML

as a simpler alternative to #15 it might be simplest to just have a REST API we can use to update a list of user accounts to a given version of the fabric8-online YAML. We could then use this mechanism in pipelines to roll out a new version to a small set of test user accounts and run tests on it; then if those pass, role it out to a larger set of beta-testers before we submit the actual PR to update the init-tenant itself for all users.

I guess POSTing the YAML to use for the fabric8-online stuff might be the most flexible; as then we could test at the PR level before we merge.

It might simplify testing if we split up the jenkins/content-repository YAML from the Che YAML so that both of those PRs could test their own changes before releasing and raising a PR?

pq: relation "tenants" does not exist

I constantly find this error in prod-preview. Is that expected ? I found these errors using this query: "*does not exist*"

Add "warm-up" of Tenant Services on user login

e.g. Che Server takes to long to start to be done on demand when requested.
A possible workaround would be to start it on user login regardless, and then just let it idle away when ever. Hopefully it would take less time then user going to codebases and pressing edit.

Investigate why _tenant page doesn't trigger downloads from GH

When there is a blob SHA set on the https://prod-preview.openshift.io/_profile/_tenant page then tenant service should download the templates from the blob, but it doesn't download anything and always updates namespaces using the default templates. So the UI is either not sending correct parameters or tenant is not correctly reading it.

Add update support

Ref fabric8-services/fabric8-wit#1028

Include tenant id in "Not found" error response

Let's provide the ID of the tenant in the JSON-API response when the record was not found in the database or the namespace was not found on OpenShift, so external services (i.e, clients) can log a proper error.

Reset environment / update tenant is broken for che

looks like defaultAuthURL is wrong in the tenant configuration -
https://github.com/fabric8-services/fabric8-tenant/blob/master/configuration/configuration.go#L355

navigating to che server route results in 404

Add graceful shutdown

Avoid the process being killed during a rolling update before ongoing updates are completed.

Jenkins pipeline on fabric8-online project send PR and auto Merge with seemingly no regards to the status of the PR

Init tenant should set the correct Keycloak URL

In fabric8-online[1] we have introduced two new configs for che-server:

keycloak-oso-endpoint
keycloak-github-endpoint

These are currently hardcoded in fabric8io configuration with KC production URLs. That's an issue because we may need to use a distinct instance of KC based on which cluster Che will be deployed to (prod / prod-preview). These 2 config maps should be configured by the tenant init script as it's already done for hostname-http

[1] https://github.com/fabric8io/fabric8-online/blob/master/apps/che/src/main/fabric8/cm.yml#L20-L21

Add JENKINS_ROOT_URL environment variable

Following this issue fabric8-services/fabric8-tenant-jenkins#65, let get the JENKINS_ROOT_URL variable setting from the template variable and assign them.

Add documentation to setup init-tenant locally

Improve local testing/development

Things that should be improved in Makefiles and README

How I setup DB so I can run integration tests locally (even without docker)
I would need that DB for running tenant locally too (even if I don't get much from it but I still would like to be able to do it. See the next item)
How I run tenant locally so I can at least verify main() starts fine and /status is available.

Something like "make dev" could start a docker container with DB and tenant by default uses it. It's similar to what we have in WIT, Auth, Cluster and other new services created via fabric8-starter.
I can do different thing by using very simple commands:

"make" - builds binaries
"make dev" - builds binaries, starts all needed containers (DB and critical dependencies), runs the service locally in DevMode.

Next steps would be (we can take at this later):

"make dev-openshift" does the same as "make dev" but instead of docker-compose it uses minishift.
do something about running tenant in Dev Mode locally so we can test stuff beyond /status
it's OK to split it to smaller sub tasks and do it step by step. To improve it gradually over some time instead of spending too much time on that right now.

update does not seem to trigger a deployment when the Jenkins DeploymentConfig is updated (OpenShift cancels it immediately)

well the update is fine; it updates the Jenkins DC just fine - but for some reason openshift then decides to ignore that, cancel the deployment so the version 0 keeps running despite version N being more recent. I found I had to manually go into the openshift console and click the Deploy button by hand.

I wonder if we need some kinda check to make sure that after the DC has been updated we ensure that openshift properly updated. Maybe we need to detect the cancelled state and maybe scale down the DC then try scale it up again? Something like that. Maybe we need to raise this as an openshift bug - it may be our quotas + non-rolling upgrades that cause the issue maybe?

Jenkins pipline CD step sends PR to promote-to-prod before centosci job is done

This cause prod to attempt to deploy new DC and OSD failing to pull image for ~5 min, assuming the centosci job finish at all

Remove user's OSO project when resetting his/her environment

removing the user from tenant as part of env reset. So the namespaces will be recreated during the next init tenant. This means cleaning the record in the DB as well as the actual OSO project

Include cluster url in logs

Usefull so we can know if errors occur on a specific OS cluster

Use different strategies for Init vs Update

We moved to use 'oc apply' due to update issues with the pure brute force POST/DELETE method, but that takes the init tenant time from about 400ms to 2:50 min. This causes some timing issues down the lines in e.g. the Forge Wizard that require the Jenkins Services to be around to discover the Jenkins Master URL to post as a GitHub WebHook Callback.

Split the Init and Update strategies, using pure POST/DELETE on Init and oc apply on Update.

add a REST API we can invoke from `gofabric8` to update a tenant passing just the OC token

we'd need to then do a reverse lookup of the KC token inside the REST endpoint so that we can do an update tenant.

this would then mean we could have a gofabric8 recreate tenant type command which deletes a tenant, invokes the update and then asserts the tenant is setup correctly etc.

It might be nice to allow a version to be passed in as an optional argument too.

Include TextFixture lib to test the DB services

new users don't get che + jenkins (but do get run + stage namespaces which is odd)

see the issue here: fabric8-ui/fabric8-ui#1606 (comment)

RFE: a hook to manipulate tenant deployment-configuration before sending to OSO

@aslakknutsen suggested that an alternative to fabric8-services/fabric8-tenant-jenkins#57 (and several others along that line) could be the adjustment of original DCs on a per-tenant basis.

We may need to add routes, maybe a container to the pod(s); we don't know yet the entirety. So it'd be great to have a facility that lets us modify DCs en-route to OSO, but without too much hard-coding within f8 proper.

Need to update init script for setting correct values for a new GitHub token endpoint in che-sever

Need to update init script for setting correct values for new GitHub token endpoint:

https://auth.openshift.io/api/token?for=https://github.com
https://auth.prod-preview.openshift.io/api/token?for=https://github.com

Related PRs

What's the point of base64 encoded passwords?

What's the point of base64 encoded passwords here:

postgres.password: bXlzZWNyZXRwYXNzc3dvcmQK

Security through obscurity?

when updating lets remove all `-test` namespaces which were created by fabric8-init-tenant

we should iterate through all the users projects which are named *-test and remove any which are annotated with

metadata:
  annotations:
    openshift.io/requester: devtools-sre

Reset environment failed

Resetting environment failed
Request

Request URL: https://prod-preview.openshift.io/api/user/services
Request Method: DELETE
Status Code: 500 Internal Server Error

Error

{
  "errors": [
    {
      "code": "internal_error",
      "detail": "panic: interface conversion: interface {} is nil, not string\npanic(0xaebf00, 0xc42005b5c0)\n\t/usr/lib/golang/src/runtime/panic.go:491 +0x283\ngithub.com/fabric8-services/fabric8-tenant/toggles.WithContext(0xf799e0, 0xc420233470, 0x0)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/toggles/toggles.go:36 +0x1be\ngithub.com/fabric8-services/fabric8-tenant/toggles.IsEnabled(0xf799e0, 0xc420233470, 0xbb91b8, 0x17, 0xc4202a0500, 0x2b)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/toggles/toggles.go:47 +0x48\ngithub.com/fabric8-services/fabric8-tenant/openshift.LoadProcessedTemplates(0xf799e0, 0xc420233470, 0xc42049f1a0, 0x2d, 0xc420469a90, 0xc, 0xc4202a05a0, 0x2b, 0xf71b20, 0xc4204fa0f0, ...)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/openshift/process_template.go:126 +0x7dd\ngithub.com/fabric8-services/fabric8-tenant/openshift.CleanTenant(0xf799e0, 0xc420233470, 0xc42049f1a0, 0x2d, 0xc420469a90, 0xc, 0xc4202a05a0, 0x2b, 0xf71b20, 0xc4204fa0f0, ...)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/openshift/clean_tenant.go:13 +0xad\ngithub.com/fabric8-services/fabric8-tenant/controller.(*TenantController).Clean(0xc4204fa2d0, 0xc420233470, 0xc420244800, 0xc42005e540)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/controller/tenant.go:258 +0x8d3\ngithub.com/fabric8-services/fabric8-tenant/app.MountTenantController.func1(0xf797e0, 0xc420233440, 0xf77b60, 0xc42005b440, 0xc420244800, 0x0, 0x0)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/app/controllers.go:114 +0xdd\ngithub.com/fabric8-services/fabric8-tenant/vendor/github.com/goadesign/goa/middleware/security/jwt.New.func1.1(0xf797e0, 0xc420232450, 0xf77b60, 0xc42005b440, 0xc420244800, 0xf797e0, 0xc420232450)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/vendor/github.com/goadesign/goa/middleware/security/jwt/jwt.go:123 +0x769\ngithub.com/fabric8-services/fabric8-tenant/app.handleSecurity.func1(0xf797e0, 0xc420232450, 0xf77b60, 0xc42005b440, 0xc420244800, 0xb1f300, 0xc42005b440)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/app/security.go:49 +0x17a\ngithub.com/fabric8-services/fabric8-tenant/app.handleTenantOrigin.func1(0xf797e0, 0xc420079d40, 0xf77b60, 0xc42005b440, 0xc420244800, 0x0, 0x0)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/app/controllers.go:181 +0x502\ngithub.com/fabric8-services/fabric8-tenant/vendor/github.com/goadesign/goa.(*Controller).MuxHandler.func1.1(0xf797e0, 0xc420079d40, 0xf77b60, 0xc42005b440, 0xc420244800, 0x0, 0x0)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/vendor/github.com/goadesign/goa/service.go:270 +0xaa\ngithub.com/fabric8-services/fabric8-tenant/vendor/github.com/fabric8-services/fabric8-wit/log.LogRequest.func1.1(0xf797e0, 0xc420079d40, 0xf77b60, 0xc42005b440, 0xc420244800, 0xc4204f6507, 0x4b6)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/vendor/github.com/fabric8-services/fabric8-wit/log/log_request.go:86 +0xcf4\ngithub.com/fabric8-services/fabric8-tenant/vendor/github.com/fabric8-services/fabric8-wit/goamiddleware.TokenContext.func1.1(0xf797e0, 0xc4200793e0, 0xf77b60, 0xc42005b440, 0xc420244800, 0x0, 0x0)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/vendor/github.com/fabric8-services/fabric8-wit/goamiddleware/jwt_token_context.go:75 +0x285\ngithub.com/fabric8-services/fabric8-tenant/vendor/github.com/goadesign/goa/middleware.Recover.func1.1(0xf797e0, 0xc4200793e0, 0xf77b60, 0xc42005b440, 0xc420244800, 0x0, 0x0)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/vendor/github.com/goadesign/goa/middleware/recover.go:37 +0xa0\ngithub.com/fabric8-services/fabric8-tenant/jsonapi.ErrorHandler.func1.1(0xf797e0, 0xc4200793e0, 0xf77b60, 0xc42005b440, 0xc420244800, 0xc420030c00, 0xc4208ff8e0)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/jsonapi/error_handler.go:38 +0x90\ngithub.com/fabric8-services/fabric8-tenant/vendor/github.com/goadesign/goa/middleware/gzip.Middleware.func2.1(0xf797e0, 0xc4200793e0, 0xf77b60, 0xc42005b440, 0xc420244800, 0xc42042e910, 0xf797e0)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/vendor/github.com/goadesign/goa/middleware/gzip/middleware.go:98 +0x3ac\ngithub.com/fabric8-services/fabric8-tenant/vendor/github.com/goadesign/goa/middleware.RequestIDWithHeaderAndLengthLimit.func1.1(0xf797e0, 0xc4200793b0, 0xf77b60, 0xc42005b440, 0xc420244800, 0xc4200791d0, 0xf797e0)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/vendor/github.com/goadesign/goa/middleware/request_id.go:63 +0x144\ngithub.com/fabric8-services/fabric8-tenant/vendor/github.com/goadesign/goa.(*Controller).MuxHandler.func1(0xf78b20, 0xc4202cab60, 0xc420244800, 0xc4200791d0)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/vendor/github.com/goadesign/goa/service.go:303 +0x336\ngithub.com/fabric8-services/fabric8-tenant/vendor/github.com/goadesign/goa.(*mux).Handle.func1(0xf78b20, 0xc4202cab60, 0xc420244800, 0x0)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/vendor/github.com/goadesign/goa/mux.go:59 +0x1cb\ngithub.com/fabric8-services/fabric8-tenant/vendor/github.com/dimfeld/httptreemux.(*TreeMux).ServeLookupResult(0xc42022e500, 0xf78b20, 0xc4202cab60, 0xc420244800, 0xc8, 0xc4202e81b0, 0x0, 0x0)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/vendor/github.com/dimfeld/httptreemux/router.go:247 +0x133\ngithub.com/fabric8-services/fabric8-tenant/vendor/github.com/dimfeld/httptreemux.(*TreeMux).ServeHTTP(0xc42022e500, 0xf78b20, 0xc4202cab60, 0xc420244800)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/vendor/github.com/dimfeld/httptreemux/router.go:268 +0xdb\ngithub.com/fabric8-services/fabric8-tenant/vendor/github.com/goadesign/goa.(*mux).ServeHTTP(0xc42015ebd0, 0xf78b20, 0xc4202cab60, 0xc420244800)\n\t/tmp/go/src/github.com/fabric8-services/fabric8-tenant/vendor/github.com/goadesign/goa/mux.go:85 +0x4c\nnet/http.(*ServeMux).ServeHTTP(0xfda7a0, 0xf78b20, 0xc4202cab60, 0xc420244800)\n\t/usr/lib/golang/src/net/http/server.go:2254 +0x130\nnet/http.serverHandler.ServeHTTP(0xc4202a48f0, 0xf78b20, 0xc4202cab60, 0xc420244800)\n\t/usr/lib/golang/src/net/http/server.go:2619 +0xb4\nnet/http.(*conn).serve(0xc4201d0820, 0xf79720, 0xc420242000)\n\t/usr/lib/golang/src/net/http/server.go:1801 +0x71d\ncreated by net/http.(*Server).Serve\n\t/usr/lib/golang/src/net/http/server.go:2720 +0x288\n",
      "status": "500",
      "title": "Internal error"
    }
  ]
}

drop the -test project

We are still deploying with the -test project, we should stop doing that. Once we are able to do new tenant deploys without that, I can go clear out the existing ones manually ( so that does not need any automation from the f8tenant side ).

Unable to compile; dependencies or instructions are missing?

While debugging an issue with platform stability of Minishift, I ran into an issue with the tenant description. Below is what I do:

$ minishift start --cpus 4 --memory 8GB --disk-size 50GB --iso-url centos
$ eval `minishift oc-env`
$ eval `minishift docker-env`
$ docker pull fabric8/fabric8-tenant:dev
$ docker pull fabric8/fabric8-tenant
$ docker pull fabric8/fabric8-dependency-wait-service:v6632df1
$ docker pull fabric8/keycloak-postgres:v15751c8
$ docker pull fabric8/fabric8-auth:v26e19df
$ docker pull fabric8/fabric8-dependency-wait-service:v6632df1
$ docker pull rhche/che-starter:7a6345
$ docker pull fabric8/configmapcontroller:2.3.7
$ docker pull fabric8/exposecontroller:2.3.28
$ docker pull fabric8/fabric8-ui:v318d700
$ docker pull fabric8/fabric8-dependency-wait-service:v6632df1
$ docker pull fabric8/generator-backend:1.0.84
$ docker pull fabric8/fabric8-tenant:v8794468
$ docker pull fabric8/fabric8-dependency-wait-service:v6632df1
$ docker pull registry.centos.org/postgresql/postgresql:9.6
$ docker pull fabric8/keycloak-postgres:v15751c8
$ docker pull fabric8/builder-clients:0.11
$ docker pull jimmidyson/pemtokeystore:v0.2.0
$ git clone https://github.com/fabric8io/fabric8-platform.git
$ cd fabric8-platform
$ ./install.sh
$ cd apps/init-tenant
$ mvn fabric8:deploy  -Pdev
$ git clone https://github.com/fabric8-services/fabric8-tenant $GOPATH/src/github.com/fabric8-services/fabric8-tenant
$ cd $GOPATH/src/github.com/fabric8-services/fabric8-tenant
$ make vendor
...
[INFO]  --> Setting version for gopkg.in/square/go-jose.v2 to f8f38de21b4dcd69d0413faf231983f5fd6634b1.
[INFO]  Exporting resolved dependencies...
[INFO]  --> Exporting github.com/armon/go-metrics
[INFO]  --> Exporting github.com/bitly/go-simplejson
[INFO]  --> Exporting github.com/dimfeld/httptreemux
[INFO]  --> Exporting github.com/fabric8-services/fabric8-auth
[INFO]  --> Exporting github.com/ajg/form
[INFO]  --> Exporting github.com/fsnotify/fsnotify
[INFO]  --> Exporting github.com/goadesign/goa
[INFO]  --> Exporting github.com/golang/lint
[INFO]  --> Exporting github.com/hashicorp/golang-lru
[INFO]  --> Exporting github.com/jinzhu/gorm
[INFO]  --> Exporting github.com/hashicorp/go-immutable-radix
[INFO]  --> Exporting github.com/jinzhu/inflection
[INFO]  --> Exporting github.com/hashicorp/hcl
[INFO]  --> Exporting github.com/fabric8-services/fabric8-wit
[INFO]  --> Exporting github.com/jteeuwen/go-bindata
[INFO]  --> Exporting github.com/dimfeld/httppath
[INFO]  --> Exporting github.com/elazarl/go-bindata-assetfs
[INFO]  --> Exporting github.com/dgrijalva/jwt-go
[INFO]  --> Exporting github.com/lib/pq
[INFO]  --> Exporting github.com/fzipp/gocyclo
[INFO]  --> Exporting github.com/magiconair/properties
[INFO]  --> Exporting github.com/pkg/errors
[INFO]  --> Exporting github.com/mitchellh/mapstructure
[INFO]  --> Exporting github.com/pelletier/go-toml
[INFO]  --> Exporting github.com/spf13/cast
[INFO]  --> Exporting github.com/spf13/jwalterweatherman
[INFO]  --> Exporting github.com/spf13/cobra
[INFO]  --> Exporting github.com/Unleash/unleash-client-go
[INFO]  --> Exporting github.com/manveru/faker
[INFO]  --> Exporting github.com/stretchr/testify
[INFO]  --> Exporting github.com/spf13/viper
[INFO]  --> Exporting github.com/sirupsen/logrus
[INFO]  --> Exporting github.com/satori/go.uuid
[INFO]  --> Exporting github.com/spf13/pflag
[INFO]  --> Exporting github.com/zach-klippenstein/goregen
[INFO]  --> Exporting github.com/pelletier/go-buffruneio
[INFO]  --> Exporting github.com/spf13/afero
[INFO]  --> Exporting github.com/wadey/gocovmerge
[INFO]  --> Exporting github.com/davecgh/go-spew
[INFO]  --> Exporting github.com/jstemmer/go-junit-report
[INFO]  --> Exporting github.com/pmezard/go-difflib
[INFO]  --> Exporting golang.org/x/net
[INFO]  --> Exporting golang.org/x/crypto
[INFO]  --> Exporting golang.org/x/sys
[INFO]  --> Exporting golang.org/x/tools
[INFO]  --> Exporting golang.org/x/text
[INFO]  --> Exporting gopkg.in/yaml.v2
[INFO]  --> Exporting gopkg.in/square/go-jose.v2
[INFO]  Replacing existing vendor dependencies
touch vendor
$ make kube-redeploy
mkdir -p bin/docker
cp Dockerfile.dev bin/docker/Dockerfile
GO15VENDOREXPERIMENT=1 GOARCH=amd64 GOOS=linux go build -o bin/docker/fabric8-tenant-linux
main.go:14:2: cannot find package "github.com/fabric8-services/fabric8-tenant/app" in any of:
        /var/workspaces/fabric8-tenant/src/github.com/fabric8-services/fabric8-tenant/vendor/github.com/fabric8-services/fabric8-tenant/app (vendor tree)
        /usr/lib/golang/src/github.com/fabric8-services/fabric8-tenant/app (from $GOROOT)
        /var/workspaces/fabric8-tenant/src/github.com/fabric8-services/fabric8-tenant/app (from $GOPATH)
main.go:15:2: cannot find package "github.com/fabric8-services/fabric8-tenant/auth" in any of:
        /var/workspaces/fabric8-tenant/src/github.com/fabric8-services/fabric8-tenant/vendor/github.com/fabric8-services/fabric8-tenant/auth (vendor tree)
        /usr/lib/golang/src/github.com/fabric8-services/fabric8-tenant/auth (from $GOROOT)
        /var/workspaces/fabric8-tenant/src/github.com/fabric8-services/fabric8-tenant/auth (from $GOPATH)
make: *** [Makefile:284: bin/docker/fabric8-tenant-linux] Error 1

It seems some dependencies are not satisfied corectly or instructions are missing

In short, I am following https://github.com/fabric8-services/fabric8-tenant#rapid-development-on-minikube--minishift (but notice that in these instructions the command make vendor is also missing)

Align log field "fields.time" to ms

Need to perform cluster-wide tenant in order to apply `2.0.90` version of fabric8-tenant-che

the latest 2.0.90 version of fabric8-tenant-che contains important updates required for workspace.next

adding new che-workspace sa with view & exec permissions in *-che namespace [1]
increasing quota of services to 15 [2]

Need to perform cluster-wide tenant update in order to rollout this change to all the prod / prod-preview tenants.

[1] fabric8-services/fabric8-tenant-che#111
[2] fabric8-services/fabric8-tenant-che#113

the api/status should do a more end2end test

either the /api/status should do a more thorough test, or we should get another endpoint that can call the backing api service, to call through f8tenant to the actual tenant services openshift cluster to validate all the moving parts enroute.

support multiple user kubernetes/openshift clusters

we should always support the openshift online cluster (there may be many of them actually) but also allow users to add/edit/delete/select their own personal clusters (e.g. local minishift or their own cluster on AWS/GCE/Azure).

So we need a REST API to add/edit/delete user clusters. Each user cluster would look something vaguely like:

type UserOpenShiftCluster struct {
	ID        uuid.UUID `sql:"type:uuid default uuid_generate_v4()" gorm:"primary_key"`
	TenantID  uuid.UUID `sql:"type:uuid"`
	CreatedAt time.Time
	UpdatedAt time.Time
	DeletedAt *time.Time
	Name      string
	ApiServerURL string
	IdentityKey string
}

Where the IdentityKey is used to find the identity/token inside the keycloak REST API at something like /auth/realms/fabric8/broker/${identityKey}/token.

When creating the UserOpenShiftCluster the server should populate some values: ID, IdentityKey, CreatedAt in particular so that IdentityKey does not clash with another existing cluster or existing OSO cluster.

allow the init-tentant binary to be invoked on the CLI pointing at a local set of template files

so that we can reuse the same golang code inside CI / CD pipelines to test new templates out before we release them

fabric8-services / fabric8-tenant Goto Github PK

fabric8-tenant's People

Contributors

Stargazers

Watchers

Forkers

fabric8-tenant's Issues

Recommend Projects

Recommend Topics

Recommend Org