fabric8io/jube

jube is a deprecated pure java implementation of kubernetes. Please look at kansible now instead!

Deprecated - please now use kansible

NOTE this project is now deprecated and replaced by kansible!

We've figured out a much simpler and cleaner way of using non-docker processes on Windows, AIX, Solaris, HPUX and old linuxes which makes it easier to mix and match native operating systems proceesses with docker containers.

So please take a look at kansible as a nicer alternative to Jube!

kansible is better than Jube because:

• kansible provides a single pane of glass for all your operating system processes on windows and Unix along with all your linux based docker containers
• kansible lets you reuse all Kubernetes features for your operating system processes too:
  • service discovery using DNS
  • high availability and scaling manually or automatically through metrics
  • liveness and readiness checks
  • centralised logging and metrics
• kansible also lets you use the power of Ansible playbooks to provision your software, install JDKs, create user accounts and so forth

jube's Issues

Disable maven indexer facade test

As it index the m2 repo which takes a long time, we could just run it as a special test.
Also it generates 700mb+ temporary file in .hawtio

Fix itest on windows

The jube itest fails on windows with a weird cannot find start.bat, despite it appears in the processes dir.

the http host ports sometimes disappear during startup so that the pod doesn't become usable in a service

Support /version in the rest API

It's used to display the kubernetes version in hawtio, looks like we'd want an object like:

{
  "major": "0",
  "minor": "2"
}

though it doesn't have to match exactly what kubernetes returns, for example we could just return the actual jube release version and figure it out on the hawtio side :-)

creating new image zips from base images can remove ports defined in the base image by accident

we should preserve ports by default in the base image so folks don't have to copy/paste port definitions into derived projects. e.g. all projects should inherit jolokia by default

create an image zip for Tomcat for fabric8/tomcat-8.0

Killed container is not detected.

If you kill a container that jube starts up /w kill -9, jube seems to think it's still running via hawtio.

Add about mbean to jube like in hawtio

so we can show the version in the web console, and CLI and whatnot

support logstash collection of logs from containers similar to kubernetes use of fluentd

see https://github.com/GoogleCloudPlatform/kubernetes/blob/master/contrib/logging/fluentd-ek/README.md

fabric8 2.0 prerequisite?

Hi,

I just tried building and tripped at the first hurdle: "Non-resolvable import POM: Could not find artifact io.fabric8:fabric8-project:pom:2.0.0-SNAPSHOT" Is there a nightly maven repo with the 2.0 fabric8 builds, or does this mean there is a prerequisite of having to build fabric8 2.0 locally first?

Add windows scripts for wildfly image

jube - deploying an app on a tomcat/wildlfy image - Configure web context-path

When you have a bunch of WAR(s) to run on a tomcat container, they get assigned their file name as the context path.

I think end users should be able to set some option in their app (eg supported by the fabric8 plugin generating the kubernetes.json) so they can configure this in their maven project.

Then the process manager can ensure the WAR files gets renamed to match the context path. And use name ROOT.war is it should be the root context path.

avoid having to add a maven dependency in the jube plugin for base images

right now if you've not downloaded a base image a build tends to fail

we should default to using the git repository/wiki inside the jube installation (to avoid clashing with other hawtio consoles)

Karaf image - Adds APP_BASE to env.sh during installation

The karaf image has a env.sh which sets some KARAF_OPTS. But when jube installs an app using the fabric8/karaf image, it adds APP_BASE to the evh.sh so it becomes

## defines the default environment settings

# TODO add something like this:
#
# export MAIN="MyClassName"

export KARAF_OPTS="-Dkaraf.shutdown.pid.file=$APP_BASE/process.pid"
export KARAF_OPTS="$KARAF_OPTS -javaagent:$APP_BASE/jolokia-agent.jar=host=0.0.0.0,port=8778,authMode=jaas,realm=karaf,user=admin,password=admin"


export APP_BASE="/Users/davsclaus/workspace/jube/app/./processes/fabric8KarafBeginnerCamelCbrController-1-fabric8-karaf-beginner-camel-cbr/apache-karaf-2.4.0"

The scripts form the image didn't expect this, so they add apache-karaf-2.4.0 as directory too, so it becomes double.

/Users/davsclaus/workspace/jube/app/./processes/fabric8KarafBeginnerCamelCbrController-1-fabric8-karaf-beginner-camel-cbr/start.sh: line 29: /Users/davsclaus/workspace/jube/app/./processes/fabric8KarafBeginnerCamelCbrController-1-fabric8-karaf-beginner-camel-cbr/apache-karaf-2.4.0/apache-karaf-2.4.0/bin/start: No such file or directory

rename karaf image to fabric8/karaf-2.4 to match the docker image

Support (prefer?) groovy scripts instead of just bat/sh for jube images

Cross-platform scripting is annoying to say the least - groovy would abstract those nasty details away. Don't think this should mean sh/bat scripts shouldn't be supported as not everyone would want to use groovy, but would simplify for those that do want to use something cross-platform.

Add windows scripts to run jube

Building jube image - Does not include src/main/resources/OSGI-INF in the classes directory

eg I have a project with src/main/resources/OSGI-INF/blueprint directory with some blueprint xml files. In the generated target/ -image.zip file, then the classes directory is empty.

It should copy over the src/main/resources structure. Maybe it doesn't support sub dirs, or the OSGI-INF directory etc.

create image zip for: dockerfile/elasticsearch

so we can share kubernetes ES logging between kubernetes and jube
https://github.com/GoogleCloudPlatform/kubernetes/blob/master/contrib/logging/fluentd-ek/README.md

configure logging to use JSON by default so its easier to work with logstash?

for java/karaf/tomcat/wildfly images we should default to use JSON logging so that things just work nicely with logstash and elasticsearch?

we should expose ZK in Jube as its already running by default - and make it so that users don't try to run the ZK app and it fails

we should expose the service too so that folks can connect to the ZK ensemble using jube / kube services if its easy to do...

expose the kubernetes and kubernetes-ro services by default inside the services REST API like the upstream kubernetes & openshift

getting a CXF + CDI war to work nicely

reduce the logging on startup

Maven Repository Config?

Hi, I'm obviously overlooking something but I can't seem to find where to configure maven repositories. Consequently I am getting the following exception when attempting to run the example quickstart...

SEVERE: Failed to create pod: fabric8QuickstartJavaCamelSpring-5. java.io.IOException: Failed to resolve mvn:io.fabric8.jube.images.fabric8/quickstart-java-camel-spring/2.0-SNAPSHOT/zip/image. org.sonatype.aether.resolution.DependencyResolutionException: Could not find artifact io.fabric8.jube.images.fabric8:quickstart-java-camel-spring:zip:image:2.0-SNAPSHOT in central (http://repo2.maven.org/maven2/)

Could it be due to my local .m2 repository not being in my home directory as it would be by default?

jube doesn't export the service IP so that links on the hawtio services page don't work

fix up the rootWar assembly so that it adds the ROOT.war into the maven folder

karaf image is not listening on jube ports using environment variables $SSH_PORT / $RMI_PORT / $HTTP_PORT etc

to run 3 karaf instances on a machine on jube we need the karaf image to listen on port numbers defined by environment variables passed in by jube; by adding them to the ports.properties file....

SSH = 8101
RMI = 1090
HTTP = 8181

then modify the karaf etc/* files to use system properties / env vars to define the port values

create an image zip for karaf for the docker image fabric8/karaf-2.4

use consistent names for properties in jube maven plugin as for docker (e.g. docker.from / docker.image rather than docker.baseImage / docker.dataImage)

nicer way to bootstrap a jube cluster & ZK ensemble

right now if you boot up jube, if there is no ZOOKEEPER_URL defined it creates one on the fly on 2181.

Then the next jube you start up you specify host:2181 as the URL and it joins.

This is cool but results in an ensemble of 1. Setting up 3/5 node ensembles outside of kuberrnetes is hard.

I wonder how hard it'd be to add a kinda incremental formation?

Eg

• node1 starts; it creates a 1 node ensemble & elects a master
• node2 and node3 start, joining with an env var used as a seed to find a Zk server
• the master now detects there are 3 hosts now (each node registers into ZK) so can now decide to upgrade to 3 node ensemble - adding the address/port of node2/3 to Zk and telling node 2/3 to become ensemble members
• all nodes watch the /jube/ensemble area in Zk and then write to local disk the Zk of where the Zk ensemble nodes are so they can use those addresses on restart as seeds to find a Zk server

Then folks just have to run 1 node; then set an env var to point to it and then boot up new nodes. Then there is a simple automatic way to create a 3/5 node ensemble.

I guess multicast discovery is another option; that's much more complicated as it's tricky electing an initial leader for the first Zk server & master (chicken & egg!) so this env var approach is maybe nice & simple.

Once there are 3/5 nodes in the ensemble the system could output the URL of the 3/5 nodes to connect to from then on.

Maybe a simple multicast system could be - once a node starts in 'create' mode; it starts a Zk ensemble - then the master advertises it's URL on multicast. Other nodes can then discover the Zk URL via multicast & be informed when the ensemble changes from 1 -> 3.

The tricky thing is if 2 nodes start in create mode; when they find each other one has to switch from ensemble/master to client so there is only one master/ensemble node initially. Maybe if the startup logic waits long enough before detecting other masters & then has a way to pick which one is the real master if 2 nodes start at the same time (eg always pick the one with the smaller URL string) then the whole thing could use multicast discovery (or env vars when discover isn't available?)

create an image zip for wildfly

make it easier to configure the remote and local maven repositories searched for images

via system properties or environment variables

App with src/data does not have the data files installed

The camel-spring quickstart from fabric8 v2 branch has some sample data in src/data eg message1.xml and message2.xml.

jube should include those files when it installs a process, eg creates the src/data but the dir is empty.

Instead it should, create the dir as "data", eg omit src. And include the files from src/data.

failed to create pod in jube if the image zip version is SNAPSHOT

even though the SNAPSHOT image zip is actually in my maven local repo, get exception from jube console like

ERROR: Failed to create pod: fabric8QuickstartKarafCxfCamelCxfCodeFirst-4. java.io.IOException: Failed to resolve mvn:io.fabric8.jube.images.fabric8/quickstart-karaf-cxf-camel-cxf-code-first/2.1-SNAPSHOT/zip/image. org.sonatype.aether.collection.DependencyCollectionException: Failed to collect dependencies for io.fabric8.jube.images.fabric8:quickstart-karaf-cxf-camel-cxf-code-first:zip:image:2.1-SNAPSHOT (runtime)
java.io.IOException: Failed to resolve mvn:io.fabric8.jube.images.fabric8/quickstart-karaf-cxf-camel-cxf-code-first/2.1-SNAPSHOT/zip/image. org.sonatype.aether.collection.DependencyCollectionException: Failed to collect dependencies for io.fabric8.jube.images.fabric8:quickstart-karaf-cxf-camel-cxf-code-first:zip:image:2.1-SNAPSHOT (runtime)
    at io.hawt.aether.OpenMavenURL.getInputStream(OpenMavenURL.java:99)
    at io.fabric8.jube.process.service.ProcessManagerService$2.downloadContent(ProcessManagerService.java:339)
    at io.fabric8.jube.process.service.ProcessManagerService$1.install(ProcessManagerService.java:174)
    at io.fabric8.jube.process.service.ProcessManagerService.installViaScript(ProcessManagerService.java:324)
    at io.fabric8.jube.process.service.ProcessManagerService.install(ProcessManagerService.java:189)
    at io.fabric8.jube.local.NodeHelper.createContainer(NodeHelper.java:247)
    at io.fabric8.jube.local.NodeHelper$1.call(NodeHelper.java:171)
    at io.fabric8.jube.local.NodeHelper$1.call(NodeHelper.java:168)
    at io.fabric8.jube.local.NodeHelper.podTransaction(NodeHelper.java:508)
    at io.fabric8.jube.local.NodeHelper.createMissingContainers(NodeHelper.java:168)
    at io.fabric8.jube.apimaster.ApiMasterService$2$1.run(ApiMasterService.java:291)
    at io.fabric8.jube.local.NodeHelper.excludeFromProcessMonitor(NodeHelper.java:526)
    at io.fabric8.jube.apimaster.ApiMasterService$2.run(ApiMasterService.java:298)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)
Caused by: org.sonatype.aether.collection.DependencyCollectionException: Failed to collect dependencies for io.fabric8.jube.images.fabric8:quickstart-karaf-cxf-camel-cxf-code-first:zip:image:2.1-SNAPSHOT (runtime)
    at org.sonatype.aether.impl.internal.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:197)
    at org.sonatype.aether.impl.internal.DefaultRepositorySystem.collectDependencies(DefaultRepositorySystem.java:308)
    at io.hawt.aether.AetherFacade.resolve(AetherFacade.java:316)
    at io.hawt.aether.AetherFacade.resolveFromUrls(AetherFacade.java:293)
    at io.hawt.aether.AetherFacade.resolveUrlAndRepositoriesToFileName(AetherFacade.java:123)
    at sun.reflect.GeneratedMethodAccessor279.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:192)
    at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:174)
    at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:117)
    at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:54)
    at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:235)
    at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
    at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:250)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
    at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:791)
    at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:305)
    at $Proxy74.resolveUrlAndRepositoriesToFileName(Unknown Source)
    at io.hawt.aether.OpenMavenURL.getInputStream(OpenMavenURL.java:94)
    ... 18 more
Caused by: org.sonatype.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for io.fabric8.jube.images.fabric8:quickstart-karaf-cxf-camel-cxf-code-first:zip:image:2.1-SNAPSHOT
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:331)
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:186)
    at org.sonatype.aether.impl.internal.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:191)
    ... 37 more
Caused by: org.apache.maven.model.resolution.UnresolvableModelException: Could not find artifact io.fabric8.quickstarts:project:pom:2.1-20141125.114310-1 in jboss-public (http://repository.jboss.org/nexus/content/groups/public)
    at org.apache.maven.repository.internal.DefaultModelResolver.resolveModel(DefaultModelResolver.java:126)
    at org.apache.maven.model.building.DefaultModelBuilder.readParentExternally(DefaultModelBuilder.java:813)
    at org.apache.maven.model.building.DefaultModelBuilder.readParent(DefaultModelBuilder.java:664)
    at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:310)
    at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:232)
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:322)
    ... 39 more
Caused by: org.sonatype.aether.resolution.ArtifactResolutionException: Could not find artifact io.fabric8.quickstarts:project:pom:2.1-20141125.114310-1 in jboss-public (http://repository.jboss.org/nexus/content/groups/public)
    at org.sonatype.aether.impl.internal.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:538)
    at org.sonatype.aether.impl.internal.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:216)
    at org.sonatype.aether.impl.internal.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:193)
    at org.apache.maven.repository.internal.DefaultModelResolver.resolveModel(DefaultModelResolver.java:122)
    ... 44 more
Caused by: org.sonatype.aether.transfer.ArtifactNotFoundException: Could not find artifact io.fabric8.quickstarts:project:pom:2.1-20141125.114310-1 in jboss-public (http://repository.jboss.org/nexus/content/groups/public)
    at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$4.wrap(WagonRepositoryConnector.java:947)
    at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$4.wrap(WagonRepositoryConnector.java:941)
    at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$GetTask.run(WagonRepositoryConnector.java:669)
    at org.sonatype.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:60)
    ... 3 more

If I change the version to non-SNAPSHOT one, say, 2.10 then it works perfectly, seems jube just skip the SNAPSHOT version

expose the kubernetes service by default like kubernetes does so folks can bind easily to the REST API

Updating env.sh / env.bat should set the ports in the top of the file

... so they are available when you use the in the same file later, which we do for Karaf, eg see screenshot. Notice how we use the SSH_PORT in the KARAF_OPTS but the port is configured later. It should be configured first.

Move the example images to itests as they are for integration testing

If you want to build examples, then build apps, and put them in the fabric8 quickstart project instead.

create an image zip for jetty for fabric8/jetty-8

support annotations on services/pods/RCs

Building jube image - JOLOKIA is added 2 times to ports.properties

In the examples such as cxf-cdi, then inside the target/ -image.zip file, there is a ports.properties file which has jolokia listed 2 times

#
#  Copyright 2005-2014 Red Hat, Inc.
#
#  Red Hat licenses this file to you under the Apache License, version
#  2.0 (the "License"); you may not use this file except in compliance
#  with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
#  Unless required by applicable law or agreed to in writing, software
#  distributed under the License is distributed on an "AS IS" BASIS,
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
#  implied.  See the License for the specific language governing
#  permissions and limitations under the License.
#

JOLOKIA = 8778

JOLOKIA = 8778

the assemblyDescriptor specified for jube-maven-plugin can't override an exist file in docker.from

if there's a file exist in docker.from image, there's no way to override it in the generated image zip even we specify it in assemblyDescriptor.

For example we can't override the etc/users.properties when build quickstart docker image based on the karaf docker image

Jube support running windows processes

In continuation of #128 then the processes started needs to be separate processes, where as the code in the current process manager rely on unix being able to start processes in the background from bash using nohub. Where as this is harder to do in windows.

So the process-manager code may need to be tinkered in jube to support windows being special.

Should we return the deleted pod ID when deleting a pod?

Noticed that deleting pods was broken in hawtio, at least when you try and delete a whole bunch at once. I wonder if returning the pod ID is getting ng-resource confused or something:

 [Kubernetes] deleting:  fabric8MQController-2
loggingInit.js:57 Error: RangeError: Maximum call stack size exceeded {message: "RangeError: Maximum call stack size exceeded", stack: (...)}
loggingInit.js:57 Stack trace:  <div class="log-stack-trace">
<p>RangeError: Maximum call stack size exceeded</p>
<p>    at new Number (native)</p>
<p>    at Number.toString (native)</p>
<p>    at isArray (http://minimi:8282/hawtio/lib/angular.js:417:19)</p>
<p>    at copy (http://minimi:8282/hawtio/lib/angular.js:603:11)</p>
<p>    at copy (http://minimi:8282/hawtio/lib/angular.js:624:28)</p>
<p>    at copy (http://minimi:8282/hawtio/lib/angular.js:608:23)</p>
<p>    at copy (http://minimi:8282/hawtio/lib/angular.js:624:28)</p>
<p>    at copy (http://minimi:8282/hawtio/lib/angular.js:608:23)</p>
<p>    at copy (http://minimi:8282/hawtio/lib/angular.js:624:28)</p>
<p>    at copy (http://minimi:8282/hawtio/lib/angular.js:608:23)</p>
</div>

It's all working fine on the replicationControllers/services pages, so I wonder if we maybe don't need to return anything other than an empty response/HTTP 204?

Fix deleting container to dot directory on windows

When a container is deleted then its saved as dotName. That rename process fails on windows, as likely there is a file handle still open somewhere in the code.

restarting the local node can detect the pids of running proceses; but cannot re-create the Pods for those processes (we need to write them somewhere - in ZK?

support the apimaster REST API as a service for easier wiring into containers

e.g. see http://storage5.static.itmages.com/i/14/1107/h_1415363110_6436041_4d0aed5a07.png

so binding to "kubernetes-ro" for read only port 80 stuff or "kubernetes" for 443 stuff. The actual port numbers don't matter so much though; its more containers can then bind to these services via their names

check if the environment variables are properly defined and log a warning (together with the export statements that should be added to a shell)

see http://fabric8.io/v2/getStartedJube.html#setting-environment-variables

Display jube version when starting up jube

so end users more easily can see what version it is.

Maybe we should add an ascii art jube logo or something to print at the end, and a nice message about where the web console is, etc.