No clear upgrade path

[fabric8io]

No clear upgrade path

[magick93]

@jstrachan - can you share you wisdom to help us with this issue?

[iocanel]

The Fabric8 pipeline library is now using https://github.com/jenkinsci/kubernetes-plugin for creating build pods. And it is strongly recommended to switch to that (since its more actively maintained).

In an effort to align the old DSL provided by https://github.com/jenkinsci/kubernetes-pipeline-plugin (which is what you used in the example) with the https://github.com/jenkinsci/kubernetes-plugin, the old syntax has been slightly changed.

So it should look something like:

kubernetes.pod('buildpod')
       .withNewContainer()
          .withImage('<ip address>:80/shiftwork/jhipster-build')
          .withPrivileged(true)
          .withHostPathMount('/var/run/docker.sock','/var/run/docker.sock')
          .withEnvVar('DOCKER_CONFIG','/home/jenkins/.docker/')
          .withSecret('jenkins-docker-cfg','/home/jenkins/.docker')
          .withSecret('jenkins-maven-settings','/root/.m2')
          .withServiceAccount('jenkins')
          .inside {

[magick93]

With the addition of .withNewContainer() we can move on, but then it seems that all of these methods are now also no longer supported:

.withHostPathMount('/var/run/docker.sock','/var/run/docker.sock')
          .withEnvVar('DOCKER_CONFIG','/home/jenkins/.docker/') //incorrect
          .withSecret('jenkins-docker-cfg','/home/jenkins/.docker') //incorrect
          .withSecret('jenkins-maven-settings','/root/.m2') //incorrect
          .withServiceAccount('jenkins') //incorrect

[magick93]

Furthermore, the example on https://github.com/jenkinsci/kubernetes-pipeline-plugin/blob/master/kubernetes-steps/readme.md:

Example

kubernetes.pod('buildpod') .withNewContainer().withName(<name1>).withImage(<image1>) .withEnv(<key1>,<value1>)

Our Code

This is our code - note the withEnv():
kubernetes.pod('buildpod') .withNewContainer() .withImage('jhipster/jhipster') .withPrivileged(true) //.withHostPathMount('/var/run/docker.sock','/var/run/docker.sock') .withEnv('DOCKER_CONFIG','/home/jenkins/.docker/') .withSecret('jenkins-docker-cfg','/home/jenkins/.docker') .withSecret('jenkins-maven-settings','/root/.m2') .withServiceAccount('jenkins') .inside {

Error

Yet we again get an error:

hudson.remoting.ProxyException: groovy.lang.MissingMethodException: No signature of method: static io.fabric8.kubernetes.pipeline.Kubernetes.withEnv() is applicable for argument types: (java.lang.String, java.lang.String) values: [DOCKER_CONFIG, /home/jenkins/.docker/] Possible solutions: with(groovy.lang.Closure) at groovy.lang.MetaClassImpl.invokeStaticMissingMethod(MetaClassImpl.java:1503) at groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1489) at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:897) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:168) at io.fabric8.kubernetes.pipeline.Kubernetes$Container.methodMissing(Kubernetes.groovy) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaClassImpl.invokeMissingMethod(MetaClassImpl.java:941) at groovy.lang.MetaClassImpl.invokePropertyOrMissing(MetaClassImpl.java:1264) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1217) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1024) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:812) at io.fabric8.kubernetes.pipeline.Kubernetes$Container.invokeMethod(Kubernetes.groovy) at groovy.lang.GroovyObject$invokeMethod.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:151) at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:21) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:115) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:103) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:149) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:146) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:16) at WorkflowScript.run(WorkflowScript:36) at ___cps.transform___(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor378.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:58) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:154) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:33) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:30) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:30) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:163) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:328) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:80) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:240) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:228) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:63) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Finished: FAILURE

Attempts

We have tried:

.withEnv( containerEnvVar(key: 'DOCKER_CONFIG', value: '/home/jenkins/.docker/') )

.withEnv([  containerEnvVar(key: 'DOCKER_CONFIG', value: '/home/jenkins/.docker/')])

.withEnv(['DOCKER_CONFIG','/home/jenkins/.docker/'])

.withEnv([['DOCKER_CONFIG','/home/jenkins/.docker/']])

.withEnv(	[key: 'DOCKER_CONFIG', value: '/home/jenkins/.docker/'])

List<EnvVar> envVars = container.getEnv();
            envVars.add(new EnvVarBuilder().withName("KUBERNETES_MASTER_URI").withValue("k8s").build());
...
  kubernetes.pod('buildpod')
  .withNewContainer()
  	.withImage('jhipster/jhipster')  	
      .withPrivileged(true)
      .withEnv(envVars)

Question

Is there anyway to get IDE support, such as code suggestions, design time error checking? As you can see I'm using trial and error to try to figure this out. A small script like this should take minutes, not days.

[iocanel]

The safest way to avoid this is to switch to the syntax used by the 'https://github.com/jenkinsci/kubernetes-plugin'

This plugin, not only is more actively developed, has a vibrant community etc, it also supports the pipeline script generator that will help you develop your pipeline in a wizard-like manner if you prefer.

The parts 'https://github.com/jenkinsci/kubernetes-pipeline-plugin' that you are using are either going to be depracated or maybe moved to 'https://github.com/jenkinsci/kubernetes-plugin' but untill then, I strongly advise you to to use 'https://github.com/jenkinsci/kubernetes-plugin'.

Now, if for any reason you still want to use that, then check the syntax in the groovy DSL

[magick93]

Im fine with migrating, however right now we have a great deal of urgency. We have had been unable to deploy our apps for 6 weeks. And really just need to get things working again before looking at new technology.

The Groovy DSL is a lot easier to read, however, there seems to be a of inconsistency between the documentation, the source code, and what actually works.

Setting Environment Variables:

Documented: .withEnv(,)
Source: https://github.com/jenkinsci/kubernetes-pipeline-plugin/blob/a10f071084c2b09f1d6a615de74aedbd8fe3a000/kubernetes-steps/src/main/resources/io/fabric8/kubernetes/pipeline/Kubernetes.groovy#L95
What actually works: .withEnvar(key, value)

In this case, what works is the same as the source but not the example.

Setting Secret

Documented: .withSecret(, )
Source: https://github.com/jenkinsci/kubernetes-pipeline-plugin/blob/a10f071084c2b09f1d6a615de74aedbd8fe3a000/kubernetes-steps/src/main/resources/io/fabric8/kubernetes/pipeline/Kubernetes.groovy#L100
What actually works: still dont know, but none of the above

Setting ServiceAccount

Similar to Secret

[iocanel]

Make sure you define secret and serviceAccount on pod level and not on container level.

[magick93]

Ok, I tried the following:

  kubernetes.pod('buildpod')
  //.withHostPathMount('/var/run/docker.sock','/var/run/docker.sock')
  .withSecret('jenkins-docker-cfg','/home/jenkins/.docker')
  .withSecret('jenkins-maven-settings','/root/.m2')
  .withServiceAccount('jenkins')
  .withNewContainer()
  	.withImage('jhipster/jhipster')  	
      .withPrivileged(true)
      .withEnvar('DOCKER_CONFIG',  '/home/jenkins/.docker/'	)      
      .inside {

And it resulted in the following error:

Pipeline] End of Pipeline java.io.NotSerializableException: org.csanchez.jenkins.plugins.kubernetes.PodEnvVar at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:860) at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:569) at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032) at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988) at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854) at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:569) at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032) at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988) at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854) at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032) at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988) at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967) at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967) at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854) at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65) at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56) at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50) at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344) at java.util.HashMap.internalWriteEntries(HashMap.java:1785) at java.util.HashMap.writeObject(HashMap.java:1362) at sun.reflect.GeneratedMethodAccessor287.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271) at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976) at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854) at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032) at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988) at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854) at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58) at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111) at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.writeObject(RiverWriter.java:132) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:452) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:427) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgramIfPossible(CpsThreadGroup.java:415) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:360) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:80) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:240) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:228) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:63) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: an exception which occurred: in field envVars in field containers in field delegate in field closures in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@61701b57 Finished: FAILURE