Jenkinsfile with declarative syntax doesn't connect to docker-engine at $DOCKER_HOST

[vfarcic]

Jenkinsfile with declarative syntax doesn't connect to docker-engine at $DOCKER_HOST

[vfarcic]

That's strange...

DOCKER_HOST=tcp://192.168.99.100:2375 docker service update --image localhost:5000/myservice:2.12 myservice

... and ...

DOCKER_HOST=tcp://192.168.99.100:2375 && docker service update --image localhost:5000/myservice:2.12 myservice

... should produce the same result.

What is your OS? Maybe the first command does not work in a specific OS.

[uprix]

Hi,

I'm working along your book 'DevOps 2.1 Toolkit'.
The issue arises in a setup host='swarm-test-1' and service='jenkins-agent' as described on p.112 ff.
So I mean the OS is boot2docker, right?

The interesting thing is,
when I try to investigate the problem and ssh into swarm-test-1 and then 'docker exec' into the jenkins_agent-service container. Then the command "DOCKER_HOST=tcp://192.168.99.100:2375 docker service update --image localhost:5000/myservice:2.12 myservice" on the commandline leads to the same error.

My understanding is that the docker-client doesn't have enough information with only 'DOCKER_HOST' to connect to the docker-server on 'swarm-1' (which is 192.168.99.100).

Next I 'export' all envvars from 'docker-machine env swarm-1' within the jenkins-agent CLI. Then the service update command is correctly working.
This makes me believing that everything else in the whole szenario is setup correctly.

That's why I wrote
"Trying this command inside the jenkins-agent container brought me to a working solution:
Also set DOCKER_MACHINE_NAME and DOCKER_CERT_PATH."

The only thing I didn't manage yet is to adapt the pipeline script in a way which makes the service update command working out of jenkins. I think I'm too unfamiliar with shell-programming.
Do you have a solution?

Thanx again,
Stephan

Addendum:
output of 'docker version' within 'jenkins-agent' :

Client:
Version: 17.09.0-ce
API version: 1.32
Go version: go1.8.3
Git commit: afdb6d4
Built: Tue Sep 26 22:39:28 2017
OS/Arch: linux/amd64

Server:
Version: 17.10.0-ce
API version: 1.33 (minimum version 1.12)
Go version: go1.8.3
Git commit: f4ffd25
Built: Tue Oct 17 19:05:23 2017
OS/Arch: linux/amd64
Experimental: false

[vfarcic]

Let me re-run the commands from "Automating Continuous Deployment Flow With Jenkins". If they fail, I'll fix it. If they don't, there a problem with your environment. If that's the case, maybe we can have a screen-sharing session and go through it together.

What is your timezone? I can re-run the commands tonight GMT.

[vfarcic]

Unfortunately, internet connection in my hotel is horrible and I could not re-run the commands. I'll do it once I get back home during the weekend.

You're correct that DOCKER_CERT_PATH is required besides DOCKER_HOST. However, both are in the job definition.

stage("Prod-like") {
  withEnv([
    "DOCKER_TLS_VERIFY=1",
    "DOCKER_HOST=tcp://${env.PROD_LIKE_IP}:2376",
    "DOCKER_CERT_PATH=/machines/${env.PROD_LIKE_NAME}"
  ]) {
    sh "docker service update \
      --image localhost:5000/go-demo:2.${env.BUILD_NUMBER} \
      go-demo"
  }
  withEnv(["HOST_IP=localhost"]) {
    try {
      for (i = 0; i <10; i++) {
        sh "docker-compose run --rm production"
      }
    } catch (e) {
      sh "docker service update --rollback go-demo"
    }
  }
}

Can you please confirm that is the pipeline you're using? If it's not, you might be reading an older edition of the book. If that's the case, I'll send you the most recent one.

[uprix]

No I do not use the quoted code.
Although it's the Same Code as in my book :-)

I intentionally switched to the declarative Pipeline Syntax as you have it in this jenkinsfile here in master I opened the issue for.

My code is (see https://github.com/vfarcic/go-demo/blob/master/Jenkinsfile):

...
    stage("Production") {
      steps {
        sh "DOCKER_HOST=tcp://${env.PROD_IP}:2375 docker service update --image localhost:5000/go-demo:2.${env.BUILD_NUMBER} go-demo_main"
      }
   }
...

Anyway could you send me a link to a declarative Pipeline Version if you have one?

Thanx! I appreciate talking to you!

Stephan

PS: normally I am UTC+1

[vfarcic]

Unless I made a mistake, that Jenkinsfile is not used in Toolkit 2.1 (only in Toolkit 2.0). Please let me know if I'm wrong.

In 2.0 I used Vagrant with Docker setup without TLS so there's no need to specify certs. If you'd like to reuse that Jenkinsfile in a cluster with TLS, you'd need to add DOCKER_CERT_PATH env. var. It would be something like:

  withEnv([
    "DOCKER_TLS_VERIFY=1",
    "DOCKER_HOST=tcp://${env.PROD_IP}:2376",
    "DOCKER_CERT_PATH=/machines/${env.PROD_NAME}"
  ]) {
    // Place your update command here
  }

It might be worth mentioning that I dropped this approach altogether. My current pipeline for a demo service is now https://github.com/vfarcic/go-demo-2/blob/master/Jenkinsfile.

[uprix]

Hi Viktor,

thanks so much for your attention and help and clarifications - I got it working now.

I think as a newbie to all this magic containerization and CD stuff I was lost in complexity.
I over and over drilled down in debugging from a command line and Jenkins pipeline perspective and compared environments and commands and connectivities in both of them to each other ...

... until I found my own typo: I had DOCKER_HOST=tcp://${env.PROD_IP}:2375 instead of ...:2376.

How stupid from my side huh?

Thanx again for your devotion. Next time I'll think twice!

Bye,
Stephan

[vfarcic]

Don't worry... I'm glad it worked. Feel free to ping me or open an issue if you run into another problem.