Giter Site home page Giter Site logo

jenkinsci / azure-container-agents-plugin Goto Github PK

View Code? Open in Web Editor NEW
19.0 7.0 27.0 496 KB

Azure Container Agents Plugin for Jenkins

Home Page: https://plugins.jenkins.io/azure-container-agents/

License: MIT License

Java 97.39% HTML 2.61%
azure agent aci azure-aci

azure-container-agents-plugin's Introduction

Azure Container Agents Plugin

Important: This plug-in is maintained by the Jenkins community and won’t be supported by Microsoft as of February 29, 2024.

Azure Container Agents Plugin can help you to run a container instance as an agent in Jenkins

How to install

You can install/update this plugin in Jenkins update center (Manage Jenkins -> Manage Plugins, search Azure Container Agents Plugin).

Pre-requirements

Azure Container Instance

Azure Container Instances offers the fastest and simplest way to run a container in Azure, without having to provision any virtual machines and without having to adopt a higher-level service.

Pre-requirements

Configure the plugin

  1. Jenkins -> Manage Jenkins -> Configure System
  2. Press Add a new cloud and choose Azure Container Instance
  3. Specify Cloud Name and it should be unique.
  4. Choose an existing Azure Credential or create a new credential.
  5. Choose Resource Group.

Configure the Container Template

  1. Specify Name and Labels
  2. Set Startup Timeout.
  3. Select Image OS Type, Windows or Linux.
  4. Fill in Docker Image. The default image is jenkins/inbound-agent and you can also use it as base image.
  5. If you use a private registry, you need to specify a credential.
  6. Specify a Command. Now the Command will override the ENTRYPOINT. Arguments. ${rootUrl}, ${secret}, ${instanceIdentity} and ${nodeName} will be replaced with JenkinsUrl, Secret, Instance identity and ComputerNodeName automatically.
  7. Specify the Working Dir. Ensure the user has write permission to this directory.
  8. Add Ports, Environment Variables and Volumes as needed.
  9. Choose a retention strategy. You can get details by clicking the help icon.
  10. Specify CPU Requirement and Memory Requirement, ACI containers costs per second. Find more details in Container Instances pricing.
  11. Decide if the container instance should use a private IP address or not. You can get details by clicking the help icon.

Configure Azure Container Instance via Groovy Script

You can use the sample below in Manage Jenkins -> Script Console. The sample only contains a few arguments. Find all the arguments in the builders package.

import com.microsoft.jenkins.containeragents.builders.*

def myCloud = new AciCloudBuilder()
.withCloudName("mycloud")
.withAzureCredentialsId("<Your Credentials Id>")
.withResourceGroup("myResourceGroup")
.addNewTemplate()
    .withName("mytemplate")
    .withLabel("aci")
    .addNewPort("80")
    .addNewEnvVar("key","value")
.endTemplate()
.build()

Jenkins.get().clouds.add(myCloud)
//inherit template from existing template
import com.microsoft.jenkins.containeragents.builders.*

def baseTemplate = new AciContainerTemplateBuilder()
.withImage("privateImage")
.addNewPort("80")
.addNewEnvVar("key", "value")
.build()

def myCloud = new AciCloudBuilder()
.withCloudName("mycloud")
.withAzureCredentialsId("<Your Credentials Id>")
.withResourceGroup("myResourceGroup")
.addNewTemplateLike(baseTemplate)
    .withName("mytemplate")
    .withLabel("aci")
.endTemplate()
.build()

Jenkins.get().clouds.add(myCloud)

Azure Kubernetes Service

If you were previously using this plugin to integrate with AKS you should use the Kubernetes plugin instead.

azure-container-agents-plugin's People

Contributors

allxiao avatar basil avatar chenkennt avatar darxriggs avatar dependabot[bot] avatar gavinfish avatar jetersen avatar lgdoor avatar microsoftopensource avatar mostafaez avatar msftgits avatar renovate[bot] avatar slide avatar sparsick avatar strangelookingnerd avatar timja avatar timja-bot avatar xuzhang3 avatar zackliu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

azure-container-agents-plugin's Issues

Connect to null failed: null

When i'm clicking on TEST CONNECTION button i'm getting: Connect to null failed: null

The credentials are fine since it recognizes my AKS clusters when i'm putting them.

Add NoDelayProvisionerStrategy to reduce provisioning time of AKS pods

Currently, Jenkins waits for existing pods to complete execution, and only after some time it provisions a new pod on Azure Kubernetes (AKS).
This was fixed for the Kubernetes plugin

Need to add this to Azure container agents plugin for reducing provisioning time.
Also, it'll be helpful if we can provide a flag to choose from

  • wait for existing pods/executors
  • create new pod for every build

Azure contains Agents - Private IP Address

We are using Azure Container Agents Plugin to deploy Jenkins Agents.

I am following instructions here: https://wiki.jenkins.io/display/JENKINS/Azure+Container+Agents+Plugin ,with Jenkins Master in-house.

We allowed Jenkins Master Host IP on the VPN tunnel from our network to Azure.

Azure Container Agents are being initiated when I run the build from Jenkins, however they are unable to communicate back to Jenkins master via our gateway since they are getting public IP's.

How do I make sure Jenkins agents are getting a IP from our address space and communicate back to our Jenkins Master.

Setting up DNS cause container keep failed to start

Jenkins and plugins versions report

I recently use this plugin and tried to deploy aci as jenkin slave, however i keep getting dns timeout due to behaviour of aci which does not inherit dns setting from vnet. When i configure a cloud in Jenkins to include the dns, the aci container keep failing and terminating.

What Operating System are you using (both controller, and any agents involved in the problem)?

Jenkins master is redhat, slave is using jenkins/inbound-agent from docker hub

Reproduction steps

1.set DNS of cloud configuration of jenkins
2. Use that aci cloud as agent to run pipeline

Expected Results

Aci created, failing and terminate

Actual Results

Terminated

Anything else?

No response

Are you interested in contributing a fix?

No response

Container Instances - Use of Private IPS

Hello,
At the moment Jenkins plugin forces us to use public address.
Allowing jnlp ports through public address works.
However we need to connect Jenkins Master by Private IP. We don't want to expose any public address ports to the Jenkins

However it seems that ACI uses it's own network and cannot directly access our Jenkins.
What can we do. it is a pretty critical component not working

Choose Vnet and subnet from a different resource group

What feature do you want to see added?

When using private ip address, you have to specify a vnet and a subnet. Currently, the plugin only support vnet from the same resource group that also the container instance will be used. In some cases, the vnet is set up in a different resource group. Therefore, the name of the resource group should be possible to specify.

This new field could be optional. So the default behaviour is using the same resource group as the container instance will use.

Todo List:

  • Add new optional field Name of resource group
  • Extend the documentation

Upstream changes

No response

ACI agents failed to allocate on ci.jenkins.io

Version report

Jenkins and plugins versions report:

Click to expand details of working configuration
Jenkins: 2.277.4
OS: Linux - 5.4.0-1031-azure
---
ace-editor:1.1
amazon-ecs:1.37
analysis-model-api:10.2.5
ansicolor:1.0.0
ant:1.11
antisamy-markup-formatter:2.1
apache-httpcomponents-client-4-api:4.5.13-1.0
authentication-tokens:1.4
aws-credentials:1.29
aws-java-sdk:1.11.995
azure-commons:1.1.3
azure-container-agents:201.v2afdce22b4cf
azure-credentials:177.v816b81058012
azure-sdk:4.vcb202d9010c1
azure-vm-agents:774.v0cee503baa25
basic-branch-build-strategies:1.3.2
beer:1.3
blueocean-autofavorite:1.2.4
blueocean-bitbucket-pipeline:1.24.7
blueocean-commons:1.24.7
blueocean-config:1.24.7
blueocean-core-js:1.24.7
blueocean-dashboard:1.24.7
blueocean-display-url:2.4.1
blueocean-events:1.24.7
blueocean-git-pipeline:1.24.7
blueocean-github-pipeline:1.24.7
blueocean-i18n:1.24.7
blueocean-jira:1.24.7
blueocean-jwt:1.24.7
blueocean-personalization:1.24.7
blueocean-pipeline-api-impl:1.24.7
blueocean-pipeline-editor:1.24.7
blueocean-pipeline-scm-api:1.24.7
blueocean-rest-impl:1.24.7
blueocean-rest:1.24.7
blueocean-web:1.24.7
blueocean:1.24.7
bootstrap4-api:4.6.0-3
bouncycastle-api:2.20
branch-api:2.6.4
build-timeout:1.20
buildtriggerbadge:2.10
caffeine-api:2.9.1-23.v51c4e2c879c8
checks-api:1.7.0
cloud-stats:0.27
cloudbees-bitbucket-branch-source:2.9.9
cloudbees-folder:6.15
code-coverage-api:1.3.2
command-launcher:1.6
conditional-buildstep:1.4.1
configuration-as-code:1.51
copyartifact:1.46
credentials-binding:1.24
credentials:2.4.1
cvs:2.19
dark-theme:0.0.12
data-tables-api:1.10.23-3
disable-github-multibranch-status:1.2
display-url-api:2.3.5
docker-commons:1.17
docker-workflow:1.26
durable-task:1.36
ec2:1.59
echarts-api:5.1.0-2
embeddable-build-status:2.0.3
extended-read-permission:3.2
external-monitor-job:1.7
favorite:2.3.3
font-awesome-api:5.15.3-2
forensics-api:1.0.0
git-client:3.7.1
git-forensics:1.0.0
git-server:1.9
git:4.7.1
github-api:1.123
github-branch-source:2.10.4
github-checks:1.0.12
github:1.33.1
groovy:2.4
handlebars:3.0.8
handy-uri-templates-2-api:2.1.8-1.0
htmlpublisher:1.25
jackson2-api:2.12.3
jacoco:3.2.0
javadoc:1.6
jaxb:2.3.0.1
jdk-tool:1.5
jenkins-design-language:1.24.7
jira:3.3
jjwt-api:0.11.2-9.c8b45b8bb173
jobConfigHistory:2.27
jquery-detached:1.2.1
jquery3-api:3.6.0-1
jquery:1.12.4-1
jsch:0.1.55.2
junit-attachments:1.6
junit-realtime-test-reporter:0.6
junit:1.49
keyboard-shortcuts-plugin:1.4
kubernetes-client-api:4.13.3-1
kubernetes-credentials:0.9.0
kubernetes:1.29.6
ldap:2.7
lockable-resources:2.10
mailer:1.34
mapdb-api:1.0.9.0
matrix-auth:2.6.7
matrix-project:1.18
maven-plugin:3.10
mercurial:2.15
metrics:4.0.2.7
momentjs:1.1.1
node-iterator-api:1.5.0
okhttp-api:3.14.9
pam-auth:1.6
parallel-test-executor:1.13
parameterized-trigger:2.40
pipeline-build-step:2.13
pipeline-github-lib:1.0
pipeline-githubnotify-step:1.0.5
pipeline-graph-analysis:1.10
pipeline-input-step:2.12
pipeline-milestone-step:1.3.2
pipeline-model-api:1.8.4
pipeline-model-definition:1.8.4
pipeline-model-extensions:1.8.4
pipeline-rest-api:2.19
pipeline-stage-step:2.5
pipeline-stage-tags-metadata:1.8.4
pipeline-stage-view:2.19
pipeline-utility-steps:2.8.0
plain-credentials:1.7
plugin-util-api:2.2.0
popper-api:1.16.1-2
pubsub-light:1.14
run-condition:1.5
scm-api:2.6.4
script-security:1.77
snakeyaml-api:1.27.0
sse-gateway:1.24
ssh-agent:1.22
ssh-credentials:1.18.1
ssh-slaves:1.31.5
structs:1.23
subversion:2.14.2
support-core:2.74
theme-manager:0.6
throttle-concurrents:2.2
timestamper:1.13
token-macro:2.15
toolenv:1.2
translation:1.16
trilead-api:1.0.13
variant:1.4
warnings-ng:9.1.0
windows-azure-storage:355.v4da08e72a251
windows-slaves:1.8
workflow-aggregator:2.6
workflow-api:2.42
workflow-basic-steps:2.23
workflow-cps-global-lib:2.19
workflow-cps:2.92
workflow-durable-task-step:2.39
workflow-job:2.40
workflow-multibranch:2.24
workflow-scm-step:2.12
workflow-step-api:2.23
workflow-support:3.8
Click to expand differences for failing configuration
azure-sdk:12.vc102aedd3c66
azure-credentials:182.v3ccd4a755864
azure-vm-agents:780.v50d067d02f76
windows-azure-storage:355.v4da08e72a251
azure-container-agents:207.v3ad9931bf69e
  • What Operating System are you using (both controller, and any agents involved in the problem)?
Controller running Ubuntu 18.04 Linux on Azure.  ACI agents running in Docker

Reproduction steps

  1. Upgrade plugins from current installed set to latest releases of azure-sdk, azure-credentials, azure-vm-agents, azure-container-agents, and windows-azure-storage
  2. Note that Azure container agents fail to allocate (maven and maven-11 agents). System log reports Cannot provision: template for label maven is not available now, because it failed to provision last time. An exception was visible in the cloud-stats page that showed:
java.lang.Exception
	at com.microsoft.jenkins.containeragents.aci.AciService.createDeployment(AciService.java:143)
	at com.microsoft.jenkins.containeragents.aci.AciContainerTemplate.provisionAgents(AciContainerTemplate.java:128)
	at com.microsoft.jenkins.containeragents.aci.AciCloud.lambda$provision$1(AciCloud.java:109)
Caused: java.lang.Exception
	at com.microsoft.jenkins.containeragents.aci.AciCloud.lambda$provision$1(AciCloud.java:136)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:80)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
  1. Resolve issue by reverting to previously installed versions of those plugins

See the detailed notes in the incident log.

Results

Expected result:

ACI agents expected to be allocated and destroyed on demand.

Actual result:

ACI agents not allocated, though there was a period where many agents were allocated (more than 250) but none of them connected to the Jenkins controller.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

github-actions
.github/workflows/cd.yaml
  • jenkins-infra/github-reusable-workflows v1
.github/workflows/jenkins-security-scan.yml
  • jenkins-infra/jenkins-security-scan v2
maven
.mvn/extensions.xml
  • io.jenkins.tools.incrementals:git-changelist-maven-extension 1.8
pom.xml
  • org.jenkins-ci.plugins:plugin 4.85
  • io.jenkins.plugins:azure-sdk 174.va_89c1df897d2
  • org.jenkins-ci.plugins:cloud-stats 336.v788e4055508b_
  • org.jenkins-ci.plugins:docker-commons 439.va_3cb_0a_6a_fb_29
  • org.jenkins-ci.plugins:azure-credentials 312.v0f3973cd1e59
  • org.jenkins-ci.plugins:windows-azure-storage 378.vb2cb9797e1e5
  • io.jenkins.tools.bom:bom-2.387.x 2543.vfb_1a_5fb_9496d
  • org.apache.maven.plugins:maven-checkstyle-plugin 3.4.0
  • com.puppycrawl.tools:checkstyle 10.17.0

  • Check this box to trigger a request for Renovate to run again on this repository

Container instances never end up running

I'm trying out the ACI based cloud but am encountering an issue with both Windows and Linux based containers. While I do end up seeing that the plugin attempts to create instances, my tasks never seem to run. Instead, the instances linger for a while and then disappear, having done nothing.

Feedback on the containers both in the portal and via az seems sparse:

image

Log of container 'linux-w4gcn' in container group 'linux-w4gcn' is not available yet. Please check container 'InstanceView' for more info
rmation or retry later.

Looking at the Jenkins logs, I do see one exception per type if I leave things to run long enough:

WARNING: AciCloud: Provision agent linux-kbt38 failed: ACI container terminated
[pool-266-thread-1] INFO com.microsoft.aad.adal4j.AuthenticationAuthority - [Correlation ID: REDACTED] Instance discovery was successful
Dec 06, 2017 2:15:57 PM okhttp3.internal.platform.Platform log
INFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?
Dec 06, 2017 2:15:57 PM hudson.slaves.NodeProvisioner$2 run
WARNING: Unexpected exception encountered while provisioning agent Linux
java.lang.Exception: java.lang.IllegalStateException: ACI container terminated
        at com.microsoft.jenkins.containeragents.aci.AciCloud$1.call(AciCloud.java:131)
        at com.microsoft.jenkins.containeragents.aci.AciCloud$1.call(AciCloud.java:80)
        at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: ACI container terminated
        at com.microsoft.jenkins.containeragents.aci.AciCloud.waitToOnline(AciCloud.java:201)
        at com.microsoft.jenkins.containeragents.aci.AciCloud.access$200(AciCloud.java:42)
        at com.microsoft.jenkins.containeragents.aci.AciCloud$1.call(AciCloud.java:107)
        ... 6 more
WARNING: Unexpected exception encountered while provisioning agent NET Framework
java.lang.Exception: java.lang.Exception: Deployment timeout
        at com.microsoft.jenkins.containeragents.aci.AciCloud$1.call(AciCloud.java:131)
        at com.microsoft.jenkins.containeragents.aci.AciCloud$1.call(AciCloud.java:80)
        at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.Exception: Deployment timeout
        at com.microsoft.jenkins.containeragents.aci.AciService.createDeployment(AciService.java:122)
        at com.microsoft.jenkins.containeragents.aci.AciContainerTemplate.provisionAgents(AciContainerTemplate.java:107)
        at com.microsoft.jenkins.containeragents.aci.AciCloud$1.call(AciCloud.java:104)
        ... 6 more

Nothing fancy about my templates, just trying the defaults for now to make sure everything works:

image

image

ACI documentation suggestion: check deployment view of resource group

For those new to ACI like me, this could be a timesaver when the cloud nodes fail to spin up properly. Finally found detailed error info on this I'd misconfigured. You could consider adding this as a hint in the Azure Container Instance section of the readme. Something like:

To view Azure-side issues starting up an ACI cloud container, lookup the resource group they're deployed to in the Azure portal and check the Deployments section. It will show the attempts to start containers and any errors that occurred.
.

Must be unique error when trying to define a pod template that mounts an azure file share.

Error in provisioning; slave=com.microsoft.jenkins.containeragents.KubernetesAgent[jenkins-agent-njx6q], template=com.microsoft.jenkins.containeragents.PodTemplate@1bf88ccc: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://cp-aks-cp-aks-5e8113-cd6613fc.hcp.eastus.azmk8s.io/api/v1/namespaces/default/pods. Message: Pod "jenkins-agent-njx6q" is invalid: spec.containers[0].volumeMounts[1].mountPath: Invalid value: "/mnt/jenkins": must be unique. Received status: Status(apiVersion=v1, code=422, details=StatusDetails(causes=[StatusCause(field=spec.containers[0].volumeMounts[1].mountPath, message=Invalid value: "/mnt/jenkins": must be unique, reason=FieldValueInvalid, additionalProperties={})], group=null, kind=Pod, name=jenkins-agent-njx6q, retryAfterSeconds=null, additionalProperties={}), kind=Status, message=Pod "jenkins-agent-njx6q" is invalid: spec.containers[0].volumeMounts[1].mountPath: Invalid value: "/mnt/jenkins": must be unique, metadata=ListMeta(resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid, status=Failure, additionalProperties={}).

Azure Container Registry and ACI not working

I created an image based on jenkins/ssh-slave and pushed it to my private Container Registry in Azure.

When I now configure the ACI container template it isn't working correctly, the container is always restarted.
Running the image locally on my machine is working.

screen shot 2018-08-28 at 23 48 32

Or is there an issue with the definition of the private docker registry?

SSH launch method is not working

I jusr tried and SSH launch method is not working. I get following log:

2020-09-03 13:08:03.789+0000 [id=91]	INFO	c.m.j.c.aci.AciCloud$1#call: Add ACI node: aci-t89l3
2020-09-03 13:08:03.938+0000 [id=91]	INFO	c.m.j.c.a.AciCleanTask$DeploymentRegistrar#registerDeployment: AzureAciCleanUpTask: registerDeployment: Registering deployment aci-bscm7tx9 in myJenkinsAgentGroup
2020-09-03 13:08:04.648+0000 [id=91]	INFO	o.internal.platform.Platform#log: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?
2020-09-03 13:08:06.085+0000 [id=91]	INFO	c.m.j.c.aci.AciService#createDeployment: Waiting for deployment aci-bscm7tx9
2020-09-03 13:09:27.096+0000 [id=91]	INFO	c.m.j.c.aci.AciService#createDeployment: Deployment aci-bscm7tx9 succeed
2020-09-03 13:09:27.628+0000 [id=91]	INFO	o.internal.platform.Platform#log: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?
2020-09-03 13:09:29.257+0000 [id=106]	INFO	c.m.j.c.remote.SSHLauncher#launch: SSHLauncher: Start to connect node aci-t89l3 : 51.145.191.189 via SSH
2020-09-03 13:09:30.676+0000 [id=106]	INFO	c.m.j.c.remote.SSHLauncher#launch: SSHLauncher: Copy slave.jar to remote host successfully
2020-09-03 13:09:31.318+0000 [id=106]	INFO	c.m.j.c.remote.SSHLauncher#launch: SSHLauncher: launching agent failed
2020-09-03 13:09:31.338+0000 [id=91]	WARNING	c.m.j.c.aci.AciCloud$1#call: AciCloud: Provision agent aci-t89l3 failed: java.lang.InterruptedException: java.io.EOFException: unexpected stream termination
2020-09-03 13:09:32.019+0000 [id=113]	INFO	o.internal.platform.Platform#log: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?
2020-09-03 13:09:33.577+0000 [id=106]	INFO	c.m.j.c.aci.AciService#deleteAciContainerGroup: Delete ACI Container Group: aci-t89l3 successfully
2020-09-03 13:09:33.651+0000 [id=48]	WARNING	hudson.slaves.NodeProvisioner#lambda$update$6: Unexpected exception encountered while provisioning agent aci
java.lang.InterruptedException: java.io.EOFException: unexpected stream termination
	at com.microsoft.jenkins.containeragents.remote.SSHLauncher.launch(SSHLauncher.java:121)
	at hudson.slaves.SlaveComputer.lambda$_connect$0(SlaveComputer.java:296)
Caused: java.util.concurrent.ExecutionException
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at com.microsoft.jenkins.containeragents.aci.AciCloud$1.call(AciCloud.java:115)
Caused: java.lang.Exception
	at com.microsoft.jenkins.containeragents.aci.AciCloud$1.call(AciCloud.java:140)
	at com.microsoft.jenkins.containeragents.aci.AciCloud$1.call(AciCloud.java:80)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2020-09-03 13:09:33.653+0000 [id=48]	WARNING	c.m.j.c.aci.AciCloud#canProvision: Cannot provision: template for label aci is not available now, because it failed to provision last time.
2020-09-03 13:09:43.617+0000 [id=43]	INFO	c.m.j.c.aci.AciCloud#provision: Start ACI container for label aci workLoad 1
2020-09-03 13:09:43.617+0000 [id=43]	INFO	c.m.j.c.aci.AciCloud#provision: Using ACI Container template: aci
2020-09-03 13:09:43.617+0000 [id=43]	INFO	h.s.NodeProvisioner$StandardStrategyImpl#apply: Started provisioning aci from Aci with 1 executors. Remaining excess workload: 0
2020-09-03 13:09:43.619+0000 [id=91]	INFO	c.m.j.c.aci.AciCloud$1#call: Add ACI node: aci-t41sn

Looks like plugin uploads slave.jar successfully but then fail when calling computer.setChannel in SSHLauncher in line 103.

Running brand new LTS installation:

  • Jenkins: 2.235.5
  • All plugins at latest version.

I will try 2.60.3 or with #53 at some point later.

Sidenote:
I noticed that code has pattern of catching exceptions and rethrowing them without passing original exception as cause or printing the original exception. This makes debugging really hard.

Use aci plugin in jenkins on top aks

Hi ,
I am using jenkins on top of aks .

I try to use the aci has a slave/pod that doing job and delete in the end .

Jenkins made to create the aci but can't login/run jnlp on that .

My only open port in my cluster is https for security .

I make to create a server with vcloud plugin and connect via jnlp/ssh .

Why it's not working to me has a basic jnlp ( I am using jnlp4) ? There is a workaround?

Thanks

How to use Fileshare as rootFS

Describe your use-case which is not covered by existing documentation.

I want to use Azure fileshare as rootFS.
I have created a fileshare and I am able to add it as volume.
`volumes:

  • mountPath: "/mnt/j"
    shareName: "- moacishare"
    credentialsId: azurestorageaccessforacifileshare`
    However, when I try to use this mounted volume as rootFS, I get errors.
    Mainly because container runs as jenkins user and mount has access and ownership for root user.

Reference any relevant documentation, other materials or issues/pull requests that can be used for inspiration.

https://learn.microsoft.com/en-us/azure/container-instances/container-instances-volume-azure-files
https://github.com/jenkinsci/docker-ssh-agent

Are you interested in contributing to the documentation?

No response

Add support for assigning Managed Identity to the container

What feature do you want to see added?

It seems that currently it is not possible to let this plugin assign a managed identity to the created containers, is that correct?

We need to access certain Azure resources from our builds. By far the most convenient and secure would be to use managed identities which are assigned to the created containers. That way we don't have to inject any hardcoded credentials into to container, which can always end up leaking.

Azure container instances supports managed identities out of the box, so this should be rather straightforward:
https://learn.microsoft.com/en-us/azure/container-instances/container-instances-managed-identity

Upstream changes

No response

Are you interested in contributing this feature?

If this project is still active and I would receive help in getting this into a new release quickly, I would be willing to give this implementation a try.

How to use with Declarative Pipeline and docker agent

Hello,

I'm trying to make my declarative pipeline run on azure container instances without employing something like docker in docker. Is there a way to enable something so Jenkinsfile like this could work out of the box on Azure Container Instances?

pipeline {
    agent {
        docker { image 'node:7-alpine' }
    }
    stages {
        stage('Test') {
            steps {
                sh 'node --version'
            }
        }
    }
}

Here's documentation: https://jenkins.io/doc/book/pipeline/docker/

EDIT: Actually it seems docker in docker doesn't even work on Container Instances

Cannot pull container from my Azure Container Registry

I'm struggling to create an ACI Container Template that pulls Docker Image from my Azure Container Registry and hoping someone can help.

As background, I have the following setup:

  1. Setup a Jenkins server with a project that executes on a container node (jenkinsci/ssh-slave) based on these instructions: Using Azure container instances as Jenkins build agents
  2. Cloned the jenkinsci/ssh-slave from https://github.com/jenkinsci/docker-ssh-slave.git and rebuilt with the tag docker-ssh-slave-example
  3. Created an Azure Container Registry and pushed a copy of the docker-ssh-slave-example container to that registry based on Deploy an Azure container registry and push a container image
  4. In Jenkins config, I added an Aci Container Template with a Docker Image of "<my registry>.azurecr.io/docker-ssh-slave-example"

I'm stuck on how to create the "Private Registry Credential". It's not clear to me what "Kind" of credential I need to create so Jenkins can obtain "<my registry>.azurecr.io/docker-ssh-slave-example"

The screenshots in #29 indicate there is a "Kind" of credential called "Azure Container Registry". When I attempt to create the "Registry credentials", there is no option by this name (screenshot below).

Please help me understand what needs to be setup to accomplish this. How does the Jenkins job authenticate with the Azure Container Registry? Does it employ a method described in Authenticate with a private Docker container registry or does it rely on a different technique?

image

credentials drop down not populated for volumes in ACI template

Jenkins and plugins versions report

Environment
Jenkins: 2.361.2.1
OS: Linux - 3.10.0-1160.71.1.el7.x86_64
---
Exclusion:0.15
ace-editor:1.1
analysis-model-api:10.16.1
ansicolor:1.0.2
ant:475.vf34069fef73c
antisamy-markup-formatter:2.7
apache-httpcomponents-client-4-api:4.5.13-138.v4e7d9a_7b_a_e61
async-http-client:1.7.24.3
audit-trail:3.11
authentication-tokens:1.4
aws-credentials:191.vcb_f183ce58b_9
aws-java-sdk:1.12.287-357.vf82d85a_6eefd
aws-java-sdk-cloudformation:1.12.287-357.vf82d85a_6eefd
aws-java-sdk-codebuild:1.12.287-357.vf82d85a_6eefd
aws-java-sdk-ec2:1.12.287-357.vf82d85a_6eefd
aws-java-sdk-ecr:1.12.287-357.vf82d85a_6eefd
aws-java-sdk-ecs:1.12.287-357.vf82d85a_6eefd
aws-java-sdk-efs:1.12.287-357.vf82d85a_6eefd
aws-java-sdk-elasticbeanstalk:1.12.287-357.vf82d85a_6eefd
aws-java-sdk-iam:1.12.287-357.vf82d85a_6eefd
aws-java-sdk-logs:1.12.287-357.vf82d85a_6eefd
aws-java-sdk-minimal:1.12.287-357.vf82d85a_6eefd
aws-java-sdk-sns:1.12.287-357.vf82d85a_6eefd
aws-java-sdk-sqs:1.12.287-357.vf82d85a_6eefd
aws-java-sdk-ssm:1.12.287-357.vf82d85a_6eefd
azure-commons:1.1.3
azure-container-agents:241.va_780fa_dc374a_
azure-credentials:242.vb_f9c4fa_6b_2b_6
azure-sdk:118.v43f74dd9ca_dc
blueocean-commons:1.25.8
blueocean-config:1.25.8
blueocean-core-js:1.25.8
blueocean-dashboard:1.25.8
blueocean-i18n:1.25.8
blueocean-jwt:1.25.8
blueocean-pipeline-api-impl:1.25.8
blueocean-pipeline-scm-api:1.25.8
blueocean-rest:1.25.8
blueocean-rest-impl:1.25.8
blueocean-web:1.25.8
bootstrap4-api:4.6.0-5
bootstrap5-api:5.2.0-1
bouncycastle-api:2.26
branch-api:2.1046.v0ca_37783ecc5
build-blocker-plugin:1.7.8
build-timeout:1.24
built-on-column:1.1
caffeine-api:2.9.3-65.v6a_47d0f4d1fe
checks-api:1.7.5
cloud-stats:0.27
cloudbees-aborted-builds:1.19
cloudbees-administrative-monitors:1.0.5
cloudbees-analytics:1.42
cloudbees-assurance:2.276.0.23
cloudbees-bitbucket-branch-source:791.vb_eea_a_476405b
cloudbees-blueocean-default-theme:0.8
cloudbees-disk-usage-simple:178.v1a_4d2f6359a_8
cloudbees-folder:6.758.vfd75d09eea_a_1
cloudbees-folders-plus:3.29
cloudbees-github-reporting:1.31
cloudbees-groovy-view:1.14
cloudbees-jenkins-advisor:3.3.3
cloudbees-jsync-archiver:5.25
cloudbees-license:9.68
cloudbees-monitoring:2.15
cloudbees-nodes-plus:1.24
cloudbees-platform-common:1.18
cloudbees-platform-data:1.29
cloudbees-plugin-usage:2.17
cloudbees-request-filter:1.7
cloudbees-ssh-slaves:2.19
cloudbees-support:3.30
cloudbees-template:4.58
cloudbees-uc-data-api:4.50
cloudbees-unified-ui:1.22
cloudbees-view-creation-filter:1.9
cloudbees-workflow-template:3.18
cloudbees-workflow-ui:2.8
command-launcher:84.v4a_97f2027398
commons-lang3-api:3.12.0-36.vd97de6465d5b_
conditional-buildstep:1.4.2
config-file-provider:3.11.1
configurationslicing:430.v966357576543
convert-to-pipeline:1.0
copyartifact:1.47
credentials:1143.vb_e8b_b_ceee347
credentials-binding:523.vd859a_4b_122e6
cucumber-reports:5.7.3
cucumber-testresult-plugin:0.10.1
data-tables-api:1.12.1-3
display-url-api:2.3.6
docker-commons:1.21
docker-workflow:521.v1a_a_dd2073b_2e
durable-task:500.v8927d9fd99d8
echarts-api:5.3.3-1
email-ext:2.91
envinject:2.881.v37c62073ff97
envinject-api:1.199.v3ce31253ed13
extended-read-permission:3.2
external-monitor-job:192.ve979ca_8b_3ccd
favorite:2.4.1
font-awesome-api:6.1.2-1
forensics-api:1.15.1
git:4.11.5
git-client:3.11.2
git-server:99.va_0826a_b_cdfa_d
github:1.35.0
github-api:1.303-400.v35c2d8258028
github-branch-source:1695.v88de84e9f6b_9
github-oauth:0.39
google-oauth-plugin:1.0.7
gradle:1.39.4
groovy:453.vcdb_a_c5c99890
h2-api:1.4.199
handlebars:3.0.8
handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953
htmlpublisher:1.30
infradna-backup:3.38.56
instance-identity:116.vf8f487400980
ionicons-api:31.v4757b_6987003
jackson2-api:2.13.3-285.vc03c0256d517
jacoco:3.3.2
jakarta-activation-api:2.0.1-1
jakarta-mail-api:2.0.1-1
javadoc:226.v71211feb_e7e9
javax-activation-api:1.2.0-4
javax-mail-api:1.6.2-7
jaxb:2.3.6-1
jdk-tool:55.v1b_32b_6ca_f9ca
jenkins-design-language:1.25.8
jersey2-api:2.37-1
jjwt-api:0.11.5-77.v646c772fddb_0
jquery:1.12.4-1
jquery-detached:1.2.1
jquery3-api:3.6.0-4
jsch:0.1.55.61.va_e9ee26616e7
junit:1119.1121.vc43d0fc45561
kubernetes:3697.v771155683e38
kubernetes-cli:1.10.3
kubernetes-client-api:5.12.2-193.v26a_6078f65a_9
kubernetes-credentials:0.9.0
ldap:2.12
lockable-resources:2.18
mailer:438.v02c7f0a_12fa_4
mapdb-api:1.0.9-28.vf251ce40855d
mask-passwords:2.13
matrix-auth:3.1.5
matrix-project:785.v06b_7f47b_c631
maven-plugin:3.19
mercurial:1260.vdfb_723cdcc81
metrics:4.2.10-389.v93143621b_050
mina-sshd-api-common:2.9.1-44.v476733c11f82
mina-sshd-api-core:2.9.1-44.v476733c11f82
mina-sshd-api-scp:2.9.1-44.v476733c11f82
mina-sshd-api-sftp:2.9.1-44.v476733c11f82
momentjs:1.1.1
monitoring:1.91.0
nectar-license:8.41
nectar-rbac:5.76
next-build-number:1.8
node-iterator-api:49.v58a_8b_35f8363
oauth-credentials:0.5
okhttp-api:4.9.3-108.v0feda04578cf
operations-center-agent:2.361.0.1
operations-center-client:2.361.0.1
operations-center-cloud:2.361.0.2
operations-center-context:2.361.0.2
pam-auth:1.10
pipeline-build-step:2.18
pipeline-graph-analysis:195.v5812d95a_a_2f9
pipeline-groovy-lib:612.v84da_9c54906d
pipeline-input-step:449.v77f0e8b_845c4
pipeline-maven:1205.vceea_7b_972817
pipeline-milestone-step:101.vd572fef9d926
pipeline-model-api:2.2114.v2654ca_721309
pipeline-model-definition:2.2114.v2654ca_721309
pipeline-model-extensions:2.2114.v2654ca_721309
pipeline-rest-api:2.24
pipeline-stage-step:296.v5f6908f017a_5
pipeline-stage-tags-metadata:2.2114.v2654ca_721309
pipeline-stage-view:2.24
pipeline-utility-steps:2.13.0
plain-credentials:139.ved2b_9cf7587b
plugin-util-api:2.17.0
popper-api:1.16.1-3
popper2-api:2.11.6-1
prism-api:1.28.0-2
publish-over:0.22
publish-over-ssh:1.24
pubsub-light:1.17
repository-connector:2.2.0
resource-disposer:0.20
run-condition:1.5
scm-api:621.vda_a_b_055e58f7
script-security:1175.v4b_d517d6db_f0
snakeyaml-api:1.31-84.ve43da_fb_49d0b
sonar:2.14
sse-gateway:1.26
ssh:2.6.1
ssh-credentials:305.v8f4381501156
ssh-steps:2.0.39.v831c5e6468b_c
sshd:3.249.v2dc2ea_416e33
structs:324.va_f5d6774f3a_d
subversion:2.16.0
support-core:1206.v14049fa_b_d860
text-finder:1.20
thinBackup:1.12
timestamper:1.20
token-macro:308.v4f2b_ed62b_b_16
trilead-api:2.72.v2a_3236754f73
unique-id:2.2.1
user-activity-monitoring:1.8
validating-string-parameter:2.8
variant:59.vf075fe829ccb
versioncolumn:2.2
versionnumber:1.10
warnings-ng:9.18.0
wikitext:3.15
windows-azure-storage:380.va3a027b784f4
windows-slaves:1.8.1
workflow-aggregator:590.v6a_d052e5a_a_b_5
workflow-api:1192.v2d0deb_19d212
workflow-basic-steps:994.vd57e3ca_46d24
workflow-cps:2759.v87459c4eea_ca_
workflow-cps-checkpoint:2.13
workflow-cps-global-lib:588.v576c103a_ff86
workflow-durable-task-step:1199.v02b_9244f8064
workflow-job:1232.v5a_4c994312f1
workflow-multibranch:716.vc692a_e52371b_
workflow-scm-step:400.v6b_89a_1317c9a_
workflow-step-api:639.v6eca_cd8c04a_a_
workflow-support:838.va_3a_087b_4055b
ws-cleanup:0.43

What Operating System are you using (both controller, and any agents involved in the problem)?

CentOS 7.9

Reproduction steps

  1. create an azure storage account
  2. create azure file share
  3. create azure storage credentials
  4. create ACI template
  5. add volume with mount path and share name. observe that storage credentials selector is empty.

Expected Results

the storage credentials selector should populate values from the jenkins credentials manager

Actual Results

the selector remains empty

Anything else?

No response

ACI should use the DNS server that is also defined in the same VNET

What feature do you want to see added?

When you define a DNS server in the VNET, where also the ACIs should be deployed and this DNS server should be used by these ACIs, you have to add a dnsConfig section in the ARM deployment template.

Therefore, the Private IP Address configuration section should be extended by a property where the name servers can be configured.

To-dos

  • Adjust UI
  • Adjust deploymeny template
  • Extend casc test
  • Extend documentation

Upstream changes

No response

unable to deploy windows container in aci

I want to run a windows container in aci , but always get Error code
ResourceDeploymentFailure
Message
The resource operation completed with terminal provisioning state 'Canceled'.
aciconfig

I can see container creating in azure portal, but after few minutes , I can see the above error message. then container deleted.

Add support for VNET integration

In mid November 2019 Microsoft announced a Linux preview support of custom virtual networks on Azure Container Instances: details here
This is very useful feature, that allows customers to hide and secure their build environment from external world.

Enhancements to azure plugins

Hi 👋

Just creating an umbrella issue to cover some maintenance changes I would like to make to the Azure plugins we're co-maintaining.

  • Switch default branch to master
  • Ensure all documentation is pulled from GitHub and not the wiki
  • Switch release notes to GitHub releases
  • Enable continuous delivery of plugins
  • Enable GitHub issues for issue tracking
  • Migrate issues from Jira to GitHub and delete the Jira component
  • Stretch goal: Create pipeline(s) for running integration tests with comment ops

Switch default branch to master

I don't see a reason to have a 'dev' branch here, all releasing is done on 'dev' and master is synced sometimes. One reason to change is that the plugin site has 'master' currently hardcoded for docs when you pass just the repository URL as the docs location, I noticed in azure-vm-agents that the docs were out of date and tracked it down to the master branch not being synced

Ensure all documentation is pulled from GitHub and not the wiki

The Jenkins wiki cannot be updated any more, most of the plugins have been switched already (by updating the url field in pom.xml and doing a release after migrating the docs).

You can check which ones haven't been migrated on the progress page.

Switch release notes to GitHub releases

Most plugins these days publish their release notes via release drafter which uses the PR title + a label to categorise, this means that users can subscribe to releases and know what's in side along with other tools that integrate with GitHub releases like the plugin site.

More info here: https://github.com/jenkinsci/.github/blob/master/.github/release-drafter.adoc
Example: https://github.com/jenkinsci/configuration-as-code-plugin/releases/tag/configuration-as-code-1.47

Enable continuous delivery of plugins

JEP-229 has provided a way for plugins to be automatically released on a merge to the main branch, this means that you don't have to worry about being around for a release and do it on your machine anymore, along with no more users asking on GitHub pull requests for a release after a change has been merged.

Guide to implementing it.
Example repo

Enable GitHub issues for issue tracking

GitHub issues tend to be a lot easier to manage as they are right with the code, they are far tighter integrated and don't require the extra maintenance work of Jira. Subscribing to issues is just a click in the repository rather than messing around with JQL filters in Jira. Users can go the repository and just click new issue, rather than trying to sign up for a Jenkins project account and figuring out the workflow. There's also a number of GitHub issue templates created by default to help projects.

Migrate issues from Jira to GitHub and delete the Jira component

So that issues don't get lost / double reported etc we should migrate all the existing issues from Jira to GitHub

A project that has been mentioned before is: https://github.com/warden/jira-issues-importer

It could be run as a dry-run on a fork first

Stretch goal: Create pipeline(s) for running integration tests with comment ops

Ideally I would like to have integration tests that aren't triggered automatically to save costs and protect against rogue PRs, but do allow triggering via a pull request. I'll probably spin this off into it's own issue

Let me know what you think @xuzhang3, and if you have any objections

Checklist:

Template:

  • Switch default branch to master
  • Ensure all documentation is pulled from GitHub and not the wiki
  • Switch release notes to GitHub releases
  • Enable continuous delivery of plugins
  • Enable GitHub issues for issue tracking
  • Migrate issues from Jira to GitHub and delete the Jira component

NPE on ci.jenkins.io deleting ACI agent

Version report

Jenkins and plugins versions report:

Jenkins: 2.263.4
OS: Linux - 5.4.0-1031-azure
---
ace-editor:1.1
amazon-ecs:1.37
analysis-model-api:9.8.1
ansicolor:0.7.5
ant:1.11
antisamy-markup-formatter:2.1
apache-httpcomponents-client-4-api:4.5.13-1.0
authentication-tokens:1.4
aws-credentials:1.28
aws-java-sdk:1.11.955
azure-commons:1.0.5
azure-container-agents:1.2.0
azure-credentials:4.0.6
azure-vm-agents:1.5.1
basic-branch-build-strategies:1.3.2
beer:1.3
blueocean:1.24.4
blueocean-autofavorite:1.2.4
blueocean-bitbucket-pipeline:1.24.4
blueocean-commons:1.24.4
blueocean-config:1.24.4
blueocean-core-js:1.24.4
blueocean-dashboard:1.24.4
blueocean-display-url:2.4.1
blueocean-events:1.24.4
blueocean-github-pipeline:1.24.4
blueocean-git-pipeline:1.24.4
blueocean-i18n:1.24.4
blueocean-jira:1.24.4
blueocean-jwt:1.24.4
blueocean-personalization:1.24.4
blueocean-pipeline-api-impl:1.24.4
blueocean-pipeline-editor:1.24.4
blueocean-pipeline-scm-api:1.24.4
blueocean-rest:1.24.4
blueocean-rest-impl:1.24.4
blueocean-web:1.24.4
bootstrap4-api:4.6.0-2
bouncycastle-api:2.20
branch-api:2.6.2
build-timeout:1.20
buildtriggerbadge:2.10
checks-api:1.5.0
cloudbees-bitbucket-branch-source:2.9.7
cloudbees-folder:6.15
cloud-stats:0.26
code-coverage-api:1.3.1
command-launcher:1.5
conditional-buildstep:1.4.1
configuration-as-code:1.47
copyartifact:1.46
credentials:2.3.15
credentials-binding:1.24
cvs:2.18
dark-theme:0.0.10
data-tables-api:1.10.23-3
disable-github-multibranch-status:1.2
display-url-api:2.3.4
docker-commons:1.17
docker-workflow:1.26
durable-task:1.35
ec2:1.56
echarts-api:5.0.1-1
embeddable-build-status:2.0.3
extended-read-permission:3.2
external-monitor-job:1.7
favorite:2.3.2
font-awesome-api:5.15.2-2
forensics-api:1.0.0
git:4.6.0
git-client:3.6.0
git-forensics:1.0.0
github:1.33.1
github-api:1.123
github-branch-source:2.9.7
github-checks:1.0.10
git-server:1.9
groovy:2.3
handlebars:1.1.1
handy-uri-templates-2-api:2.1.8-1.0
htmlpublisher:1.25
jackson2-api:2.12.1
jacoco:3.1.1
javadoc:1.6
jaxb:2.3.0.1
jdk-tool:1.5
jenkins-design-language:1.24.4
jira:3.2
jjwt-api:0.11.2-9.c8b45b8bb173
jobConfigHistory:2.26
jquery:1.12.4-1
jquery3-api:3.5.1-3
jquery-detached:1.2.1
jsch:0.1.55.2
junit:1.48
junit-attachments:1.6
junit-realtime-test-reporter:0.6
keyboard-shortcuts-plugin:1.4
ldap:1.26
lockable-resources:2.10
mailer:1.32.1
mapdb-api:1.0.9.0
matrix-auth:2.6.5
matrix-project:1.18
maven-plugin:3.8
mercurial:2.12
metrics:4.0.2.7
momentjs:1.1.1
node-iterator-api:1.5.0
okhttp-api:3.14.9
pam-auth:1.6
parallel-test-executor:1.13
parameterized-trigger:2.39
pipeline-build-step:2.13
pipeline-github-lib:1.0
pipeline-githubnotify-step:1.0.5
pipeline-graph-analysis:1.10
pipeline-input-step:2.12
pipeline-milestone-step:1.3.2
pipeline-model-api:1.8.4
pipeline-model-definition:1.8.4
pipeline-model-extensions:1.8.4
pipeline-rest-api:2.19
pipeline-stage-step:2.5
pipeline-stage-tags-metadata:1.8.4
pipeline-stage-view:2.19
pipeline-utility-steps:2.6.1
plain-credentials:1.7
plugin-util-api:2.0.0
popper-api:1.16.1-2
pubsub-light:1.13
run-condition:1.5
scm-api:2.6.4
script-security:1.76
snakeyaml-api:1.27.0
sse-gateway:1.24
ssh-agent:1.21
ssh-credentials:1.18.1
ssh-slaves:1.31.5
structs:1.22
subversion:2.14.0
support-core:2.72.1
theme-manager:0.6
throttle-concurrents:2.1
timestamper:1.11.8
token-macro:2.13
toolenv:1.2
translation:1.16
trilead-api:1.0.13
variant:1.4
warnings-ng:8.10.0
windows-azure-storage:1.1.7
windows-slaves:1.7
workflow-aggregator:2.6
workflow-api:2.41
workflow-basic-steps:2.23
workflow-cps:2.90
workflow-cps-global-lib:2.18
workflow-durable-task-step:2.38
workflow-job:2.40
workflow-multibranch:2.22
workflow-scm-step:2.12
workflow-step-api:2.23
workflow-support:3.8
  • What Operating System are you using (both controller, and any agents involved in the problem)?
Controller:
Jenkins: 2.263.4
OS: Linux - 5.4.0-1031-azure
Agents:
Azure Container Instances as managed by ci.jenkins.io

Reproduction steps

  • Earlier today ci.jenkins.io was overloaded with a flood of requests for Azure container instances. Highly likely that we allocated until our ACI limit was reached
  • Attempts to delete those container instances are now reporting a null pointer exception in the Jenkins logs.

Results

Expected result:

Azure container instances should be deleted even after we've hit the upper limit of allowed container instances as defined in our Azure account.

Actual result:

Azure containers fail to delete and report a null pointer exception in the Jenkins logs. The null pointer exception looks like this:

Mar 08, 2021 10:04:50 PM WARNING com.microsoft.jenkins.containeragents.aci.AciService deleteAciContainerGroup
Delete ACI deployment: aci-ruby-mcwf4zst failed
java.lang.NullPointerException
	at com.microsoft.jenkins.containeragents.aci.AciService.deleteAciContainerGroup(AciService.java:290)
	at com.microsoft.jenkins.containeragents.aci.AciAgent$1.run(AciAgent.java:100)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

Combining ACI build agents with sidecar containers

I'm not sure if I'm actually asking the same question as in issue #20. But my question is this: I want to achieve something similar like described here: https://jenkins.io/doc/book/pipeline/docker/#running-sidecar-containers

I want to launch an Jenkins slave agent to kick off my build proces, but for my complex build I need additional services to be available, for example an MySQL database. I want to start an additional ACI container making this service available to my build. Is this achievable with the current Azure Container Agents plugin?

Many jobs based on the same agent template produce many failed deployment

Jenkins and plugins versions report

Environment
Jenkins: 2.319.3
OS: Linux - 5.4.0-1068-azure
---
ace-editor:1.1
ant:1.13
antisamy-markup-formatter:2.7
apache-httpcomponents-client-4-api:4.5.13-1.0
authentication-tokens:1.4
azure-container-agents:222.va22f04771da0
azure-credentials:198.vf9c2fdfde55c
azure-sdk:85.v4817a_b_8a_7124
blueocean-commons:1.25.2
blueocean-rest:1.25.2
bootstrap4-api:4.6.0-3
bootstrap5-api:5.1.3-4
bouncycastle-api:2.25
branch-api:2.7.0
build-timeout:1.20
caffeine-api:2.9.2-29.v717aac953ff3
checks-api:1.7.2
cloud-stats:0.27
cloudbees-folder:6.17
command-launcher:1.6
copyartifact:1.46.2
credentials:1074.v60e6c29b_b_44b_
credentials-binding:1.27.1
display-url-api:2.3.5
docker-commons:1.19
durable-task:493.v195aefbb0ff2
echarts-api:5.2.2-2
email-ext:2.87
extended-read-permission:3.2
font-awesome-api:5.15.4-5
git:4.10.3
git-client:3.11.0
git-server:1.10
github:1.34.1
github-api:1.301-378.v9807bd746da5
github-branch-source:2.11.4
gradle:1.38
handlebars:3.0.8
jackson2-api:2.13.1-246.va8a9f3eaf46a
javax-activation-api:1.2.0-2
javax-mail-api:1.6.2-5
jaxb:2.3.0.1
jdk-tool:1.5
jjwt-api:0.11.2-9.c8b45b8bb173
jnr-posix-api:3.1.7-1
jquery3-api:3.6.0-2
jsch:0.1.55.2
junit:1.54
ldap:2.7
lockable-resources:2.14
mailer:408.vd726a_1130320
matrix-auth:3.0.1
matrix-project:1.20
momentjs:1.1.1
okhttp-api:4.9.3-105.vb96869f8ac3a
pam-auth:1.7
pipeline-build-step:2.15
pipeline-github-lib:36.v4c01db_ca_ed16
pipeline-graph-analysis:188.v3a01e7973f2c
pipeline-input-step:446.vf27b_0b_83500e
pipeline-milestone-step:100.v60a_03cd446e1
pipeline-model-api:1.9.3
pipeline-model-definition:1.9.3
pipeline-model-extensions:1.9.3
pipeline-rest-api:2.21
pipeline-stage-step:291.vf0a8a7aeeb50
pipeline-stage-tags-metadata:1.9.3
pipeline-stage-view:2.21
plain-credentials:1.8
plugin-util-api:2.13.0
popper-api:1.16.1-2
popper2-api:2.11.2-1
resource-disposer:0.17
scm-api:595.vd5a_df5eb_0e39
script-security:1131.v8b_b_5eda_c328e
snakeyaml-api:1.29.1
ssh-credentials:1.19
ssh-slaves:1.33.0
sshd:3.1.0
structs:308.v852b473a2b8c
timestamper:1.17
token-macro:270.v8c2ce50dc4fc
trilead-api:1.0.13
windows-azure-storage:368.v7443dd3deffe
workflow-aggregator:2.6
workflow-api:1136.v7f5f1759dc16
workflow-basic-steps:2.24
workflow-cps:2648.va9433432b33c
workflow-cps-global-lib:552.vd9cc05b8a2e1
workflow-durable-task-step:1121.va_65b_d2701486
workflow-job:1145.v7f2433caa07f
workflow-multibranch:706.vd43c65dec013
workflow-scm-step:2.13
workflow-step-api:622.vb_8e7c15b_c95a_
workflow-support:813.vb_d7c3d2984a_0
ws-cleanup:0.40

What Operating System are you using (both controller, and any agents involved in the problem)?

Contoller based on Ubuntu 18.04
Agents are Container Instances in Azure based on following Container Template

      <templates>
        <com.microsoft.jenkins.containeragents.aci.AciContainerTemplate>
          <name>test-private</name>
          <label>test-private</label>
          <image>jenkins/inbound-agent</image>
          <osType>Linux</osType>
          <command>jenkins-agent -url ${rootUrl} ${secret} ${nodeName}</command>
          <rootFs>/home/jenkins</rootFs>
          <privateIpAddress>
            <vnet>jenkins-vnet</vnet>
            <subnet>jenkins-agent-subnet</subnet>
            <resourceGroup>jenkins-vnet-rg</resourceGroup>
          </privateIpAddress>
          <timeout>5</timeout>
          <ports/>
          <cpu>1</cpu>
          <memory>1.5</memory>
          <retentionStrategy class="com.microsoft.jenkins.containeragents.strategy.ContainerOnceRetentionStrategy">
            <idleMinutes>10</idleMinutes>
          </retentionStrategy>
          <envVars/>
          <privateRegistryCredentials/>
          <volumes/>
          <isAvailable>true</isAvailable>
        </com.microsoft.jenkins.containeragents.aci.AciContainerTemplate>

Reproduction steps

  1. Jenkins VM is installed in Azure
  2. Agent Container using Private IP Address Option.
  3. Start parallel several (I took 5) jobs based on following pipeline script
pipeline {
    agent { label "test-private" }

    stages {
        stage('Hello') {
            steps {
                echo 'Hello World'
            }
        }
    }
}
  1. Wait 5 minutes and then check the logs

Expected Results

No failing deployments and the whole test should not need 5 minutes.

Actual Results

After modified

LOGGER.log(Level.WARNING, "AciCloud: Provision agent {0} failed: {1}",
to

                                LOGGER.log(Level.WARNING, "AciCloud: Provision agent {0} failed: {1}",
                                        new Object[] {agent.getNodeName(), e});
                                e.printStackTrace();

you can see the following information (only snippet) in /var/log/jenkins/jenkins.log

2022-02-11 08:56:31.513+0000 [id=36]    INFO    c.m.j.c.aci.AciCloud#provision: Start ACI container for label test-private workLoad 1
2022-02-11 08:56:31.515+0000 [id=36]    INFO    c.m.j.c.aci.AciCloud#provision: Using ACI Container template: test-private
2022-02-11 08:56:31.524+0000 [id=74]    INFO    c.m.j.c.aci.AciCloud#lambda$provision$1: Add ACI node: test-private-mswkt
2022-02-11 08:56:31.528+0000 [id=36]    INFO    h.s.NodeProvisioner$StandardStrategyImpl#apply: Started provisioning test-private-mswkt from Aci with 1 executors. Remaining excess workload: 0
2022-02-11 08:56:31.913+0000 [id=74]    INFO    c.m.j.c.a.AciCleanTask$DeploymentRegistrar#registerDeployment: AzureAciCleanUpTask: registerDeployment: Registering deployment test-private-zmv9hzln in jenkins-rg
2022-02-11 08:56:32.286+0000 [id=74]    INFO    c.a.c.util.logging.ClientLogger#info: Package versions: jackson-annotations=2.13.1, jackson-core=2.13.1, jackson-databind=2.13.1, jackson-dataformat-xml=2.13.1, jackson-datatype-jsr310=2.13.1, azure-core=1.24.1, Troubleshooting version conflicts: https://aka.ms/azsdk/java/dependency/troubleshoot
2022-02-11 08:56:39.314+0000 [id=90]    INFO    c.a.c.util.logging.ClientLogger#performLogging: Azure Identity => getToken() result for scopes [https://management.core.windows.net//.default]: SUCCESS
2022-02-11 08:56:39.314+0000 [id=90]    INFO    c.a.c.util.logging.ClientLogger#info: Acquired a new access token.
2022-02-11 08:56:39.944+0000 [id=74]    INFO    c.m.j.c.aci.AciService#createDeployment: Waiting for deployment test-private-zmv9hzln
2022-02-11 08:57:11.510+0000 [id=32]    INFO    c.m.j.c.aci.AciCloud#provision: Start ACI container for label test-private workLoad 1
2022-02-11 08:57:11.511+0000 [id=32]    INFO    c.m.j.c.aci.AciCloud#provision: Using ACI Container template: test-private
2022-02-11 08:57:11.511+0000 [id=73]    INFO    c.m.j.c.aci.AciCloud#lambda$provision$1: Add ACI node: test-private-pcsmp
2022-02-11 08:57:11.512+0000 [id=32]    INFO    h.s.NodeProvisioner$StandardStrategyImpl#apply: Started provisioning test-private-pcsmp from Aci with 1 executors. Remaining excess workload: -0.28
2022-02-11 08:57:11.544+0000 [id=73]    INFO    c.m.j.c.a.AciCleanTask$DeploymentRegistrar#registerDeployment: AzureAciCleanUpTask: registerDeployment: Registering deployment test-private-m4cwglm5 in jenkins-rg
2022-02-11 08:57:11.786+0000 [id=73]    INFO    c.m.j.c.aci.AciService#createDeployment: Waiting for deployment test-private-m4cwglm5
2022-02-11 08:57:39.993+0000 [id=95]    INFO    h.TcpSlaveAgentListener$ConnectionHandler#run: Connection #1 failed: java.io.EOFException
2022-02-11 08:57:40.060+0000 [id=96]    INFO    h.TcpSlaveAgentListener$ConnectionHandler#run: Accepted JNLP4-connect connection #2 from /20.123.8.94:9025
2022-02-11 08:57:41.510+0000 [id=41]    INFO    c.m.j.c.aci.AciCloud#provision: Start ACI container for label test-private workLoad 1
2022-02-11 08:57:41.510+0000 [id=41]    INFO    c.m.j.c.aci.AciCloud#provision: Using ACI Container template: test-private
2022-02-11 08:57:41.511+0000 [id=109]   INFO    c.m.j.c.aci.AciCloud#lambda$provision$1: Add ACI node: test-private-267hx
2022-02-11 08:57:41.511+0000 [id=41]    INFO    h.s.NodeProvisioner$StandardStrategyImpl#apply: Started provisioning test-private-267hx from Aci with 1 executors. Remaining excess workload: -0.391
2022-02-11 08:57:41.534+0000 [id=109]   INFO    c.m.j.c.a.AciCleanTask$DeploymentRegistrar#registerDeployment: AzureAciCleanUpTask: registerDeployment: Registering deployment test-private-fkx4v9fw in jenkins-rg
2022-02-11 08:57:41.764+0000 [id=109]   INFO    c.m.j.c.aci.AciService#createDeployment: Waiting for deployment test-private-fkx4v9fw
2022-02-11 08:57:47.007+0000 [id=174]   INFO    o.j.p.workflow.job.WorkflowRun#finish: test-private #49 completed: SUCCESS
2022-02-11 08:57:56.939+0000 [id=36]    INFO    c.m.j.c.s.ContainerOnceRetentionStrategy#done: terminating test-private-mswkt since PlaceholderExecutable:ExecutorStepExecution.PlaceholderTask{runId=test-private#49,label=test-private-mswkt,context=CpsStepContext[3:node]:Owner[test-private/49:test-private #49],cookie=c1b12013-ba81-4716-8597-771889790e9e,auth=null} seems to be finished
2022-02-11 08:57:56.956+0000 [id=140]   INFO    j.s.DefaultJnlpSlaveReceiver#channelClosed: Computer.threadPoolForRemoting [#52] for test-private-mswkt terminated: java.nio.channels.ClosedChannelException
2022-02-11 08:57:57.134+0000 [id=199]   INFO    c.a.c.util.logging.ClientLogger#performLogging: Azure Identity => getToken() result for scopes [https://management.core.windows.net//.default]: SUCCESS
2022-02-11 08:57:57.134+0000 [id=199]   INFO    c.a.c.util.logging.ClientLogger#info: Acquired a new access token.
2022-02-11 08:57:57.841+0000 [id=145]   INFO    c.m.j.c.aci.AciService#deleteAciContainerGroup: Delete ACI Container Group: test-private-mswkt successfully
2022-02-11 08:58:00.342+0000 [id=74]    WARNING c.m.j.c.aci.AciCloud#lambda$provision$1: AciCloud: Provision agent test-private-mswkt failed: java.lang.Exception: Deployment test-private-zmv9hzln status: Failed
java.lang.Exception: Deployment test-private-zmv9hzln status: Failed
        at com.microsoft.jenkins.containeragents.aci.AciService.createDeployment(AciService.java:65)
        at com.microsoft.jenkins.containeragents.aci.AciContainerTemplate.provisionAgents(AciContainerTemplate.java:130)
        at com.microsoft.jenkins.containeragents.aci.AciCloud.lambda$provision$1(AciCloud.java:113)
        at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
        at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:80)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
2022-02-11 08:58:01.510+0000 [id=35]    WARNING hudson.slaves.NodeProvisioner#update: Unexpected exception encountered while provisioning agent test-private-mswkt
java.lang.Exception: Deployment test-private-zmv9hzln status: Failed
        at com.microsoft.jenkins.containeragents.aci.AciService.createDeployment(AciService.java:65)
        at com.microsoft.jenkins.containeragents.aci.AciContainerTemplate.provisionAgents(AciContainerTemplate.java:130)
        at com.microsoft.jenkins.containeragents.aci.AciCloud.lambda$provision$1(AciCloud.java:113)
        at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
        at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:80)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
.....
2022-02-11 09:00:21.512+0000 [id=32]    INFO    h.s.NodeProvisioner$StandardStrategyImpl#apply: Started provisioning test-private-n4s69 from Aci with 1 executors. Remaining excess workload: 0
2022-02-11 09:00:21.528+0000 [id=74]    INFO    c.m.j.c.a.AciCleanTask$DeploymentRegistrar#registerDeployment: AzureAciCleanUpTask: registerDeployment: Registering deployment test-private-hcqp2z3p in jenkins-rg
2022-02-11 09:00:21.846+0000 [id=74]    INFO    c.m.j.c.aci.AciService#createDeployment: Waiting for deployment test-private-hcqp2z3p
2022-02-11 09:01:28.549+0000 [id=264]   INFO    h.TcpSlaveAgentListener$ConnectionHandler#run: Connection #9 failed: java.io.EOFException
2022-02-11 09:01:28.657+0000 [id=265]   INFO    h.TcpSlaveAgentListener$ConnectionHandler#run: Accepted JNLP4-connect connection #10 from /13.79.74.203:22465
2022-02-11 09:01:36.318+0000 [id=303]   INFO    o.j.p.workflow.job.WorkflowRun#finish: test-private #50 completed: SUCCESS
2022-02-11 09:01:42.578+0000 [id=74]    INFO    c.m.j.c.aci.AciService#createDeployment: Deployment test-private-hcqp2z3p succeed
2022-02-11 09:01:42.579+0000 [id=74]    INFO    c.m.j.c.aci.AciCloud#waitToOnline: Waiting agent test-private-n4s69 to online
2022-02-11 09:01:42.674+0000 [id=74]    WARNING c.m.j.c.aci.AciCloud#lambda$provision$1: AciCloud: Provision agent test-private-n4s69 failed: java.lang.NullPointerException
java.lang.NullPointerException
        at com.azure.resourcemanager.containerinstance.implementation.ContainerGroupImpl.initializeChildrenFromInner(ContainerGroupImpl.java:217)
        at com.azure.resourcemanager.resources.fluentcore.arm.models.implementation.GroupableParentResourceImpl.<init>(GroupableParentResourceImpl.java:32)
        at com.azure.resourcemanager.containerinstance.implementation.ContainerGroupImpl.<init>(ContainerGroupImpl.java:73)
        at com.azure.resourcemanager.containerinstance.implementation.ContainerGroupsImpl.wrapModel(ContainerGroupsImpl.java:43)
        at com.azure.resourcemanager.containerinstance.implementation.ContainerGroupsImpl.wrapModel(ContainerGroupsImpl.java:24)
        at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113)
        at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816)
        at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151)
        at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107)
        at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816)
        at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249)
        at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74)
        at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816)
        at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249)
        at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816)
        at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249)
        at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398)
        at reactor.core.publisher.MonoFlatMap$FlatMapInner.onSubscribe(MonoFlatMap.java:238)
        at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55)

On Azure Monitor page, you can see that the affected deployments were canceled.

Anything else?

No response

Provision failures does not clean up deployments

We're seeing a number of failures on ci.jenkins.io which stem from Azure Container Agents failing to provision. It looks like failed ACI deployments don't get cleaned up automatically by the plugin.

ACI is returning the following failure:

{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-debug for usage details.","details":[{"code":"Conflict","message":"{\r\n \"error\": {\r\n \"code\": \"ServiceUnavailable\",\r\n \"message\": \"The requested resource is not available in the location 'eastus2' at this moment. Please retry with a different resource request or in another location. Resource requested: '2' CPU '4' GB memory 'Linux' OS\"\r\n }\r\n}"}]}

Which is annoying, but whatever. The bug here is that the plugin does not clean up this failed deployment, so we have exceeded our quota of Deployments of 800, all failures like this one!

I recall this exact same issue in the Azure VM Agents plugin at one point as well.

[JENKINS-57204] created by rtyler

If using Private IP Address and resource group of vnet is empty, the resource group is not used from the Azure Container Instance Configuration

Jenkins and plugins versions report

Environment
Jenkins: 2.332.1
OS: Linux - 5.4.0-1072-azure
---
ace-editor:1.1
ant:1.13
antisamy-markup-formatter:2.7
apache-httpcomponents-client-4-api:4.5.13-1.0
authentication-tokens:1.4
azure-container-agents:224.v03c1dc7f46f2
azure-credentials:216.ve0b_4a_485ffc2
azure-sdk:106.v552de1e64d56
blueocean-commons:1.25.3
blueocean-rest:1.25.3
bootstrap4-api:4.6.0-3
bootstrap5-api:5.1.3-6
bouncycastle-api:2.25
branch-api:2.7.0
build-timeout:1.20
caffeine-api:2.9.2-29.v717aac953ff3
checks-api:1.7.2
cloud-stats:0.27
cloudbees-folder:6.708.ve61636eb_65a_5
command-launcher:1.6
copyartifact:1.46.3
credentials:1074.v60e6c29b_b_44b_
credentials-binding:1.27.1
display-url-api:2.3.5
docker-commons:1.19
durable-task:493.v195aefbb0ff2
echarts-api:5.3.0-2
email-ext:2.87
extended-read-permission:3.2
font-awesome-api:6.0.0-1
git:4.10.3
git-client:3.11.0
git-server:1.10
github:1.34.3
github-api:1.301-378.v9807bd746da5
github-branch-source:1583.v18d333ef7379
gradle:1.38
handlebars:3.0.8
jackson2-api:2.13.2-260.v43d711474c77
javax-activation-api:1.2.0-2
javax-mail-api:1.6.2-5
jaxb:2.3.0.1
jdk-tool:1.5
jjwt-api:0.11.2-9.c8b45b8bb173
jnr-posix-api:3.1.7-3
jquery3-api:3.6.0-2
jsch:0.1.55.2
junit:1.56
ldap:2.8
lockable-resources:2.14
mailer:408.vd726a_1130320
matrix-auth:3.1
matrix-project:758.v7a_ea_491852f3
momentjs:1.1.1
okhttp-api:4.9.3-105.vb96869f8ac3a
pam-auth:1.7
pipeline-build-step:2.16
pipeline-github-lib:36.v4c01db_ca_ed16
pipeline-graph-analysis:188.v3a01e7973f2c
pipeline-input-step:446.vf27b_0b_83500e
pipeline-milestone-step:100.v60a_03cd446e1
pipeline-model-api:2.2064.v5eef7d0982b_e
pipeline-model-definition:2.2064.v5eef7d0982b_e
pipeline-model-extensions:2.2064.v5eef7d0982b_e
pipeline-rest-api:2.23
pipeline-stage-step:291.vf0a8a7aeeb50
pipeline-stage-tags-metadata:2.2064.v5eef7d0982b_e
pipeline-stage-view:2.23
plain-credentials:1.8
plugin-util-api:2.14.0
popper-api:1.16.1-2
popper2-api:2.11.4-1
resource-disposer:0.17
scm-api:595.vd5a_df5eb_0e39
script-security:1140.vf967fb_efa_55a_
snakeyaml-api:1.29.1
ssh-credentials:1.19
ssh-slaves:1.806.v2253cedd3295
sshd:3.1.0
structs:308.v852b473a2b8c
timestamper:1.17
token-macro:280.v97a_82642793c
trilead-api:1.0.13
windows-azure-storage:373.v582b31a65906
workflow-aggregator:2.7
workflow-api:1143.v2d42f1e9dea_5
workflow-basic-steps:941.vdfe1b_a_132c64
workflow-cps:2660.vb_c0412dc4e6d
workflow-cps-global-lib:564.ve62a_4eb_b_e039
workflow-durable-task-step:1121.va_65b_d2701486
workflow-job:1174.vdcb_d054cf74a_
workflow-multibranch:711.vdfef37cda_816
workflow-scm-step:2.13
workflow-step-api:622.vb_8e7c15b_c95a_
workflow-support:815.vd60466279fc8
ws-cleanup:0.40

What Operating System are you using (both controller, and any agents involved in the problem)?

Controller is Ubuntu 18.04 on Azure

Reproduction steps

Using following configuration:

    <com.microsoft.jenkins.containeragents.aci.AciCloud plugin="[email protected]">
      <name>Aci</name>
      <credentialsId>azure-jenkins-principal</credentialsId>
      <logAnalyticsCredentialsId></logAnalyticsCredentialsId>
      <resourceGroup>jenkins-rg</resourceGroup>
      <templates>
        <com.microsoft.jenkins.containeragents.aci.AciContainerTemplate>
          <name>test</name>
          <label>test</label>
          <image>jenkins/inbound-agent</image>
          <osType>Linux</osType>
          <command>jenkins-agent -url ${rootUrl} ${secret} ${nodeName}</command>
          <rootFs>/home/jenkins</rootFs>
          <privateIpAddress>
            <vnet>jenkins-vmVNET</vnet>
            <subnet>jenkins-agent-subnet</subnet>
            <resourceGroup></resourceGroup>
          </privateIpAddress>
          <timeout>5</timeout>
          <ports/>
          <cpu>1</cpu>
          <memory>1.5</memory>
          <retentionStrategy class="com.microsoft.jenkins.containeragents.strategy.ContainerOnceRetentionStrategy">
            <idleMinutes>10</idleMinutes>
          </retentionStrategy>
          <envVars/>
          <privateRegistryCredentials/>
          <volumes/>
          <isAvailable>true</isAvailable>
        </com.microsoft.jenkins.containeragents.aci.AciContainerTemplate>
      </templates>
    </com.microsoft.jenkins.containeragents.aci.AciCloud>

Expected Results

The deployment template in debug log should have set vnetResourceGroupName to jenkins-rg but it isn't.

Actual Results

Property vnetResourceGroupName in the deployment template is empty. Therefore, the container instance fails during creating in Azure.

Anything else?

No response

Insufficient document, no validation steps mentioned

Describe your use-case which is not covered by existing documentation.

followed the steps mentioned in https://plugins.jenkins.io/azure-container-agents/

The steps are getting over at "Configure the Container Template", There is no step to create and connect to the container node or to validate the steps.

Please guide me to the correct document for the setup, I am using my Jenkins in my local system and the expectation is to connect to azure container instance from Jenkins using the agent

Reference any relevant documentation, other materials or issues/pull requests that can be used for inspiration.

No response

Launch Method not saving

Jenkins and plugins versions report

Jenkins: 2.319.3
OS: Linux - 5.11.0-1028-azure

Office-365-Connector:4.15.2
ace-editor:1.1
active-directory:2.25.1
antisamy-markup-formatter:2.7
apache-httpcomponents-client-4-api:4.5.13-1.0
atlassian-jira-software-cloud:1.4.5
authentication-tokens:1.4
authorize-project:1.4.0
azure-ad:191.vfc8019068670
azure-cli:0.9
azure-container-agents:222.va22f04771da0
azure-credentials:198.vf9c2fdfde55c
azure-sdk:85.v4817a_b_8a_7124
blueocean-commons:1.25.2
blueocean-rest:1.25.2
bootstrap4-api:4.6.0-3
bootstrap5-api:5.1.3-4
bouncycastle-api:2.25
branch-api:2.7.0
build-timeout:1.20
caffeine-api:2.9.2-29.v717aac953ff3
checks-api:1.7.2
cloud-stats:0.27
cloudbees-folder:6.17
command-launcher:1.6
configuration-as-code:1346.ve8cfa_3473c94
copyartifact:1.46.2
credentials:1074.v60e6c29b_b_44b_
credentials-binding:1.27.1
dashboard-view:2.18
display-url-api:2.3.5
docker-commons:1.19
durable-task:493.v195aefbb0ff2
echarts-api:5.2.2-2
email-ext:2.87
extended-read-permission:3.2
font-awesome-api:5.15.4-5
git:4.10.3
git-client:3.11.0
git-server:1.10
github:1.34.1
github-api:1.301-378.v9807bd746da5
github-branch-source:2.11.4
handlebars:3.0.8
jackson2-api:2.13.1-246.va8a9f3eaf46a
javax-activation-api:1.2.0-2
javax-mail-api:1.6.2-5
jaxb:2.3.0.1
jdk-tool:1.5
jjwt-api:0.11.2-9.c8b45b8bb173
jnr-posix-api:3.1.7-1
jquery3-api:3.6.0-2
jsch:0.1.55.2
junit:1.54
kubernetes-cli:1.10.3
kubernetes-client-api:5.11.2-182.v0f1cf4c5904e
kubernetes-credentials:0.9.0
ldap:2.7
lockable-resources:2.14
mailer:408.vd726a_1130320
matrix-auth:3.0.1
matrix-project:1.20
momentjs:1.1.1
okhttp-api:4.9.3-105.vb96869f8ac3a
pam-auth:1.7
pipeline-build-step:2.15
pipeline-github-lib:36.v4c01db_ca_ed16
pipeline-graph-analysis:188.v3a01e7973f2c
pipeline-input-step:446.vf27b_0b_83500e
pipeline-milestone-step:100.v60a_03cd446e1
pipeline-model-api:1.9.3
pipeline-model-definition:1.9.3
pipeline-model-extensions:1.9.3
pipeline-rest-api:2.21
pipeline-stage-step:291.vf0a8a7aeeb50
pipeline-stage-tags-metadata:1.9.3
pipeline-stage-view:2.21
plain-credentials:1.8
plugin-util-api:2.13.0
popper-api:1.16.1-2
popper2-api:2.11.2-1
resource-disposer:0.17
role-strategy:3.2.0
scm-api:595.vd5a_df5eb_0e39
script-security:1131.v8b_b_5eda_c328e
snakeyaml-api:1.29.1
ssh-credentials:1.19
ssh-slaves:1.33.0
sshd:3.1.0
structs:308.v852b473a2b8c
timestamper:1.17
token-macro:270.v8c2ce50dc4fc
trilead-api:1.0.13
windows-azure-storage:371.v19a0f02c7603
workflow-aggregator:2.6
workflow-api:1136.v7f5f1759dc16
workflow-basic-steps:2.24
workflow-cps:2648.va9433432b33c
workflow-cps-global-lib:552.vd9cc05b8a2e1
workflow-durable-task-step:1121.va_65b_d2701486
workflow-job:1145.v7f2433caa07f
workflow-multibranch:706.vd43c65dec013
workflow-scm-step:2.13
workflow-step-api:622.vb_8e7c15b_c95a_
workflow-support:813.vb_d7c3d2984a_0
ws-cleanup:0.40

What Operating System are you using (both controller, and any agents involved in the problem)?

Ubuntu 20.04.3 LTS (GNU/Linux 5.11.0-1028-azure x86_64)

Reproduction steps

  1. Manage Jenkins
  2. Manage Nodes & Clouds
  3. Configure Clouds
  4. Change Launch Method from Inbound agent to SSH

Expected Results

Launch method is changed to SSH from inbound agent

Actual Results

Launch method remains as inbound agent

Anything else?

No response

Agents not cleaned up in the interface after being used

It's not clear to me what is causing this, but it just seems to naturally happen over time. Agents seem to be left in offline states but are not deleted at the end of their workload.

We periodically run doDoDelete() through the Script Console to delete these :-/

[JENKINS-58997] created by rtyler

Connect to null failed: null reopen

AKS Pod Template has exception logs recorded on April 6, messaging that NullPointerException
in Jenkins master, resulted in any pod unavailable as a ssh slave. It seems that the event is similar
cause of the ticket, "Connect to null failed: null #34", suggested that the past problem could have
been re-invoked again.

In detail log/messages are shown bellow;
Error in provisioning; slave=com.microsoft.jenkins.containeragents.KubernetesAgent[ssh-slave-rj4zb], template=com.microsoft.jenkins.containeragents.PodTemplate@31b0e567: java.lang.NullPointerException Apr 06, 2019 1:31:21 AM INFO com.microsoft.jenkins.containeragents.KubernetesCloud$ProvisionCallback call Removing Jenkins node: ssh-slave-rj4zb Apr 06, 2019 1:31:21 AM INFO com.microsoft.jenkins.containeragents.KubernetesCloud deletePod Terminating container instance for slave ssh-slave-rj4zb Apr 06, 2019 1:31:21 AM INFO okhttp3.internal.platform.Platform log ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path? Apr 06, 2019 1:31:21 AM INFO okhttp3.internal.platform.Platform log ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path? Apr 06, 2019 1:31:22 AM WARNING com.microsoft.jenkins.containeragents.KubernetesCloud deletePod Failed to terminate pod for slave ssh-slave-rj4zb java.lang.NullPointerException at com.microsoft.jenkins.containeragents.KubernetesService.getKubernetesClient(KubernetesService.java:223) at com.microsoft.jenkins.containeragents.KubernetesCloud.connect(KubernetesCloud.java:101) at com.microsoft.jenkins.containeragents.KubernetesCloud.deletePod(KubernetesCloud.java:285) at com.microsoft.jenkins.containeragents.KubernetesAgent$1.run(KubernetesAgent.java:100) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
To confirm, Jenkins AKS pod template configuration tested connection to AKS Cluster by
"Test Connection" button on Jenkins Web UI. "Connect to null failed: null" showed as an error
message. Service Principal as credential has correctly authenticated.

In addition, AKS version both with 1.12.4 and 1.12.6 lead to the same problem.
And azure-container-agents-plugin has recently no source code with any commit,
possibly not involved in a cause of the trouble.
https://github.com/jenkinsci/azure-container-agents-plugin

In terms of kubelet, as following in normal;
$ kubectl config get-contexts
$ kubectl config use-context

Above all, please help information useful to resolve the trouble.

Jenkins Pipeline parallel jobs

First: awesome plugin!

But unfortunately I am running into a problem:

I have a Jenkins pipeline job with multiple parallel subjobs. They all should use the same ACI container template and consequently spin up multiple container instances (groups?) so that they can run in parallel with each of them having their own reserved cpu and memory. See this Jenkinsfile example. But the following happens:

  • the parallel subjobs request an ACI slave
  • one such ACI (group?) is created in Azure
  • both jobs wait for the container to come online
  • the first subjob gets the single executor on that slave once the container is started
  • once the container is started the first subjob builds
  • second subjob requests another slave
  • second subjob waits for the new container to come online
  • once the second container is started the second subjob builds

Between the start of the two containers around 3 minutes pass but they should be started at the same time ideally.

To solve this there should be the option to create a dedicated slave for each request. Is this possible?

Thanks!

Allow me to push code for Private IP

Hi,

I managed (at least think I did) to create the code for a new parameter in the cloud template config to allow choosing between Private or Public IP. I still have an error with the style that you might figure out.
If you allow me to upload the new branch to create a PR, you might fix that in a minute and get this feature done.
Regards

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.