docker-archive/cloud-integration-beta

Docker CLI with ACI integration (beta)

cloud-integration-beta's Issues

Not able to install the script (install.sh)

I ran the following given command in the document

curl -L https://raw.githubusercontent.com/docker/aci-integration-beta/main/scripts/install_linux.sh | sh

It gives me below error
Checks passed!
Downloading CLI...
curl: no URL specified!
curl: try 'curl --help' or 'curl --manual' for more information

I also tested the given Dockerfile-testInstall
It is also giving same error.

I guess the issue is in the below lines

# Download CLI to temporary directory
download_dir=$($sh_c 'mktemp -d')
$sh_c "${download_cmd} ${download_dir}/docker-aci ${DOWNLOAD_URL}"

Might be the variables aren't populating

Missing support to environment variables

Hi,
I don't know if this is a bug or a not supported feature but when I try to deploy an example compose file for a wordpress the system can't start. The problem is that the ACI doesn't set any environment variable:

I used this yml to deploy the aci:

version: '3.4'

services:
   db:
     image: mysql:5.7
     restart: always
     environment:
       MYSQL_ROOT_PASSWORD: somewordpress
       MYSQL_DATABASE: wordpress
       MYSQL_USER: wordpress
       MYSQL_PASSWORD: wordpress

   wordpress:
     depends_on:
       - db
     image: wordpress:latest
     ports:
       - "80:80"
     restart: always
     environment:
       WORDPRESS_DB_HOST: db:3306
       WORDPRESS_DB_USER: wordpress
       WORDPRESS_DB_PASSWORD: wordpress
       WORDPRESS_DB_NAME: wordpress

Is this feature in the roadmap?

"docker logs --follow" output garbled

Prereqs

Docker Desktop Edge build 46511
Windows Terminal 1.1.1671.0 (I install Windows Terminal via Scoop)

Repro steps

1. Set up Windows Terminal so that it displays about 50 rows and 120 columns
2. Deploy some container to ACI
3. Do docker logs --follow container_name

Expected

The logs should be displayed as they are produced, without overwriting previous content in the terminal

Actual

The output of the command overwrites docker logs command invocation and even stuff that came before. Also, the same logs are re-printed repeatedly, garbling the previous output, see below.

 * Serving Flask app "app" (lazy loading)*****************************
 * Environment: productionloper Command Prompt v16.6.3
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.******************************
 * Debug mode: offs a development server. Do not use it in a production deployment.
 * Running on http://0.0.0.0:31117/ (Press CTRL+C to quit)
10.240.255.56 - - [07/Jul/2020 23:24:30] "GET / HTTP/1.1" 200 -oduction deployment.       PORTS
10.240.255.55 - - [07/Jul/2020 23:24:30] "GET /favicon.ico HTTP/1.1" 404 -ing             40.91.90.144:31117->31117/tcp
10.240.255.56 - - [07/Jul/2020 23:25:50] "GET / HTTP/1.1" 200 -oduction deployment.
10.240.255.55 - - [07/Jul/2020 23:25:53] "GET / HTTP/1.1" 200 -/1.1" 404 -
10.240.255.56 - - [07/Jul/2020 23:25:55] "GET / HTTP/1.1" 200 -oduction deployment.
10.240.255.56 - - [07/Jul/2020 23:26:02] "GET / HTTP/1.1" 200 -/1.1" 404 -
10.240.255.56 - - [07/Jul/2020 23:26:03] "GET / HTTP/1.1" 200 -oduction deployment.
10.240.255.55 - - [07/Jul/2020 23:26:04] "GET / HTTP/1.1" 200 -/1.1" 404 -
10.240.255.56 - - [07/Jul/2020 23:26:05] "GET / HTTP/1.1" 200 -
10.240.255.56 - - [07/Jul/2020 23:26:05] "GET / HTTP/1.1" 200 -/1.1" 404 -
10.240.255.55 - - [07/Jul/2020 23:26:06] "GET / HTTP/1.1" 200 -
10.240.255.56 - - [07/Jul/2020 23:26:06] "GET / HTTP/1.1" 200 -/1.1" 404 -
10.240.255.55 - - [07/Jul/2020 23:26:07] "GET / HTTP/1.1" 200 -
10.240.255.55 - - [07/Jul/2020 23:26:08] "GET / HTTP/1.1" 200 -
10.240.255.56 - - [07/Jul/2020 23:26:08] "GET / HTTP/1.1" 200 -
10.240.255.55 - - [07/Jul/2020 23:26:08] "GET / HTTP/1.1" 200 -
10.240.255.56 - - [07/Jul/2020 23:26:08] "GET / HTTP/1.1" 200 -
10.240.255.55 - - [07/Jul/2020 23:26:08] "GET / HTTP/1.1" 200 -
10.240.255.56 - - [07/Jul/2020 23:26:08] "GET / HTTP/1.1" 200 -
10.240.255.55 - - [07/Jul/2020 23:26:08] "GET / HTTP/1.1" 200 -
10.240.255.56 - - [07/Jul/2020 23:26:08] "GET / HTTP/1.1" 200 -
10.240.255.55 - - [07/Jul/2020 23:26:08] "GET / HTTP/1.1" 200 -
10.240.255.56 - - [07/Jul/2020 23:26:08] "GET / HTTP/1.1" 200 -
10.240.255.55 - - [07/Jul/2020 23:26:08] "GET / HTTP/1.1" 200 -
10.240.255.56 - - [07/Jul/2020 23:26:08] "GET / HTTP/1.1" 200 -

Why aren't all the existing Docker commands and flags available with ACI

Some of the Docker commands and flags are specific to local development and workflows and so they are not applicable to ACI. Examples include docker image commands. There are also commands that we plan to support in the future but just haven't got around to yet.

Please let us know if there are commands or flags that you would like to see!

How to authenticate with private ACR registry?

Running docker compose up on a docker-compose.yml file with images located in a private ACR registry results in a ACI error:

containerinstance.ContainerGroupsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="InvalidImageRegistryPassword" Message="The password in the 'imageRegistryCredentials' of container group 'test' cannot be empty."

Or running a single container:

$ docker run ***.azurecr.io/poc/nginx
[+] Running 0/1
 ⠧ wizardly-meitner  Waiting                                                                                                                                                            0.7s
containerinstance.ContainerGroupsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="InvalidImageRegistryPassword" Message="The password in the 'imageRegistryCredentials' of container group 'wizardly-meitner' cannot be empty."

With pure ACI ARM template its possible to define image registry credentials like:

"imageRegistryCredentials": [
  {
    "server": "imageRegistryLoginServer",
    "username": "imageRegistryUsername",
    "password": "imageRegistryPassword"
  }
]

How to authenticate with a private ACR registry when deploying to ACI?

Docker Login Azure issues

When running 'docker login azure' on a VM in azure, it asks for a username and pass from stdin instead of opening the browser to login. Any ideas why this might be happening and how to fix it? Its a windows 10 VM with docker for desktop installed.

WSL2 Azure Login not portable accross various WSL setups

Description

WSL2 Azure login opens a browser to perform Azure login steps. Currently it opens a browser with wslview <url>. This is not portable accross WSL setups and sometimes blocks users to log into Azure.
We need to

• potentially find a more portable way to open a browser from WSL2
• as a fallback, use Azure Device code flow to not start a browser but ask the user to manually open a web page & copy an authentication code, like the az command line does on WSL2

Steps to reproduce the issue:

Thanks @karolz-ms for repro details :
On my Ubuntu 20.04 distro:

karolz@karolzp1:~/code$ wslview http://www.microsoft.com
(nothing happens)
karolz@karolzp1:~/code$ echo $?
1

On my Ubuntu 18.04 distro:

karolz@karolzp1:~/code$ wslview http://www.microsoft.com
wslview: command not found

☹ I am pretty sure this is due to me disabling the Windows interop. Just FYI:

karolz@karolzp1:~/code$ cat /etc/wsl.conf 
[interop]
enabled=false
appendWindowsPath=false

Output of docker version:
tested on build shared with MSFT v0.1.12-alpha1

How do I build an image when using an ACI context

At the moment you can only build images using a local context and then push them to a registry so that they can be used by ACI. We are thinking about how to make this experience better but would love your ideas for what you would like to see!

ACR Auto-login not working in some scenario

ACR Auto-login has been implemented following this doc that describes how to exchange the Azure mgmt token into a token that ACR accepts: https://github.com/Azure/acr/blob/master/docs/AAD-OAuth.md#getting-credentials-programatically.

Some users at Microsoft are seeing errors while trying to deploy ACR images:
@BigMorty:

Not working for me, here is what I did and the error I received...

C:\Users\mikemort\source\repos\OrderProc>docker logout mikemortacr.azurecr.io
Removing login credentials for mikemortacr.azurecr.io

C:\Users\mikemort\source\repos\OrderProc>docker logout azure
Removing login credentials for Azure

C:\Users\mikemort\source\repos\OrderProc>docker login azure
login succeeded

C:\Users\mikemort\source\repos\OrderProc>docker context use MyACI
MyACI

C:\Users\mikemort\source\repos\OrderProc>docker run -d mikemortacr.azurecr.io/myexpressapp:2.2
Could not automatically login to mikemortacr.azurecr.io from your Azure login. Assuming you already logged in to the ACR > registry
[+] Running 0/1

Group goofy-rhodes Waiting 3.0s
containerinstance.ContainerGroupsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="InaccessibleImage" Message="The image 'mikemortacr.azurecr.io/myexpressapp:2.2' in container group 'goofy-rhodes' is not accessible. Please check the image and registry credential."
C:\Users\mikemort\source\repos\OrderProc>

@gtardif :

To help debugging this, could you try to manually run the call to obtain the ACR token (as described here), I assume in your context it will fail, the error message might help.

registry="contosoregistry.azurecr.io"
tenant="409520d4-8100-4d1d-ad47-72432ddcc120"
aad_access_token="eyJ...H-g"
curl -v -X POST -H "Content-Type: application/x-www-form-urlencoded" -d \
    "grant_type=access_token&service=$registry&tenant=$tenant&access_token=$aad_access_token" \
    https://$registry/oauth2/exchange

You can get the values for tenant & access token from ~/.azure/dockerAccessToken.json

@BigMorty:

Here is the error I received - {"errors":[{"code":"UNAUTHORIZED","message":"retrieving permissions failed"}]}

Support for Azure account with multiple tenants

Context

My Azure account have 2 tenants :

• the corporate and main one, used only for AD, with no subscription
• my team's tenant, with multiple subscriptions

$ docker version
Client: Docker Engine - Community
 Azure integration  0.1.4
 Version:           19.03.12
 API version:       1.40
 Go version:        go1.13.10
 Git commit:        48a66213fe
 Built:             Mon Jun 22 15:41:33 2020
 OS/Arch:           darwin/amd64
 Experimental:      true

Issue

The docker context create aci command seems to use only the first one, and I have not found a option to change it :

$ docker context create aci azure
no subscriptions found

Proposition

When using the az account list, there is homeTenantId property, this is what I want to be able to configure

$ docker login azure --tenant-id <tenant-id>
# or
docker context create aci azure --tenant-id <tenant-id>

Alternative proposition

The docker client should iterate over all account's tenants when searching for subscriptions, as the az client does.

Cannot map container ports on ACI

ACI does not support port mapping so commands like docker run -d -p 8080:80 nginx will fail. You will need to make sure that your docker run commands specify the same container and host ports. e.g.: docker run -d -p 80:80 nginx.

The same applies for services defined in a Compose file.

Docker Start/Stop Support

Previously we had no docker start or docker stop support in the Docker ACI integration. This was because ACI does not support a stop or start commands in the same way that Moby does.

We have now added these commands accepting the difference to get feedback on whether this is an acceptable approach.

From a user perspective the main difference with ACI is that when your container that has been previously stopped is started again it will have not retained the state within the container. You will still retain your IP address as this is maintained by the container group and you can still inspect the logs of the stopped container.

If you find this difference acceptable or have a use case where this does not work please let us know on this issue.

As a note generally container state is best stored in a volume, to find out more about Docker volumes see https://docs.docker.com/storage/volumes/

Open sourcing of the ACI integration

The first release of the ACI integration is binary only and does not include the source code. We chose to do this because we need to get the code into a state that those external to Docker can contribute before we can open source it. This includes making sure we have a clear roadmap, documentation for contributing, etc.

Docker will open source the code in the near future.

unsupported Compose file version: 2.3

Attempting to run docker compose up, getting an error returned "unsupported Compose file version: 2.3"

System info:

➜ docker version
Client: Docker Engine - Community
 Azure integration  0.1.4
 Version:           19.03.12
 API version:       1.40
 Go version:        go1.13.10
 Git commit:        48a66213fe
 Built:             Mon Jun 22 15:41:33 2020
 OS/Arch:           darwin/amd64
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          19.03.12
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.10
  Git commit:       48a66213fe
  Built:            Mon Jun 22 15:49:27 2020
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          v1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

➜ docker-compose --version
docker-compose version 1.26.0, build d4451659

What is the minimum docker compose file version that's supported in this ACI context?

Support for service principal authentication

In order to use Docker CLI integration with ACI in CI/CD scenarios, docker login azure should support authentication using service principal name/password (a.k.a. application ID/secret).

Sample scenario (using Azure CLI): https://docs.microsoft.com/en-us/azure/container-instances/container-instances-github-action

This is how equivalent Azure CLI command looks like:

# Log in with a service principal using client secret. Use -p=secret if the first character of the password is '-'.
az login --service-principal -u http://azure-cli-2016-08-05-14-31-15 -p VerySecret --tenant contoso.onmicrosoft.com

I believe the relevant Azure Golang SDK method to get the token is acquireTokenClientSecretFlow: https://github.com/Azure/go-autorest/blob/master/autorest/adal/cmd/adal.go#L120

Add the ability to connect Azure file shares via Docker Compose

You can already connect Azure file resources as volumes via flag "-v" by "docker run". Similarly, you want to do this in Docker Compose.

Support for Compose v2

With the Compose specification removing the requirement for the version field in a Compose file, Compose v2 and v3 syntax will be supported by the ACI integration. This is a work in progress but should arrive in the near future.

Initial question about this was posted here: #13

Support for Runtime options with Memory, CPUs, and GPUs

Please add / clarify support for Runtime options with Memory, CPUs, and GPUs.

For example:

version: "3.8"
services:
  redis:
    image: redis:alpine
    deploy:
      resources:
        limits:
          cpus: '0.50'
          memory: 50M
        reservations:
          cpus: '0.25'
          memory: 20M

Links

1. Resource availability for Azure Container Instances in Azure regions

Images can't be listed when using an ACI context

ACI pulls images directly from a container image registry and does not have a cache like that when you work locally. This means that docker image commands are not applicable to ACI.

docker login azure @ WSL2: xdg-open executable file not found in $PATH

I am not able to login to azure from WSL2:

docker login azure
exec: "xdg-open": executable file not found in $PATH

Client: Docker Engine - Community
Azure integration 0.1.7
Version: 19.03.12

Docker Desktop Community
Version: 2.3.3.1 (46608)
Channel: edge

no subscriptions found

Greetings,

really wanted to try this, but I am having trouble with no subscriptions found here are the logs, and information of my environment

jeanepaul@DESKTOP-T3KIFDC MINGW64 ~
$ az login
You have logged in. Now let us find all the subscriptions to which you have access...
The following tenants don't contain accessible subscriptions. Use 'az login --allow-no-subscriptions' to have tenant level access.
[
  {
    "cloudName": "AzureCloud",
    "homeTenantId": "c879b447-9284",
    "id": "86bf3679",
    "isDefault": true,
    "managedByTenants": [
      {
        "tenantId": "2f4a9838"
      }
    ],
    "name": "Microsoft Azure Sponsorship",
    "state": "Enabled",
    "tenantId": "c879b447",
    "user": {
      "name": "j.soliva",
      "type": "user"
    }
  }
]

jeanepaul@DESKTOP-T3KIFDC MINGW64 ~
$ az account list -o table
Name                         CloudName    SubscriptionId                        State    IsDefault
---------------------------  -----------  ------------------------------------  -------  -----------
Microsoft Azure Sponsorship  AzureCloud   86bf3679                              Enabled  True

jeanepaul@DESKTOP-T3KIFDC MINGW64 ~
$ docker context create aci azurecontext
no subscriptions found

$ docker version
Client: Docker Engine - Community
 Azure integration  0.1.10
 Version:           19.03.12
 API version:       1.40
 Go version:        go1.13.10
 Git commit:        48a66213fe
 Built:             Mon Jun 22 15:43:18 2020
 OS/Arch:           windows/amd64
 Experimental:      false

Thank you!