tiangolo/docker-with-compose

Docker image with Docker Compose installed for CI.

Hey! I'm @tiangolo (Sebastián Ramírez) 👋

I'm a software developer from Colombia. 🇨🇴

I currently live in Berlin, Germany. 🇩🇪

I have been building APIs and tools for Machine Learning and data systems, in Latin America, the Middle East, and now Europe, with different teams and organizations. 🌎

I created FastAPI, Typer and a bunch of other open source tools. 🚀

I like to build things with Deep Learning/Machine Learning, distributed systems, SQL and NoSQL databases, Docker, Python, TypeScript (and JavaScript), modern backend APIs, and modern frontend frameworks. 🤖

I'm currently dedicating a high percentage of my time to FastAPI, Typer, and my other open source projects. At the same time, I'm also helping a limited number of teams and organizations as an external consultant. If you would like to have my help with your team and product, feel free to contact me. 🤓

If my open source projects are useful for your product/company you can also sponsor my work on them. ☕

You can find me on:

• My website: tiangolo.com
• Twitter as @tiangolo
• GitHub as @tiangolo (you are here)
• LinkedIn
• Dev.to
• Medium

docker-with-compose's Issues

Stack generation have breaking changes from 2021-09-18 to latest

The latest push did break some of our deployments.

Initially posted in #32 but does not per se have the same issue.

Before:

FROM tiangolo/docker-with-compose:2021-09-18
WORKDIR /tmp
COPY docker-compose.yml ./
CMD ['docker-compose', '-f', 'docker-compose.yml', 'config']

After:

FROM tiangolo/docker-with-compose:latest
WORKDIR /tmp
COPY docker-compose.yml ./
CMD ['docker-compose', '-f', 'docker-compose.yml', 'config']

Basic file input docker-compose.yml

version: '3.8'
 
services:
  db:
    image: postgres:14-alpine
    environment:
      POSTGRES_USER: api
      POSTGRES_PASSWORD: azerty
      POSTGRES_DB: api
    volumes:
      - app-db:/var/lib/postgresql/data
 
volumes:
  app-db:

Output before:

services:
  db:
    environment:
      POSTGRES_DB: api
      POSTGRES_PASSWORD: azerty
      POSTGRES_USER: api
    image: postgres:14-alpine
    volumes:
    - app-db:/var/lib/postgresql/data:rw
version: '3.8'
volumes:
  app-db: {}

Output now:

name: tmp
services:
  db:
    environment:
      POSTGRES_DB: api
      POSTGRES_PASSWORD: azerty
      POSTGRES_USER: api
    image: postgres:14-alpine
    networks:
      default: null
    volumes:
    - type: volume
      source: app-db
      target: /var/lib/postgresql/data
      volume: {}
networks:
  default:
    name: tmp_default
volumes:
  app-db:
    name: tmp_app-db

• version: '3.8' is not present on the generated stack.yml (the target host fallback to version: 1 which is problematic. even though it is on our (input) files.
• volumes descriptions have changed unexpectedly so there are re-created (not found)
• it seems that this image generates a compose-compliant (v2) schema file while the command docker stack is not ready.

from remote, not actually the docker-with-compose image

docker stack deploy -c docker-stack.yml --with-registry-auth "xxxx"
unsupported Compose file version: 1.0

hint: the docker-stack.yml seems to be generated using compose v2 instead of docker-compose legacy version.

Typo in the README.md file

Inside the Description section of the README.md file, there is a typo:

The main purpose of this image is to help in Continuous Integration environments that need the docker binary, the docker-compose binary and posibly (possibly) require doing other things, like running Bash scripts.

Docker-compose now requires Alpine cargo package

The Dockerfile fails to build now because apt-get install cargo is required for docker-compose. See:

https://docs.docker.com/compose/install/#alternative-install-options

GitLab CI/CD - Couldn't connect to docker daemon

Hi everyone
I just tried to use this useful image but getting error trying a simple docker-compose build ?

.gitlab-ci.ymlseems to me like copy-pasted from official docs, not sure I'm missing something? pretty new to GitLab builds, so I could miss something obvious.

Output from CI:

thanks
nicola

New version of Docker Compose: 1.26.2

When I run docker-compose -version on my Mac I get:

1.26.2, build eefe0d31

Inside the tiangolo/docker-with-compose container I get:

1.26.0

I have an issue using contexts that only repros in the container, and I assume it's from the version difference. When I built the container manually, i.e. ran docker:latest then ran apk add --no-cache py3-pip python3-dev libffi-dev openssl-dev curl gcc libc-dev make && pip3 install docker-compose my issue disappeared.

Can you rebuild your docker image, to get the newest docker-compose included?

Not working with gitlab

This image wasn't working for me in gitlab. (I'm running tests for the fastAPI project generator library) I was using the following:

image: tiangolo/docker-with-compose

before_script:
  - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
  - docker info
  - pip install docker-auto-labels

variables:
  DOCKER_TLS_CERTDIR: ""

stages:
  - test

tests:
  stage: test
  script:
    - sh ./scripts/test.sh

But would get the following error: error during connect: Post http://docker:2375/v1.40/auth: dial tcp: lookup docker on 169.254.169.254:53: no such host (see the job here)

Any ideas why this image isn't working? Is it due to the TLS changes with docker:dind?

Using the official docker/compose image, and changing the entrypoint, is working for me:

image:
  name: docker/compose:1.25.1-rc1
  entrypoint: ["/bin/sh", "-c"]

variables:
  DOCKER_TLS_CERTDIR: ""
  DOCKER_HOST: tcp://docker:2375/
  DOCKER_DRIVER: overlay2

services:
  - docker:dind

before_script:
  - docker info
  - docker-compose version

stages:
  - test

tests:
  stage: test
  script:
    - sh ./scripts/test.sh

proposal: include `docker-auto-labels`

In full-stack-fastapi-postgresql cookiecutter repo docker-auto-labels were used, and I think that including this package could speedup each CI pipeline a bit.

Bitbucket Pipelines: Error response from daemon: authorization denied by plugin pipelines: --privileged=true is not allowed

First of all, thanks a lot for this project.

Just wanted to let you (and others) know that the latest version (latest / 2022-11-10) doesn't work in Bitbucket Pipelines anymore:

+ docker-compose build
#1 [internal] booting buildkit
#1 pulling image moby/buildkit:buildx-stable-1
#1 pulling image moby/buildkit:buildx-stable-1 2.6s done
#1 creating container buildx_buildkit_default 0.0s done
#1 ERROR: Error response from daemon: authorization denied by plugin pipelines: --privileged=true is not allowed
------
 > [internal] booting buildkit:
------
Error response from daemon: authorization denied by plugin pipelines: --privileged=true is not allowed

I switched back to version 2021-09-18, which works fine.
Not sure if this is anything you can fix at all.

Bash

The README says:

This image allows running arbitrary commands like Bash scripts,

But the image does not contain bash. I propose we apk add bash.

Running docker-compose on python3 would work better

docker-compose has some annoying bugs running on python2, e.g. not seeing log output of services that have been idle for some time

Furthermore, docker-compose is considering deprecating using it on Python 2 altogether.

We should use a base image with Python 3 and run docker-compose on that instead of Python 2.

Different dependencies installed with apk

Running apk to install nodejs no longer installed npm depdendency.

Note that nghttp2-libs is installed, and nodejs-npm is not.

Left: Using current image (tiangolo/docker-with-compose:latest)
Right: Using image from a day ago

According to the docker-with-compose build details, the build was just updated.

Different versions of alpine are noted in the fetch comments.

/usr/local/bin/docker-entrypoint.sh: exec: line 61: /bin/bash: not found

/usr/local/bin/docker-entrypoint.sh: exec: line 61: /bin/bash: not found

update docker-compose to 1.25

Can't access compose container port on localhost in Gitlab CI

Hi,

I have a docker-compose.yaml that runs the Keycloak SSO server and Postgresql. The Keycloak HTTP server listens on port 8080 and so I have

-ports
  8080:8080

in the docker-compose.yaml.

In my .gitlab-ci.yml I do: docker-compose -f path/to/docker.compose.yaml up -d. I then start a watchdog so that I know when Keycloak is available by doing a curl -I http://localhost:8080 in a loop-with-sleep checking curl's exit status.

However, I get connection refused even when I know the Keycloak is up and running. I'm sure there's something I don't understand here. What is it?

add make

Currently, I have to install make everytime I want to run make commands. IMO, a good amount of people may be using make to run docker-compose commands.

Is it out of the scope to add the make installation to this package?

This is my current .gitlab-ci.yml file

image: tiangolo/docker-with-compose

services:
  - docker:dind

before_script:
  - apk add --update make

stages:
  - lint
  - test

lint:
  stage: lint
  script:
    - make ci-lint

test:
  stage: test
  script:
    - make ci-test

docker-compose up -d fails

Error -

Couldn't connect to Docker daemon at http+docker://localhost - is it running?

If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.

[FEATURE] Document how to mount the Docker sock

Document how to mount the Docker sock as a volume so that Docker Compose inside the container can have access to the local Docker service/daemon.

This has to be done instead of Docker-in-Docker: http://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/

Probably with the ideas here: https://medium.com/better-programming/about-var-run-docker-sock-3bfd276e12fd

CURL

can you add curl installation? may with same tag on docker image

FROM docker:latest

RUN apk add --no-cache curl
RUN apk add --no-cache py-pip
RUN pip install docker-compose

proposal: include `gettext`

gettext allows me to use envsubst which is nice for configuring rendering docker-compose.yml with external secrets, networks etc as their names better to be stored as CI variables rather as hardcoded value in docker-compose.yml.

somehow not working anymore with gitlab

I had your image (thanks very much) set up with a private gitlab instance, but it just stopped working. Though there are no obvious changes on the server where it is executed nor in your repo. But maybe you have some idea?
config.toml and erros message below.

Running with gitlab-runner 11.11.2 (ac2a293c)
  on homeserver something
Using Docker executor with image tiangolo/docker-with-compose ...
Starting service docker:dind ...
Pulling docker image docker:dind ...
Using docker image sha256:6ce0d31cf4d670f84b0f74b1b15910f9e3fde4105232e14b0d360a6a49c05362 for docker:dind ...
Waiting for services to be up and running...
Pulling docker image tiangolo/docker-with-compose ...
Using docker image sha256:cf0fa05056110e8c9866200f6879b615ea55be3e54f558826aab5b0882537f70 for tiangolo/docker-with-compose ...
Running on runner-something-0 via ubuntu...
DEPRECATION: this GitLab server doesn't support refspecs, gitlab-runner 12.0 will no longer work with this version of GitLab
Fetching changes...
Checking out 13232142 as somebranch...

Skipping Git submodules setup
$ docker-compose up -d
Couldn't connect to Docker daemon at http://docker:2375 - is it running?

If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.
ERROR: Job failed: exit code 1

config.toml

concurrent = 1
check_interval = 0

[session_server]
session_timeout = 1800

[[runners]]
name = "homeserver"
url = "https://gitlab.something.eu/"
token = "<sometoken>"
executor = "docker"
[runners.custom_build_dir]
[runners.docker]
tls_verify = false
image = "tiangolo/docker-with-compose"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
[runners.cache]
  [runners.cache.s3]
  [runners.cache.gcs]

Time ot sunset this repo?

Now that compose has been added into the official docker image (docker-library/docker#361) what is the purpose of this repository anymore?

Is it time to tell folks to stop using it? Should it become a plain mirror for docker:latest?