compose-spec/compose-spec

The Compose specification

Compose Specification

{:.no_toc}

The Compose specification establishes a standard for the definition of multi-container platform-agnostic applications. The specification can be found here.

• ToC {:toc}

Use cases

To provide more context on the Compose specification the following section gives example use cases for each part of the specification.

Development tools

Developers can use a Compose file to specify a container-based application that will run as a set of containers on a local container engine. The Compose implementation in this scenario could offer some specific features (local volume binding, live-reload) to better address development needs. The application definition and Compose model is the same used as that used for other use cases. Platform features expected by the specification (like configs and secrets) can be mocked with local resources.

Kubernetes deployment

Kubernetes container orchestration relies on a set of abstract concepts and APIs to manage networking services, container deployments and their lifecycles. While this offers flexibility to address many operator use cases, it makes simple use cases, like the developer use case, more complicated to express than they need to be. Projects like Kompose or Okteto Stacks show how the simpler Compose model can be translated into Kubernetes API payloads and make the Compose file the source of truth for development and deployment.

Cloud providers

Some cloud providers offer proprietary container hosting solutions based on in-house orchestrators and custom APIs. The Compose specification offers a simple model that can be mapped to these hosting solutions so that users can reuse Compose files that they already have and so that they do not need to learn custom configuration file formats. Platform specific features can be added either using Compose extensions or a dedicated configuration file along side the Compose file.

Contributing

Development happens on GitHub for the specification. Issues are used to track bugs and actionable items. Longer discussions can happen on the mailing list.

The specification and code is licensed under the Apache 2.0 license found in the LICENSE file.

Implementations

Docker Compose is the Reference Implementation of the Compose Specification.

Compose Specification is also implemented by:

• Kompose
• Nerdctl
• Okteto Stacks
• Docker Cloud Integrations
• Podman Compose

Metadata
Status	Work in progress
Created	2020-01-02

compose-spec's Issues

Singularity Compose Comparison

As discussed in #57 (comment), singularity-compose is a simple orchestration tool for Singularity containers and I would like to review the shared (and unshared) parts of the (now specification!) that is defined here for compose.

Version

version is also defined for Singularity Compose. And like docker-compose, it's not currently using semver (x.x.x) but instead x.x. There hasn't been a second version yet, so (unlike the specification here warants) we don't do any checks / exit for the version string. I can easily add this:

Tools not implementing required version MUST reject the configuration file.

if it's suggested best practice.

Services

Singularity compose uses "instances" as a shoe in for services, and the reason is because "instances" is the lingo the community is used to. It's the same thing however - a list of containers being run as services (singularity instances). I could also allow this as a field specified, although then I'd need to check for one or the other and exit on fail. If you validated a singularity-compose.yml file against the schema here it wouldn't pass.

Build

Within the service definition, Singularity Compose also has bulid to indicate building from a recipe, and under build has context to indicate a build context. Instead of dockerfile since it's referred to as a "Singularity recipe" we just have recipe. Here is an example:

  nginx:
    build:
      context: ./nginx
      recipe: Singularity.nginx

The only other supported argument under build is options, so a user can specify a list of options. Each one can be a key value pair to pass to singularity(--key value) or a single string to pass as a flag (e.g., fakeroot).

Image

The other option for a container is to go from an image, and the only difference here is that the namespace is intended for Singularity. E.g., when we pull a docker image we do singularity pull docker://busybox so a recipe might look like:

  nginx:
    image: docker://vanessa/sregistry_nginx

Networks

We don't have a concrete section for networks - it's primarily controlled / created if the user specifies ports (or not) and depending on the permission level that singularity-compose is run with (e.g., some singularity instance commands require sudo, some can be done with --fakeroot). You can see the command issued under the hood here.

Volumes

Volumes looks essentially the same:

    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf
      - ./app:/code
      - ./static:/var/www/static
      - ./images:/var/www/images

But without support for a mount level (e.g., :ro for read only.) I also added volumes_from which I think used to be in compose (but isn't anymore).

Secrets and Config

There is no support for secrets or config in the singularity-compose.yml. If a user needs to add an environment secret, the easiest thing to do is bind a file to where it will be discovered in the container. I've never actually used "config" in docker-compose either so I'm not familiar with it!

Ports

Ports are largely the same, but just in the list format with - "8000:8000" - there aren't special options / additional args beyond that.

Post

This is the one grouping that I defined that is entirely missing from compose. The reason is that I wanted to support additional commands (run outside the containers or build) after it, (e.g., a change in permissions for some mounted folders, or something like that) so I added post, which can also have post.commands. For example, here is downloading a file for use after building the services, and here we are creating folders on the host (with correct permissions) for the Nginx upload module. It's probably something that someone could put in a separate script and then run a separate command after, but I added it so the builds (singularity-compose up) could be completely reproducible.

Others

There isn't any definition for healthchecks, adding permissions, deployment, etc. It's the sort of thing where I implemented the absolute minimal needed, and I only add more features as they are requested. I suspect since the build.options exists, much customization is handled there. You can see the entire specification here. Let me know if this is what you had in mind for me to open an issue about!

Need support for non-CPU accelerators in docker-compose

What is the problem you're trying to solve
The docker-compose specification does not support mixing GPU and CPU containers on the same machine. The runtime: tag that allowed us to do this was removed in 3.x. However, I believe that this is reflective of a larger problem that I'd like to see addressed. Devices will have multiple "accelerators" in addition to CPUs such as Nvidia GPU, AMD GPU, ARM GPU (pi), Google TPU, Intel TPU,... etc etc. Consequently, we need a clear way to request that a container gain access to a particular accelerator and that the orchestrator respects this when scheduling the container. Ideally orchestrators like Kube should also be able to play here.

Describe the solution you'd like
A run time constraint that describes the kind of accelerator needed will be ideal and the plumbing needed for the container to map the accelerator specified should be set up when the orchestrator schedules the container onto the machine hosting the specified accelerator.

Additional context
Please see Github Issue: Nvidia docker-compose

Call for maintainers

Hey All,

Since the Compose Specification is now being handed over to the community there are a lot of potential changes that have been proposed already and we suspect more coming as people start to get involved.
To help manage these and support the growing community we are looking for about 3 maintainers of the Compose Specification.

We request developers who use and love Compose today, who have experience building developer tooling and prefer those who have previous experience working on Open Source Specifications previously. Experience working with Compose outside of local development is also preferred with users of Compose on K8s, Kompose or services like ECS a plus.
For all considering being maintainers having an interest in how to optimize the Compose format for developers locally while not overloading it or just rebuilding KubeYaml is a must. Compose is loved as it is lightweight and simple to use - we all need to maintain this going forward.

Maintenance entails:

• Responding to new issues
• Supporting fortnightly community discussion
• Being active and engaged on the Slack channel
• Supporting wider initiatives such as submissions to Open source foundations
• Optionally, adding new features or suggestions of your own!

If you wish to request being a maintainer then please drop a message on the Compose Specification slack channel on the CNCF Slack.

If you are unsure feel free to ask more questions here.

Clarify where CI falls within the Vision

What is the problem you're trying to solve
Currently, Vision document relies on contrasting local development with production deployment. This (rather easily) outlines which aspects of Compose belong fall into which bucket. For example, Container runtime configuration is described as falling into Deploy category. When CI is added to the mix, things get rather muddied up and the separation isn't as clear any more. Considering that docker-compose is great for CI, we should discuss how we can include this type of usage in the vision.

Describe the solution you'd like
Mention CI use case in the vision, possibly updating the Scope table along the way.

Process for spec changes?

Should we define a spec change process like the Python PEPs https://www.python.org/dev/peps/?

I am wondering how we will version the spec or test conformity. Checking by which enhancements are implemented is one way that works quite well.

Define "User"

What is the problem you're trying to solve
The specification should clearly outline User configuration options

Describe the solution you'd like
Something along the lines of Username or UID (format: <name|uid>[:<group|gid>])

Additional context
Add any other context about the feature request here.

Add option to specify services to be started by default

The Problem / Use Case

docker-compose is often used as a simple way to setup and run an application locally for development purposes. For this not always all defined services are required to run - or additional services may be useful for development. But starting the application with a simple docker-compose up without explicitly specifying a list of services always starts all defined services.

For example: I like to define an additional service running phpMyAdmin alongside a mysql database service in order to have a convenient way to inspect and manipulate the database in development. But this service should only ever run in development and may even there be optional/only started on-demand.

Currently the recommended way for this use case is to split the services into multiple files and pass them on the command line

docker-compose -f docker-compose.dev.yml up

Since the additional services may depend on (e.g. the database service) or extend already defined services this may require to combine multiple files when starting the development environment:

docker-compose -f docker-compose.yml -f docker-compose.dev.yml up

In an automated environment this is no issue since it just needs to be scripted once. But in a development environment this is a huge inconvenience for the developer:

• the command lines become very long to type instead of a simple docker-compose up
• the developer has to remember/lookup which services are defined in which files and pass them as parameters
• the same set of files needs to be passed to every docker-compose command since services defined in other files will be ignored, i.e. one can not just stop the application with docker-compose stop/docker-compose down since the additional services will be ignored

This is a real pain for developers which is why various solutions to this issue were repeatedly requested over years, see linked issues below.

The desired new feature

There should be a simple way to define the set of "default services" that are started when docker-compose up is called without any arguments.
This way additional services can still be defined in the same docker-compose.yml file which can then be started on-demand:

# start set of "default services"
docker-compose up -d

# start additional service for development
docker-compose up phpmyadmin

There are various ways the default set could be specified in the docker-compose.yml file:

1. provide a file-wide (white)list of services to be started by default, which when not specified defaults to all defined services
2. specify a (black)list of services which should not be started by default but only when explicitly listed or when pulled in as a dependency
3. specify on a per service basis if it should be included in the default set or not; when not specified, the service will be included

Additional context

This or a similar feature was requested for docker-compose repeatedly and in the discussions many more example use cases were explained by the users showing this is a real pain point for them:
docker/compose#6742 (most recent discussion), docker/compose#1896 (main discussion), docker/compose#697, docker/compose#912, docker/compose#1294, docker/compose#1439, docker/compose#1547, docker/compose#2803, docker/compose#3027, docker/compose#3289

Further related issues: docker/compose#1661, docker/compose#1988, docker/compose#2773, docker/compose#3684, docker/compose#4096, docker/compose#4650, docker/compose#6676

Personal note

I am missing such a feature with docker-compose since a very long time. I think it can be implemented with quite reasonable effort but provides an immense improvement convenience-wise for local development.
One possible solution I already have implemented which I will explain in the comments.

I'm very committed to making this feature happen whether with my proposed solution or any other different way which I'm also open too. My desired outcome is the improved developer-convenience for the presented use case.
So I am grateful for any feedback and suggestions!

Add toc to the spec

What is the problem you're trying to solve
add toc to the spec file

Create a "use cases" document

What is the problem you're trying to solve
Specification would benefit a "usages" (non-normative) document for community to describe the various use-cases Compose file format is being used.

Describe the solution you'd like
A dedicated document aside normative specification

Additional context
Proposed during community meeting on May 7th

Upade JSON schema

spec schema uses json schema draft 4
https://github.com/xeipuuv/gojsonschema do support up to draft v7, we should not rely on an obsolete version

Define "Image"

What is the problem you're trying to solve
Clearly define what is allowed for "Image"

Describe the solution you'd like
I would think we'd highlight the possible values. Right now it states "can either be a repository/tag, a digested reference or a partial image ID." I think it should be something like:
wouldn't it be registry/repository:tag or registry/repository@digest

sanitize healthcheck definition

What is the problem you're trying to solve
healthcheck configuration is based on a string list, first item defining the command type (CMD, CMD-SHELL or NONE). This is confusing and error prone.

Describe the solution you'd like
healthcheck test to be deprecated and we introduce support for command, using same syntax as container specification command - either single string (shell) or list (plain command). NONE is redundant with disable

Support for BuildKit extensions (secrets)

What is the problem you're trying to solve

With multi-stage builds, it's useful to be able to e.g. access an SSH server only from within the build image. Docker now supports that with BuildKit. It adds new command line flags and Dockerfile syntax. Compose should allow defining build secrets, just like it supports runtime secrets.

Building using BuildKit is enabled through an environment variable (DOCKER_BUILDKIT=1) or a Docker configuration file setting.

Compose already supports runtime secrets, which are files "mounted" inside a container. BuildKit seems very similar, but with some differences:

1. --ssh, which simply mounts a Unix socket and sets the SSH_AUTH_SOCK environment variable for RUN commands that request it.
2. The dst flag, which specifies an alternative directory (default is still /run/secrets/).
3. Though it's undocumented, it seems setting owner and mode is supported.

Describe the solution you'd like

This is a high-level discussion around the requirements of a solution. A more concrete suggestion would probably be more suited for a PR.

Tie in to the existing secrets. A file secret is defined by

• Type, but only file is currently supported
• ID (used to reference the secret in a RUN
• Source (the file path)

An SSH secret is defined by

• ID
• Agent socket paths

Both of them are referenced/dispatched by specifying

• Source (only for file secrets; implied for SSH)
• Target
• Required (boolean)
• UID
• GID
• Mode

This seems similar enough that it should be possible to extend the secrets section, and allow referencing it in an image's build section. The SSH type is specific to BuildKit, so parsing code would have to know which type is allowed in build/runtime. Using a separate section for build secrets is a possibility.

Lastly, adding a "builder" selector that allows choosing "buildkit" as the builder could set the right environment variable to allow parsing these flags. (The Dockerfile still needs to have the syntax line to support the RUN --mount variant.)

Additional context

A spec PR was suggested as the first step, in docker/compose#7296 (comment).

The --secret command line option is parsed into a github.com/moby/buildkit/session/secrets/secretsprovider.FileSource. The --ssh option is parsed into a github.com/moby/buildkit/session/sshforward/sshprovider.AgentConfig.

The secret mount is dispatched at https://github.com/moby/buildkit/blob/195f9e598cc7ba82aabefe60d564df7329cb38dd/frontend/dockerfile/dockerfile2llb/convert_secrets.go, and the ssh mount is dispatched at https://github.com/moby/buildkit/blob/195f9e598cc7ba82aabefe60d564df7329cb38dd/frontend/dockerfile/dockerfile2llb/convert_ssh.go.

Document schema using `title` and `description`

compose schema 3.9 do define the technical structure of a compose file but don't give any information about the intent and role of elements.

Publish schema on http://schemastore.org/json/

What is the problem you're trying to solve
schema can be used to offer IDE autocompletion and validation.

Describe the solution you'd like
schemastore.org do define an API for IDE to download schema
(https://www.jetbrains.com/help/idea/json.html)
we just have to publish compose schema to offer users autocompletion

Additional context
Initialy proposed by docker/compose#2089 (comment)

Mention of grants with no context

What is the problem you're trying to solve
In the Long Syntax description of Config, it says:

You can grant a service access to multiple configs and you can mix long and short syntax.

The discussion around config does not make it clear what "grant a service access to multiple configs" means. The use of the term "grant" implies that there is an access control system that must be used. What is this trying to say? Is it merely that multiple configs can be referenced from one service, or is there an implied permission model here?

Add support for `device_cgroup_rules`

What is the problem you're trying to solve
The Docker container engine offers the --device-cgroup-rule option to add devices to allowed devices list. This is very helpful if hardware needs to be accessed from within the container to make sure only a minimal set of hardware is accessible (helps to avoid the use of --privileged).

Describe the solution you'd like
In Docker Compose format 2.3 and later this has been part of the Docker Compose spec. I would like to readd this field to the latest version of the Docker Compose spec.

Additional context
This is a "Container runtime configuration", which in turn is part of "Deploy" area and as defined in the VISION.md document, this area is in scope of Compose.

The device_cgroup_rules as specified in Compose v2 reference:
https://docs.docker.com/compose/compose-file/compose-file-v2/#device_cgroup_rules

Support condition: service_healthy from v2

What is the problem you're trying to solve
We use Docker primarily for developing locally and running integration/system
tests on CI. This approach works brilliantly for us, as docker-compose.yaml
wholly describes the runtime environment needed to run an application. However,
we have a tough time dealing with the readiness problem. It's not enough to
just start the Compose services in the right order, as some infrastructure
takes time to become ready to serve the requests.

This negatively affects the experience with docker-compose, both when working
locally and, especially, when designing CI tests. This is because the 'naive
approach' of only using depends_on leads to a race condition by default. Such
tests will succeed if infrastructure comes up in time, and fail otherwise.
Developers quickly learn to disregard flaky tests, at which point these are as
useful as having no tests.

Additional context
The general stance on this from Docker folks seems to summarised in
Startup order docs page:

Your application needs to be resilient to these types of failures. To handle
this, design your application to attempt to re-establish a connection to the
database after a failure. If the application retries the connection, it can
eventually connect to the database.

While the sentiment is definitely positive, I disagree with tightly coupling
readiness with resilience against network loss. An app can be resilient, but
choose to do sanity checks at startup. For example, a service could check if
a database is available at startup and, if it's down, exit out immediately.
This will cause the supervisor to restart the service and, eventually, begin
alarming to let SREs know that something is wrong. In my opinion, this approach
is just as valid as having the service remain in 'starting' state until the
database is available.

What is more, this seems to be a very deployment-oriented advice. If you use
docker-compose for working locally or CI, you probably just want the
containers to be ready and aren't too worried about resilience.

All of the workarounds suggest injecting some form of 'wait' script into the
command or entry-point. While this works for simpler setups, it doesn't really
scale well:

• Duplication - dependencies are now listed in both depends_on and the
  wait script(s). Additionally, wait scripts now have to be (more or less)
  duplicated in all projects using docker-compose.
• Leaky abstraction - developers now have to know how to check if a
  particular bit of infrastructure is ready (e.g., is it TCP? what's the port?
  is it HTTP? what URL should I check?). This becomes increasingly difficult in
  CI situations, where the database/cache/queue is pre-populated with data for
  reproducible tests (since just checking a port is no longer enough).
• More dependencies - additional software is now required inside the app
  container. This degrades the overall UX from using Docker, as images are now
  larger and take longer to rebuild/download. In an anecdotal example, why
  should developers install several scripting interpreters just to check if
  dependencies are ready?
• Complexity - each individual wait script needs to implement functionality
  that healthcheck already provides: initial delay, retrying and eventual
  timeout.

Describe the solution you'd like
I believe that condition: service_healthy is a suitable and very elegant
solution to this issue. It would be great to carry it over from version 2 of
compose to the newer versions of the format.

It doesn't suffer from the same issues that the workarounds have:

• There is a single source of truth when it comes to container dependencies
  (i.e., depends_on in the docker-compose.yaml).
• Separation of concerns - infrastructure containers are
  responsible for performing health checks on themselves and indicating
  readiness for the main app container.
• Works out-of-the-box - no additional software has to be installed in
  the app container in order to 'wait' for dependencies.
• Simplicity - opting in for readiness is as easy as setting
  condition: service_healthy for your service.

Define remote services and depend on them using healthchecks

What is the problem you're trying to solve
Docker-compose file services depends on services on another machine and we need to run our service only if services on remote machine are up.

Describe the solution you'd like
To use depends_on to run services on current machine after services on another machine that described as a service in docker-compose.yml file on current machine. Use healthcheck to verify whether remote service is up or down

Additional context
That's very basic way to solve the same problems in easier way that are solved using other types of container orchestration

The `domainname` section is marked as TODO

I just noticed that there was no issue for this, and thought I would file a tracking issue.

Clarify spec intent in Vision docs regarding implementations

What is the problem you're trying to solve
VISION.md should make it clear we don't expect any Compose implementation to support all attributes. Actually neither docker-compose not docker swarm do support everything set by compose v3.8 file format.

Describe the solution you'd like
A clear explanation for the specification to make attributes optional and define recommended behaviour for unsupported features.

Additional context
Discussed during community meeting on May 7th

Generate documentation from json schema

Can use https://github.com/adobe/jsonschema2md, https://github.com/AnalyticalGraphicsInc/wetzel, or any other relevant tool

Define compose specification labels

What is the problem you're trying to solve
Specification uses com.docker.compose.* for compose labels. Should not be docker specific

Describe the solution you'd like
Choose a neutral label namespace for compose-spec

Additional context
I assume we want a dedicated domain name for online doc

Introduce support for "sidecars" containers

What is the problem you're trying to solve

Whenever some of the compose attributes only make sense when containers are grouped on a single host, sharing namespaces, many others focus on the idea to distribute services on a cluster with replication.

In the latter case, a common practice is to attach to the service container a set of "sidecars" to support additional infrastructure services, typically monitoring, logs collection, tracing, security, service-mesh routing, etc.

Describe the solution you'd like

At the lowest level, a sidecar could be defined using the same data model as services, excluding all deployment metadata as it is not expected to be deployed on it's own. We could enable this with a nested container definition in services:

If we split the service model into container definition and deployment/lifecycle directives, a compose service would then be defined by a ContainerSpec, deployment metadata and nested sidecars ContainerSpecs.

services:
  web:
    image: apache/httpd
    sidecars:
        - image: fluentbit
          command: --parser=apache

Additional context

This approach do not offer an annotation-based sidecar approach as used by most Kubernetes integrations, which rely on runtime injection for sidecars based on annotations declared on services. The current proposal can be consider an initial step to support this (better) approach, as a higher level model could be translated into this lower level set of sidecars for Compose implementations which don't support runtime injection, and have to rely on some pre-processing.

Define Compose lifecycle

What is the problem you're trying to solve
There's a few places where Compose specification do assume some ordering in resources management (attaching to networks, mounting volumes, handling depends_on). But the spec do not define application lifecycle.

Describe the solution you'd like
A clear lifecycle document to clarify how resource get created/updated/removed or left for further usage (typically: volumes) and required ordering.

Add support for `ipc: service` syntax

What is the problem you're trying to solve
docker supports --ipc flag, which according to docs:

Shared memory segments are used to accelerate inter-process communication at memory speed, rather than through pipes or through the network stack. Shared memory is commonly used by databases and custom-built (typically C/OpenMPI, C++/using boost libraries) high performance applications for scientific computing and financial services industries. If these types of applications are broken into multiple containers, you might need to share the IPC mechanisms of the containers, using "shareable" mode for the main (i.e. “donor”) container, and "container:" for other containers.

Assuming I have an application that makes use of shared memory, I can bring it up with docker run --ipc=container:name .... All good here 👍

However, if I'm using docker-compose - I am unable to get the same behaviour. This is because, according to docs, it's simply a pass-through option. This forces developers to co-host everything in the same container, which is difficult to work with and doesn't scale.

Describe the solution you'd like
It would be amazing if ipc option in docker-compose supported service:[service name] syntax, similar to network_mode. For example:

version: '2.4'
services:
  myrtr:
    image: myipcrouter
  mysvc:
    build:
      context: .
      dockerfile: Dockerfile
    ipc: service:myrtr

This would make the container for the service (i.e, mysvc) have --ipc=container:<id of myrtr>.

Additional context
Original issue submitted here docker/compose#7220, PR for implementation is available here docker/compose#7417. @ndeloof suggested an issue/PR is submitted here as well.

Compose-spec reference "com.docker.compose.*" labels which are docker-compose specific

What is the problem you're trying to solve
The spec enforce use of "com.docker.compose.*" labels. This is used only by docker-compose implementation (not even docker stack)

Describe the solution you'd like
Drop reference to labels

Create technology compatibility kit

What is the problem you're trying to solve
As Compose specification do define a neutral format to declare applications, and user should be able to switch between tools, we need some way to ensure those are compatible and respect the specification

Describe the solution you'd like
A test suite checking various aspect of the specification, that couldbe applied to various tools

Additional context
Comparable to Java EE TCK but in a simpler form :P

Specification is Docker and/or Linux specific in many places

What is the problem you're trying to solve
Feature definition is Docker/Linux specific in a few place, and make it hard to understand how it could apply to other implementations/platforms

sample :

init run an init process (PID 1) inside the container that forwards signals and reaps processes

Specification should better define the issue this options is expected to solve for users and not consider the implementation details. Here, init/PID 1 being responsible to collect dead processes (zombies) is a technical detail.

Describe the solution you'd like
A higher level definition of the user issue this option is expected to address.

Consider support for compose v2 `extends`

What is the problem you're trying to solve
extends was supported in compose file format v2 and removed in v3. Has since be requested by many users, see moby/moby#31101

Describe the solution you'd like
Re-introduce this feature, with a full description so we don't suffer implementation glitches between docker-compose's de facto python implementation and compose-go

Additional context
see docker/compose#4315 for debates about the validity of this feature and it's drawbacks. Also consider option to use of compose file overrides and yaml anchors to achieve comparable results.

Windows support

What is the problem you're trying to solve
We need to outline what configurations and values are allowed on Windows workloads.

Describe the solution you'd like
The existing Compose file version 3 reference has many references to what's valid for Windows. We need to eventually bring those over. Interestingly enough, the existing compose documentation is missing some of the not allowed items like "/etc/hosts" but we could pull this information from the Task Definition Parameters if needed.

Include docs

Review https://github.com/docker/docker.github.io/tree/master/compose for relevant information we should use to document the compose spec, i.e file format, not docker-compose (tool) usage and behaviour.

Include an archived version of fig.sh website

What is the problem you're trying to solve
We need to remember where we come from as part of specification history

Describe the solution you'd like
include an archived version of legacy fig.sh website

Additional context

Document canonical mapping with Kubernetes

What is the problem you're trying to solve
Kubernetes is a major target for compose specification, so we should establish canonical mapping for compose elements

Describe the solution you'd like
Can leverage https://github.com/kubernetes/kompose/blob/master/docs/conversion.md

• define custom fields when relevant

Additional context
Also see https://github.com/docker/compose-on-kubernetes/blob/master/docs/mapping.md

Allow vendor-specific extensions

What is the problem you're trying to solve
Compose specification is platform-neutral, and express a generic application model. But if we don't want this to just be an academic model, but a pragmatic one people can actually use for real-world application, we need to offer a way for vendors to support custom attributes. Compose specification do allready have such a support with x-* extension fields on top-level elements.

Describe the solution you'd like

1. A generalization of the extension field support
2. Recommendation to use vendor prefixes and document known ones

Doing so one could use a compose file like this one :

service:
  backend:
    deploy:
      placement:  
        x-aws-region:   "eu-west-3"
        x-azure-region: "france-central"

Unrecognized x- attributes should be ignored, which means the same compose file can both run locally, on AWS and Azure. If this proprietary extension is widely adopted and supported by many vendors, it would make it easier to debate about inclusion in a future version of the specification.

Use of vendor prefixes also limit the risk for conflicting extensions with distinct meaning by vendors.

Additional context
Add any other context about the feature request here.

Define pull policy

What is the problem you're trying to solve
Specification require image to be pulled then built if absent, and allow implementation ot offer options to tweak this behaviour. Users should be able to choose what's the best option for them.

Describe the solution you'd like
Introduce pull_policy (always|never|if_not_present|build) so that users can define within compose file how to manage image pulls vs build.

Don't require version

What is the problem you're trying to solve
Better than being version-based, Compose spec should be feature driven

Describe the solution you'd like
As my compose file use some xyz attribute, I should not have to specify a version attribute, but just let the Compose implementation tell me this attribute is not supported.

Proposal: Scheduled Jobs

What is the problem you're trying to solve?

This proposal is to gather community feedback on the idea of adding "scheduled jobs" to the compose-spec.

What are scheduled jobs?

A scheduled job is a task which is run at some defined interval (defined by a cron syntax). Unlike services, these executions of the container are ephemeral - they are spun up when it's time, and exit once they're complete.

What problem does this solve?

There are many problems that are well suited to being modeled as happening only periodically. Some examples are:

• Sending nightly emails
• Sending notifications to chatrooms
• Purging database records
• Polling some data source
• Some batch/analytic job

The imagination is the limit on types of workloads that can be modeled as scheduled jobs!

Is there a need for this?

There's certainly president. There are lots of third party tools which helps customers model their scheduled jobs via docker-compose. The Ofelia project, for example, exposes the schedule that a command should be run via labels on a service.

So another question would be, why not just follow this model - allow customers to set up services and tag them with some metadata? I think this model works well - but it forces people to write their scheduled jobs as services. While this works fine locally, as customers transition their compose apps to cloud, they'll be faced with the fact that services are often more expensive (operationally and financial) than a simple scheduled job.

What might it look like?

services:
  web:
    build: ./web-app
    depends_on:
      - db
      - redis
  redis:
    image: redis
  db:
    image: postgres
jobs:
  email-sender:
    schedule: @daily
    build: ./email-sender
    command: bundle exec send-email
    environment:
      from: [email protected]
    depends_on:
      -db

Another option would be to add some scheduled action attribute to services:

services:
  web:
    build: ./web-app
    scheduled-actions:
      clean-db:
         when: @daily
         action: "rake db clean"
    depends_on:
      - db
      - redis
  redis:
    image: redis
  db:
    image: postgres

Execute a command after run/entrypoint

What is the problem you're trying to solve
It would be great if we could execute a command when the container is up or after the entrypoint. Currently the only workaround is to do so in entrypoint or manually execute the command after the container is up. The former prevents users from using the official Docker images.

My specific use case is simulating a local multi-datacenter Consul cluster.To do so I need to execute a join command that connects nodes together. Creating a custom Docker image, that is a copy of an already available official Docker image, just to add one command seems a bit silly.

There is an issue about this at docker/compose#1809 which have a lot of good points!

Describe the solution you'd like
Something like this would work:

service:
    image: database
    onrun: echo "Executing command after run!"

This already exists in Kubernetes through its lifecycle hooks. As someone pointed out, in the issue linked above, since Docker compose and Kubernetes have many of the same use cases it would be reasonable to support this.

Kubernetes Implementation: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#hook-handler-implementations

Support inline security profiles

What is the problem you're trying to solve
When running a container which requires a custom (seccomp) security profile this needs to be stored in a separate file. It would be nice if this can be stored as part of the docker-compose.yml like all other container runtime configuration.

Describe the solution you'd like
The profile probably should be in a shared section so it can be reused, similar to volumes:

...
    security_opt:
        - "seccomp=custom"

seccomp_profiles:
  custom: |
     {
        "defaultAction": "SCMP_ACT_ERRNO",
        "syscalls": [
             {
                 "name": "accept",
                 "action": "SCMP_ACT_ALLOW",
                 "args": null
             },
             ...
     }

Maybe we can make this much more generic, and basically add a generic "config file storage":

...
    security_opt:
        - "seccomp=my-seccomp-profile.json"

files:
    name: my-seccomp-profile.json
    content: |
     {
        "defaultAction": "SCMP_ACT_ERRNO",

Additional context

Move (or copy) resource-specific legacy properties to deploy section

We have legacy service properties that could be placed in the deploy section.
Example: cpu_percent, cpu_shares, cpu_*, mem_limit, mem_swappiness, mem_*.

Naming limitations for `container_name`

If I recall correctly, the container_name field is more restrictive than just any UTF-8-encoded string. That should be specified in the spec.

Define "host or none"

What is the problem you're trying to solve
Clarify the "host or none" section.

Describe the solution you'd like
Literally a copy/paste of what's in the Compose file version 3 reference for "host or none" would be a great addition/replacement.

Define "project"

What is the problem you're trying to solve
Specification make reference to "project_name" which is specific to docker-compose implementation.

Describe the solution you'd like
Specification should define what a "project" is, or use another noun ("application" ?)

Define constrained values with enums

In a few places the spec do define values as string, but they are actually constrained.
Typical example: "propagation": {"type": "string"} but the documentation is more constraining and says "May be one of rprivate, private, rshared, shared, rslave, slave."

Proposal: hierarchical secrets

What is the problem you're trying to solve
Docker secret use a simple model to map the content of a secret as plain text to a file under /run/secrets.
Both Kubernetes and AWS Secret Manager use a more complex model where secrets are stored as key-indexed documents (map[string]string).

The intent of this proposal is to extend Compose spec secret definition to cover both use cases in a consistent way.

Describe the solution you'd like
I propose we introduce support for keys in secret definition:

secrets:
  my_secret:
    external: true
    keys: ["foo", "bar"]

Doing so will result into retrieving external secret my_secret from infrastructure as a hierarchical document, and create files foo and bar in secret volume mounted inside container:

➜  ls -l /run/secrets/my_secret
total 0
-r--r--r--    1 root  root     bar
-r--r--r--    1 root  root     foo

Not being able to parse secret content as a hierarchical document will result into an error.

keys will support special value * to inject all entries in the secret as files.

if keys is not defined, and Compose implementation support this, the raw secret content is written into a file and attached to the container as secret volume:

secrets:
  my_secret:
    external: true

➜  cat /run/secrets/my_secret
{
  "foo": "***",
  "bar": "***"
}

So,

• Kubernetes Secrets can be covered with optionally limiting bound keys. As raw secret is not supported on this platform, keys could default to *.
• Swarm do not supported hierarchical secrets, and would reject (or ignore) a Compose file using `keys``
• AWS ECS can both support hierarchical secrets being injected, as well as plain text secrets.

Additional context
see https://github.com/docker/ecs-plugin/issues/207

Investigate v2 support

What is the problem you're trying to solve
Docker compose file format version 2.x is still in use by many people
And only differ from v3 by a few removed attribute + addition of deploy

Describe the solution you'd like
spec could accept 2.x format. Maybe oculd also reconsider some of the removed attributes for inclusion in v3. excluded attribute would then be documented but flagged as Deprecated

investigate support for init container

What is the problem you're trying to solve
init container(s) run before service container in sequence and can prepare environment. Think "containerized entrypoint"

We might want to support such init containers in compose spec, with obvious mapping for Kubernetes

Describe the solution you'd like
syntax proposal and lifecycle document

Additional context
See docker/compose#6855

allow compose.yml as well as docker-compose.yml

What is the problem you're trying to solve
Currently the default Compose file name is vendor specific.

Describe the solution you'd like
We cannot break existing users, so by default, if no Compose file specified, try docker-compose.yml first then compose.yml, but make compose.yml the preferred form.

depends_on cannot mapping directly to kubernetes

What is the problem you're trying to solve
depends_on cannot mapping directly to kubernetes

Describe the solution you'd like
mark it as optional

Additional context
kubernetes/kompose#1053
docker/compose#6855