yelp/dumb-init

A minimal init system for Linux containers

dumb-init

dumb-init is a simple process supervisor and init system designed to run as PID 1 inside minimal container environments (such as Docker). It is deployed as a small, statically-linked binary written in C.

Lightweight containers have popularized the idea of running a single process or service without normal init systems like systemd or sysvinit. However, omitting an init system often leads to incorrect handling of processes and signals, and can result in problems such as containers which can't be gracefully stopped, or leaking containers which should have been destroyed.

dumb-init enables you to simply prefix your command with dumb-init. It acts as PID 1 and immediately spawns your command as a child process, taking care to properly handle and forward signals as they are received.

Why you need an init system

Normally, when you launch a Docker container, the process you're executing becomes PID 1, giving it the quirks and responsibilities that come with being the init system for the container.

There are two common issues this presents:

1. In most cases, signals won't be handled properly.

   The Linux kernel applies special signal handling to processes which run as PID 1.

   When processes are sent a signal on a normal Linux system, the kernel will first check for any custom handlers the process has registered for that signal, and otherwise fall back to default behavior (for example, killing the process on SIGTERM).

   However, if the process receiving the signal is PID 1, it gets special treatment by the kernel; if it hasn't registered a handler for the signal, the kernel won't fall back to default behavior, and nothing happens. In other words, if your process doesn't explicitly handle these signals, sending it SIGTERM will have no effect at all.

   A common example is CI jobs that do docker run my-container script: sending SIGTERM to the docker run process will typically kill the docker run command, but leave the container running in the background.

2. Orphaned zombie processes aren't properly reaped.

   A process becomes a zombie when it exits, and remains a zombie until its parent calls some variation of the wait() system call on it. It remains in the process table as a "defunct" process. Typically, a parent process will call wait() immediately and avoid long-living zombies.

   If a parent exits before its child, the child is "orphaned", and is re-parented under PID 1. The init system is thus responsible for wait()-ing on orphaned zombie processes.

   Of course, most processes won't wait() on random processes that happen to become attached to them, so containers often end with dozens of zombies rooted at PID 1.

What dumb-init does

dumb-init runs as PID 1, acting like a simple init system. It launches a single process and then proxies all received signals to a session rooted at that child process.

Since your actual process is no longer PID 1, when it receives signals from dumb-init, the default signal handlers will be applied, and your process will behave as you would expect. If your process dies, dumb-init will also die, taking care to clean up any other processes that might still remain.

Session behavior

In its default mode, dumb-init establishes a session rooted at the child, and sends signals to the entire process group. This is useful if you have a poorly-behaving child (such as a shell script) which won't normally signal its children before dying.

This can actually be useful outside of Docker containers in regular process supervisors like daemontools or supervisord for supervising shell scripts. Normally, a signal like SIGTERM received by a shell isn't forwarded to subprocesses; instead, only the shell process dies. With dumb-init, you can just write shell scripts with dumb-init in the shebang:

#!/usr/bin/dumb-init /bin/sh
my-web-server &  # launch a process in the background
my-other-server  # launch another process in the foreground

Ordinarily, a SIGTERM sent to the shell would kill the shell but leave those processes running (both the background and foreground!). With dumb-init, your subprocesses will receive the same signals your shell does.

If you'd like for signals to only be sent to the direct child, you can run with the --single-child argument, or set the environment variable DUMB_INIT_SETSID=0 when running dumb-init. In this mode, dumb-init is completely transparent; you can even string multiple together (like dumb-init dumb-init echo 'oh, hi').

Signal rewriting

dumb-init allows rewriting incoming signals before proxying them. This is useful in cases where you have a Docker supervisor (like Mesos or Kubernetes) which always sends a standard signal (e.g. SIGTERM). Some apps require a different stop signal in order to do graceful cleanup.

For example, to rewrite the signal SIGTERM (number 15) to SIGQUIT (number 3), just add --rewrite 15:3 on the command line.

To drop a signal entirely, you can rewrite it to the special number 0.

Signal rewriting special case

When running in setsid mode, it is not sufficient to forward SIGTSTP/SIGTTIN/SIGTTOU in most cases, since if the process has not added a custom signal handler for these signals, then the kernel will not apply default signal handling behavior (which would be suspending the process) since it is a member of an orphaned process group. For this reason, we set default rewrites to SIGSTOP from those three signals. You can opt out of this behavior by rewriting the signals back to their original values, if desired.

One caveat with this feature: for job control signals (SIGTSTP, SIGTTIN, SIGTTOU), dumb-init will always suspend itself after receiving the signal, even if you rewrite it to something else.

Installing inside Docker containers

You have a few options for using dumb-init:

Option 1: Installing from your distro's package repositories (Debian, Ubuntu, etc.)

Many popular Linux distributions (including Debian (since stretch) and Debian derivatives such as Ubuntu (since bionic)) now contain dumb-init packages in their official repositories.

On Debian-based distributions, you can run apt install dumb-init to install dumb-init, just like you'd install any other package.

Note: Most distro-provided versions of dumb-init are not statically-linked, unlike the versions we provide (see the other options below). This is normally perfectly fine, but means that these versions of dumb-init generally won't work when copied to other Linux distros, unlike the statically-linked versions we provide.

Option 2: Installing via an internal apt server (Debian/Ubuntu)

If you have an internal apt server, uploading the .deb to your server is the recommended way to use dumb-init. In your Dockerfiles, you can simply apt install dumb-init and it will be available.

Debian packages are available from the GitHub Releases tab, or you can run make builddeb yourself.

Option 3: Installing the .deb package manually (Debian/Ubuntu)

If you don't have an internal apt server, you can use dpkg -i to install the .deb package. You can choose how you get the .deb onto your container (mounting a directory or wget-ing it are some options).

One possibility is with the following commands in your Dockerfile:

RUN wget https://github.com/Yelp/dumb-init/releases/download/v1.2.5/dumb-init_1.2.5_amd64.deb
RUN dpkg -i dumb-init_*.deb

Option 4: Downloading the binary directly

Since dumb-init is released as a statically-linked binary, you can usually just plop it into your images. Here's an example of doing that in a Dockerfile:

RUN wget -O /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.2.5/dumb-init_1.2.5_x86_64
RUN chmod +x /usr/local/bin/dumb-init

Option 5: Installing from PyPI

Though dumb-init is written entirely in C, we also provide a Python package which compiles and installs the binary. It can be installed from PyPI using pip. You'll want to first install a C compiler (on Debian/Ubuntu, apt-get install gcc is sufficient), then just pip install dumb-init.

As of 1.2.0, the package at PyPI is available as a pre-built wheel archive and does not need to be compiled on common Linux distributions.

Usage

Once installed inside your Docker container, simply prefix your commands with dumb-init (and make sure that you're using the recommended JSON syntax).

Within a Dockerfile, it's a good practice to use dumb-init as your container's entrypoint. An "entrypoint" is a partial command that gets prepended to your CMD instruction, making it a great fit for dumb-init:

# Runs "/usr/bin/dumb-init -- /my/script --with --args"
ENTRYPOINT ["/usr/bin/dumb-init", "--"]

# or if you use --rewrite or other cli flags
# ENTRYPOINT ["dumb-init", "--rewrite", "2:3", "--"]

CMD ["/my/script", "--with", "--args"]

If you declare an entrypoint in a base image, any images that descend from it don't need to also declare dumb-init. They can just set a CMD as usual.

For interactive one-off usage, you can just prepend it manually:

$ docker run my_container dumb-init python -c 'while True: pass'

Running this same command without dumb-init would result in being unable to stop the container without SIGKILL, but with dumb-init, you can send it more humane signals like SIGTERM.

It's important that you use the JSON syntax for CMD and ENTRYPOINT. Otherwise, Docker invokes a shell to run your command, resulting in the shell as PID 1 instead of dumb-init.

Using a shell for pre-start hooks

Often containers want to do some pre-start work which can't be done during build time. For example, you might want to template out some config files based on environment variables.

The best way to integrate that with dumb-init is like this:

ENTRYPOINT ["/usr/bin/dumb-init", "--"]
CMD ["bash", "-c", "do-some-pre-start-thing && exec my-server"]

By still using dumb-init as the entrypoint, you always have a proper init system in place.

The exec portion of the bash command is important because it replaces the bash process with your server, so that the shell only exists momentarily at start.

Building dumb-init

Building the dumb-init binary requires a working compiler and libc headers and defaults to glibc.

$ make

Building with musl

Statically compiled dumb-init is over 700KB due to glibc, but musl is now an option. On Debian/Ubuntu apt-get install musl-tools to install the source and wrappers, then just:

$ CC=musl-gcc make

When statically compiled with musl the binary size is around 20KB.

Building the Debian package

We use the standard Debian conventions for specifying build dependencies (look in debian/control). An easy way to get started is to apt-get install build-essential devscripts equivs, and then sudo mk-build-deps -i --remove to install all of the missing build dependencies automatically. You can then use make builddeb to build dumb-init Debian packages.

If you prefer an automated Debian package build using Docker, just run make builddeb-docker. This is easier, but requires you to have Docker running on your machine.

See also

• Docker and the PID 1 zombie reaping problem (Phusion Blog)
• Trapping signals in Docker containers (@gchudnov)
• tini, an alternative to dumb-init
• pid1, an alternative to dumb-init, written in Haskell

dumb-init's Issues

Generate checksum

Could you guys generate the checksum in each release? Something like gosu does

WEXITSTATUS without checking WIFEXITED

From waitpid(2):

WEXITSTATUS(status)
returns the exit status of the child. [...] This macro should only be employed if WIFEXITED returned true.

dumb-init does not check WIFEXITED before calling WEXITSTATUS, which could lead to funny behaviour when the child process is terminated by a signal.

Duplicate Work with Tini Init

Hi All,

I love this solution. However, there is parallel work that appears at first glance to be similar at https://github.com/krallin/tini. Is there some notable difference between the two that I can't discern?

Cheers,

A sample spec file to build dumb-init as an RPM

I attach an example RPM SPEC file (dumb-init.spec) that can be used to build an RPM of this application. The user should change the version number for the 'this_ver' macro to match the downloaded version number before building the RPM. This one has coded in it version 1.1.2
dumb-init.spec.txt

Mention dependencies for building the debian package

Hi there,

just tried to build the debian package using "make builddeb" on Ubuntu 16.04. Just thought, it might be worth mentioning in the doku that (at least) the following packages need to be installed:

• devscripts (for buildep)
• musl-tools
• debhelper
• help2man
• python-pytest
• python-mock

Kind Regards,
Beet

`tty_test` passes even with broken tty (test pollution?)

1. Make a change that should break the test by causing SIGTTOU to be sent to the child (e.g. use the branch from abandonded PR #26).
2. Run all tests, it works: py.test tests/ -vvv
3. Run just one test, it fails: py.test tests/ -vvv -k tty_test

UB in signal_handler

See section 7.14.1.1 of the C99 standard:

If the signal occurs other than as the result of calling the abort or raise function, the
behavior is undefined if the signal handler refers to any object with static storage duration
other than by assigning a value to an object declared as volatile sig_atomic_t

signal_handler reads use_setsid and child_pid, which have static storage duration.

A possible solution would be to handle signals synchronously and avoid these limitations. See sigprocmask(2) and sigwait(3), or sinit for examples.

Be polite, respectful and reasonable

I was shocked to see how minor yet meaningful contribution #84 by @jirutka was turned down in a rude way by team member @bukzor ("closed as propaganda") which was then promptly justified by another team member @chriskuehl with no apologies. Comment(s) were subsequently removed and conversation locked without any reasoning or discussion. Nobody was given a chance for '+1' or '-1'.

That kind of rude and unfair treatment of contributors (and contributions) makes me wonder whether it was a good idea to get involved into this project. Contributors deserve more respect. Dear dumb-init team, please mind your manners and don't turn down contributions for ill-advised reasons.
Everyone would benefit from more appreciation to each others' efforts. Please be reasonable, respectful and polite.

Switch from Travis to CircleCI

With circleci we should be able to run our integration tests in CI (they rely on Docker).

Rewrite tests in Python to improve readability and reduce flakiness

The current tests are written in bash, and we have to use some annoying hacks (for example, using a named fifo to send individual lines) that are a little bit flaky (occasionally fails with "Interrupted system call" when writing to the FIFO).

Now that we have one test in Python anyway, we might as well rewrite the rest of them like that, too. It should make them simpler and more reliable.

The tests are actually very simple when you take out the hacks we had to do because of bash.

Add man page to Debian package

help2man can easily create a manpage given a binary with --help and --version, we can easily use this once #37 is merged.

child_processes_test.test_no_setsid_doesnt_signal_entire_group is flaky

This failed on the Debian armhf build:
https://buildd.debian.org/status/fetch.php?pkg=dumb-init&arch=armhf&ver=1.1.2-1&stamp=1469941024&file=log

=================================== FAILURES ===================================
_________________ test_no_setsid_doesnt_signal_entire_group[1] _________________

    @pytest.mark.usefixtures('both_debug_modes', 'setsid_disabled')
    def test_no_setsid_doesnt_signal_entire_group():
        """When dumb-init is not running in setsid mode, it should only signal its
        immediate child.
        """
        pids = spawn_and_kill_pipeline()

        def assert_four_living_pids():
            assert len(living_pids(pids)) == 4

        sleep_until(assert_four_living_pids)

        for pid in living_pids(pids):
>           os.kill(pid, signal.SIGKILL)
E           OSError: [Errno 3] No such process

tests/child_processes_test.py:63: OSError
----------------------------- Captured stderr call -----------------------------
[dumb-init] Running in debug mode.
[dumb-init] Not running in setsid mode.
[dumb-init] Child spawned with PID 16950.
[dumb-init] Received signal 15.
[dumb-init] Forwarded signal 15 to children.
[dumb-init] Received signal 17.
[dumb-init] A child with PID 16950 was terminated by signal 15.
[dumb-init] Forwarded signal 15 to children.
[dumb-init] Child exited with status 143. Goodbye.
yes: standard output: Broken pipe
error
error
error
error
error
error
error
error
error
error
===================== 1 failed, 172 passed in 7.46 seconds =====================
debian/rules:30: recipe for target 'override_dh_auto_test' failed
make[1]: *** [override_dh_auto_test] Error 1
make[1]: Leaving directory '/«PKGBUILDDIR»'
debian/rules:9: recipe for target 'build-arch' failed
make: *** [build-arch] Error 2

All the assertions succeeded (so the test should have passed), but then we tried to kill a process that was already dead.

The pipeline looks like this:

proc = Popen((
    'dumb-init',
    'sh', '-c',
    "yes 'oh, hi' | tail & yes error | tail >&2"
))

The output from the test is really helpful. It looks like the yes error command was killed, and the tail it was piping to had time to print the last 10 lines and exit before we killed it, resulting in the error above.

We should probably just ignore errors when killing these?

ENTRYPOINT vs CMD

Shouldn't dumb-init be used as ENTRYPOINT instead of CMD?

As I understand, default entrypoint for docker container is /bin/sh -c.
If you prefix your app myapp with dumb-init in CMD, what would be actually ran is sh -c "dumb-init myapp", so PID1 would be still sh -c, not dumb-init.

SIGCHLD doesn't reap properly

If this is intended to be pid 1, it needs to properly reap orphan processes by handling SIGCHLD. There should be a handler for SIGCHLD that essentially does:

while (waitpid((pid_t)(-1), 0, WNOHANG) > 0) {};

more info here:

http://stackoverflow.com/questions/20688982/zombie-process-vs-orphan-process

Still see dead process

Hi, I'm using this dumb-init as my entry point in my dock image from centos 6
ENTRYPOINT ["/usr/local/bin/dumb-init", "-c"]
The container is spun up by jenkins docker plugin as executor slaves with command bash -c '/usr/sbin/sshd -D'

After job finishes, I still see the zombi process issue in my docker machine

root 6985 1 0 12:07 ? 00:00:01 /usr/bin/docker -d -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --storage-driver aufs --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=amazonec2

root 9138 6985 0 12:08 ? 00:00:00 [dumb-init]

500 9653 9138 37 12:08 ? 00:06:14 [java]

First one is docker service. I'm not able to kill either of latter ones. I'm kinda new to Docker, any idea?

Failed exec produces misleading error message when passing flags

(venv) ckuehl@dementors:~/test$ dumb-init -- doesnotexist
[dumb-init] --: No such file or directory

It's because we still print argv[1], which is a remnant from the days where dumb-init did no argument parsing (except to read the program to execute):

        // if this point is reached, exec failed, so we should exit nonzero
        PRINTERR("%s: %s\n", argv[1], strerror(errno));
        return 2;

Error: cannot set terminal process group (-1): Inappropriate ioctl for device

When I have dumb-init in ENTRYPOINT like this

...
ENTRYPOINT ["/usr/bin/dumb-init"]
...

and then try to run bash with tty and interactively, I get this error:

docker run --rm -t -i dumb_init_image bash
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
...

How can i receive signal in the jvm?

I use /bin/bash to run jvm as a child process, but i can't receive signal in the jvm.

Dockerfile:

ENTRYPOINT ["/dumb-init", "--", "/docker-entrypoint.sh"]

and docker-entrypoint.sh:

#!/usr/bin/dumb-init /bin/bash

source `dirname $0`/init-product.sh

if [ -z "$POOL_TYPE" ]; then
  source `dirname $0`/init.sh $RESOURCE_ID
  echo "****FINISHED****"
else
  echo "****FINISHED****"
  java -Xmx64m -Xms16m -Xmn8m -Xss1m -XX:ReservedCodeCacheSize=5m -XX:NewRatio=3 -jar /usr/local/agent/agent-1.0.1.jar $POOL_TYPE
fi

dumb-init with docker + npm run start

Dumb-init does not seem to work for me when it is used as the main process in a docker container and when the underlying command uses npm.

Run a docker container with entrypoint ["dumb-init", "--"] and command ["npm", "run", "start"], where the start script is defined in a package.json to be "node index.js". Add this to "index.js":

'use strict';

const internals = {};

process.on('SIGTERM', function() {
    console.log('received signal SIGTERM');
    internals.exit();
});

const inverval = setInterval(() => {
    console.log('abc');
}, 1 * 1000);

internals.exit = function() {
    clearInterval(inverval);
};

Now if I send SIGTERM to PID 1 (the dumb-init process), the process just exits without ever printing received signal SIGTERM. However, when I exec into a running docker container and run this procedure in a separate process, where dumb-init -- ... does not have PID 1, then the signal is printed correctly before exiting.

FTBFS: tests failures on [armhf,mips,mips64el,mipsel]

1.1.2 currently FTBFS on

• armhf
• mips
• mips64el
• mipsel

(and some other architectures) due to test failures.

Interestingly 1.0.3 used to built successfully on those architectures...

wait for all children?

We've encountered a couple third party systems (jenkins is one) that restart via (fork, die, self-exec).
This causes docker to stop since the run process is dead.
Dumb-init has the same behavior.

Would it be possible or a good idea to get dumb-init to help with this problem?

cc @dgholz

Real-time signals not fowarded

Dumb-init doesn't forward real-time signals.

Also, the use of signal is technically discouraged, because it's platform-dependent and its behavior could vary on different compile flags.

create, send signal to a process group

Use case is a bash script like so:

yes oh, hi! | pv -qL 10 &
yes error | pv -qL 10 >&2

When dumb-init gets sigterm (or any other signal), I want the whole process tree (bash, yes, pv, yes, pv) to get the same signal.

Implementation:

• on startup dumb-init initiates a process group:
  http://man7.org/linux/man-pages/man2/setsid.2.html
• on signal, dumb-init sends that signal to the process group it created:
  http://man7.org/linux/man-pages/man2/kill.2.html

Making dumb-init-latest available

Hi there,
the precompiled dumb-init Debian package is (at the time of writing) called dumb-init_1.0.2_amd64.deb, i.e. has a specific version number. This makes it difficult to pull the package from within a script, as the version number may change at any time. Would there be a chance to always add a package e.g. named dumb-init-latest ?
In any case thanks and best regards,
Beet

wget installation does not work on Alpine

Steps to reproduce

$ docker run -it mhart/alpine-node:6 sh
# wget -O /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.0.1/dumb-init_1.0.1_amd64

Expected result

dumb-init is installed, as it happens when using the ubuntu container, e.g..

Actual result

An error message is returned by wget:

Connecting to github.com (192.30.252.120:443)
wget: can't execute 'ssl_helper': No such file or directory
wget: error getting response: Connection reset by peer

Any thoughts on this?

child_processes_test.test_fails_nonzero_with_bad_exec is flaky on CI

This happened on CI during make test (not inside Docker):

21:49:33 py27 runtests: commands[0] | python -m pytest
21:49:33 ============================= test session starts ==============================
21:49:33 platform linux2 -- Python 2.7.6, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
21:49:33 rootdir: /ephemeral/jenkins_prod_slave/workspace/mirrors-Yelp-dumb-init, inifile: pytest.ini
21:49:33 plugins: timeout-1.0.0
21:49:33 collected 173 items
21:49:33 
21:49:39 tests/child_processes_test.py ....................F...
21:49:39 tests/cli_test.py ......................................................................
21:49:39 tests/exit_status_test.py ................................................
21:49:40 tests/proxies_signals_test.py ..........................
21:49:41 tests/shell_background_test.py ....
21:49:41 tests/tty_test.py .
21:49:41 
21:49:41 =================================== FAILURES ===================================
21:49:41 _________________ test_fails_nonzero_with_bad_exec[0-0-args0] __________________
21:49:41 
21:49:41 args = ('/doesnotexist',)
21:49:41 
21:49:41     @pytest.mark.parametrize('args', [
21:49:41         ('/doesnotexist',),
21:49:41         ('--', '/doesnotexist'),
21:49:41         ('-c', '/doesnotexist'),
21:49:41         ('--single-child', '--', '/doesnotexist'),
21:49:41     ])
21:49:41     @pytest.mark.usefixtures('both_debug_modes', 'both_setsid_modes')
21:49:41     def test_fails_nonzero_with_bad_exec(args):
21:49:41         """If dumb-init can't exec as requested, it should exit nonzero."""
21:49:41         proc = Popen(('dumb-init',) + args, stderr=PIPE)
21:49:41 >       _, stderr = proc.communicate()
21:49:41 
21:49:41 tests/child_processes_test.py:141: 
21:49:41 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
21:49:41 /usr/lib/python2.7/subprocess.py:793: in communicate
21:49:41     stderr = _eintr_retry_call(self.stderr.read)
21:49:41 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
21:49:41 
21:49:41 func = <built-in method read of file object at 0x2a56f60>, args = ()
21:49:41 
21:49:41     def _eintr_retry_call(func, *args):
21:49:41         while True:
21:49:41             try:
21:49:41 >               return func(*args)
21:49:41 E               Failed: Timeout >5s
21:49:41 
21:49:41 /usr/lib/python2.7/subprocess.py:476: Failed
21:49:41 ===================== 1 failed, 172 passed in 7.75 seconds =====================

Not quite sure what could have caused this...

Test failure on armv7hl Fedora

Moving from #115 to track this specific issue better.

Here's the traceback:

=================================== FAILURES ===================================
_____________ test_all_processes_receive_term_on_exit_if_setsid[1] _____________
    @pytest.mark.usefixtures('both_debug_modes', 'setsid_enabled')
    def test_all_processes_receive_term_on_exit_if_setsid():
        """If the child exits for some reason, dumb-init should send TERM to all
        processes in its session if setsid mode is enabled."""
>       child_pid, child_stdout = spawn_process_which_dies_with_children()
tests/child_processes_test.py:109: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/child_processes_test.py:95: in spawn_process_which_dies_with_children
    assert match, line
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
obj = b''
    def _format_assertmsg(obj):
        """Format the custom assertion message given.

        For strings this simply replaces newlines with '\n~' so that
        util.format_explanation() will preserve them instead of escaping
        newlines.  For other objects py.io.saferepr() is used first.

        """
        # reprlib appears to have a bug which means that if a string
        # contains a newline it gets escaped, however if an object has a
        # .__repr__() which contains newlines it does not get escaped.
        # However in either case we want to preserve the newline.
        if py.builtin._istext(obj) or py.builtin._isbytes(obj):
            s = obj
            is_repr = False
        else:
            s = py.io.saferepr(obj)
            is_repr = True
        if py.builtin._istext(s):
            t = py.builtin.text
        else:
            t = py.builtin.bytes
>       s = s.replace(t("\n"), t("\n~")).replace(t("%"), t("%%"))
E       TypeError: string argument without an encoding
/usr/lib/python3.5/site-packages/_pytest/assertion/rewrite.py:394: TypeError
----------------------------- Captured stderr call -----------------------------
[dumb-init] Running in debug mode.
[dumb-init] Child spawned with PID 19601.
[dumb-init] setsid complete.
[dumb-init] Received signal 17.
[dumb-init] A child with PID 19601 exited with exit status 0.
[dumb-init] Forwarded signal 15 to children.
[dumb-init] Child exited with status 0. Goodbye.
_____________ test_all_processes_receive_term_on_exit_if_setsid[0] _____________
    @pytest.mark.usefixtures('both_debug_modes', 'setsid_enabled')
    def test_all_processes_receive_term_on_exit_if_setsid():
        """If the child exits for some reason, dumb-init should send TERM to all
        processes in its session if setsid mode is enabled."""
>       child_pid, child_stdout = spawn_process_which_dies_with_children()
tests/child_processes_test.py:109: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/child_processes_test.py:95: in spawn_process_which_dies_with_children
    assert match, line
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
obj = b''
    def _format_assertmsg(obj):
        """Format the custom assertion message given.

        For strings this simply replaces newlines with '\n~' so that
        util.format_explanation() will preserve them instead of escaping
        newlines.  For other objects py.io.saferepr() is used first.

        """
        # reprlib appears to have a bug which means that if a string
        # contains a newline it gets escaped, however if an object has a
        # .__repr__() which contains newlines it does not get escaped.
        # However in either case we want to preserve the newline.
        if py.builtin._istext(obj) or py.builtin._isbytes(obj):
            s = obj
            is_repr = False
        else:
            s = py.io.saferepr(obj)
            is_repr = True
        if py.builtin._istext(s):
            t = py.builtin.text
        else:
            t = py.builtin.bytes
>       s = s.replace(t("\n"), t("\n~")).replace(t("%"), t("%%"))
E       TypeError: string argument without an encoding
/usr/lib/python3.5/site-packages/_pytest/assertion/rewrite.py:394: TypeError
==================== 2 failed, 171 passed in 21.36 seconds =====================

(note that the pytest stuff in the traceback is a red-herring caused by a pytest bug)

@muayyad-alsadi mentioned that it fails consistently on python3 but passes on python2:

it's consistantly failing, here is another build with same result.
https://kojipkgs.fedoraproject.org//work/tasks/553/15460553/build.log

Labeling as "tests" since it's almost certainly a test failure and not a dumb-init bug (since it passes when running the same tests under python2.7).

Understanding session behavior

I'm trying to understand when running dumb-init under a session (i.e. not using --single-child) is useful, and I'm having a hard time reproducing the problem case. Here's a script I'm running that just runs yes as both background and foreground:

#!/bin/sh
yes > /dev/null &
yes > /dev/null

And then I'm running dumb-init -c -- /script.sh:

$ ps aux | grep sh
...
docker   13014  0.8  0.0   1100     4 pts/0    Ss+  19:57   0:00 dumb-init -c -- /bin/sh script.sh
docker   13020  0.0  0.0   4448   776 pts/0    S+   19:57   0:00 /bin/sh script.sh
docker   13033  0.0  0.0   9760  1028 pts/1    S+   19:57   0:00 grep sh

$ ps aux | grep yes
...
docker   13021 99.2  0.0   4352   640 pts/0    R+   19:57   0:10 yes
docker   13022  101  0.0   4352   692 pts/0    R+   19:57   0:11 yes
docker   13039  0.0  0.0   9620   888 pts/1    R+   19:57   0:00 grep yes

When I run kill 13014, the sh process quits, and all yes processes quit as well. It's immediate, so this isn't the "TERM then wait 10s before KILL" behavior. This looks like good behavior, even though I'm using the -c flag. Am I incorrectly reproducing this?

Job control signal forwarding without setsid test is flaky

https://circleci.com/gh/chriskuehl/dumb-init/95 failed due to test flakes:

• httpredir redirected us to a bad mirror (not much can be done about this, we are very familiar with this problem...)
• Our test for job control signal forwarding with setsid disabled is flaky:

=================================== FAILURES ===================================
____________________________ test_prints_signals[1] ____________________________

both_debug_modes = None, setsid_disabled = None

    def test_prints_signals(both_debug_modes, setsid_disabled):
        """In non-setsid mode, dumb-init should forward the signals SIGTSTP,
        SIGTTOU, and SIGTTIN, and then suspend itself.
        """
        proc = Popen(
            ('dumb-init', sys.executable, '-m', 'tests.lib.print_signals'),
            stdout=PIPE,
        )

        assert re.match(b'^ready \(pid: (?:[0-9]+)\)\n$', proc.stdout.readline())

        for signum in SUSPEND_SIGNALS:
            assert process_state(proc.pid) in ['running', 'sleeping']
            proc.send_signal(signum)
            assert proc.stdout.readline() == '{0}\n'.format(signum).encode('ascii')
>           assert process_state(proc.pid) == 'stopped'
E           assert 'running' == 'stopped'
E             - running
E             + stopped

tests/shell_background_test.py:73: AssertionError
----------------------------- Captured stderr call -----------------------------
[dumb-init] Running in debug mode.
[dumb-init] Not running in setsid mode.
[dumb-init] Child spawned with PID 5001.
[dumb-init] Received signal 20.
[dumb-init] Not running in setsid mode, so forwarding the original signal (20).
[dumb-init] Forwarded signal 20 to children.
[dumb-init] Suspending self due to TTY signal.
===================== 1 failed, 94 passed in 2.29 seconds ======================

The equivalent test for setsid mode (behavior is a bit different due to kernel limitations) sleeps a bit. It's pretty obvious why there is a race here. We can probably avoid sleeping altogether using wait (to check for the process suspending/resuming due to signals).

Support tty / interactive scripts

My Dockerfile runs a bash script which sometimes need to be interactive (it can run a curl -u USERNAME and expects the user to type in a password).

ENTRYPOINT ["/usr/bin/dumb-init", "--", "/config-and-run.sh"]

That fails with dumb-init, I'm not given a chance to enter my password.

$ docker run -ti myimage
...
Enter host password for user 'jamshid':
curl: (22) The requested URL returned error: 400 Bad Request
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell

Would be nice if dumb-init supported it. FYI a similar project https://github.com/krallin/tini does support it.

dumb-init succeeds when its exec fails

This blocks 1.0 imo.

$ wattttttt ok
-bash: wattttttt: command not found
$ echo $?
127

$ dumb-init wattttttt ok
$ echo $?
0

Add "latest" tag so Dockerfile/build script doesn't need to be updated

instead of the 0.5.0 tag:
wget https://github.com/Yelp/dumb-init/releases/download/v0.5.0/dumb-init_0.5.0_amd64.deb

make it possible to reuse the URL, maybe something like this:
wget https://github.com/Yelp/dumb-init/releases/download/latest/dumb-init_amd64.deb

(.deb or binaries)
cheers!

itest_lucid failing because of broken ubuntu:lucid base image

docker-library/official-images#1902

prefix output

It would be better if dumb-init would prefix its output with [dumb-init].

$ pgctl start uwsgi
[pgctl] Starting: uwsgi
[pgctl] ERROR: 'uwsgi' timed out at 30 seconds: not ready
==> playground/uwsgi/stdout.log <==
Compiling www_pages/templates/lib/about.tmpl
Compiling www_pages/templates/lib/user_details_following.tmpl
Compiling www_pages/templates/lib/careers.tmpl
Compiling www_pages/templates/lib/user_details/profile_closed.tmpl
Compiling www_pages/templates/lib/user_details/profile_locked.tmpl
Compiling www_pages/templates/lib/engineering/engineering.tmpl
Compiling www_pages/templates/engineering/events.tmpl
Compiling www_pages/templates/engineering/event.tmpl
Compiling www_pages/templates/engineering/engineering.tmpl
YELP_ASSETS_DEV=1 ./virtualenv_run/bin/yelp-assets-build

==> playground/uwsgi/stderr.log <==
your memory page size is 4096 bytes
detected max file descriptor number: 10240
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
*** Cache "healthcheck" initialized: 2MB (key: 2136 bytes, keys: 68352 bytes, data: 2097152 bytes, bitmap: 0 bytes) preallocated ***
probably another instance of uWSGI is running on the same address (:9003).
bind(): Address already in use [core/socket.c line 761]
A child with PID 2762741 exited with exit status 1.
Forwarded signal 15 to child.
Child exited with status 1. Goodbye.
[pgctl] Stopping: uwsgi
[pgctl] Stopped: uwsgi
[pgctl] ERROR: Some services failed to start: uwsgi

dumb-init package for alpine linux

Would it be possible to add packages for alpine linux?

Now it is possible to install it via python but it would be nice to have it available in package repository same as debian.

apk --update-cache py-pip
pip install dumb-init

Running dumb-init as "non-root"

Hi there,

I have created a container with dumb-init as ENTRYPOINT and a USER-statement right before that in the Dockerfile, i.e. dumb-init runs as non-root. As all processes running inside of the container run under the same user-id (as they are spawned by dumb-init), I cannot see a reason why dumb-init could not just perform the work it is designed to do (i.e. reap zombies and relay signals), even when running as non-root. After all, it has all necessary permissions to deal with processes under the same user-id.

Is this thinking correct, or should dumb-init indeed run as root ?

In this case, one would have to explicitly switch the user once dumb-init has started (e.g. using gosu), so as not to run other processes as root.

Thanks and Kind Regards,
Beet

Official dumb-init image on Docker Hub?

I see a few dumb-init images on the docker hub, but none look "official".

Might be worth publishing an Ubuntu-based image with dumb-init added, maybe some other base OSes.

cli_test.test_verbose is flaky depending on fork race

21:49:40 =================================== FAILURES ===================================
21:49:40 _______________________________ test_verbose[-v] _______________________________
21:49:40 
21:49:40 flag = '-v'
21:49:40 
21:49:40     @pytest.mark.parametrize('flag', ['-v', '--verbose'])
21:49:40     def test_verbose(flag):
21:49:40         """dumb-init should print debug output when asked to."""
21:49:40         proc = Popen(('dumb-init', flag, 'echo', 'oh,', 'hi'), stdout=PIPE, stderr=PIPE)
21:49:40         stdout, stderr = proc.communicate()
21:49:40         assert proc.returncode == 0
21:49:40         assert stdout == b'oh, hi\n'
21:49:40 >       assert re.match(
21:49:40             (
21:49:40                 b'^\[dumb-init\] Child spawned with PID [0-9]+\.\n'
21:49:40                 b'\[dumb-init\] setsid complete\.\n'
21:49:40                 b'\[dumb-init\] Received signal 17\.\n'
21:49:40                 b'\[dumb-init\] A child with PID [0-9]+ exited with exit status 0.\n'
21:49:40                 b'\[dumb-init\] Forwarded signal 15 to children\.\n'
21:49:40                 b'\[dumb-init\] Child exited with status 0\. Goodbye\.\n$'
21:49:40             ),
21:49:40             stderr,
21:49:40         ), stderr
21:49:40 E       AssertionError: [dumb-init] setsid complete.
21:49:40 E         [dumb-init] Child spawned with PID 12915.
21:49:40 E         [dumb-init] Received signal 17.
21:49:40 E         [dumb-init] A child with PID 12915 exited with exit status 0.
21:49:40 E         [dumb-init] Forwarded signal 15 to children.
21:49:40 E         [dumb-init] Child exited with status 0. Goodbye.
21:49:40 E         
21:49:40 E       assert None
21:49:40 E        +  where None = <function match at 0x2af9ace03c80>('^\\[dumb-init\\] Child spawned with PID [0-9]+\\.\n\\[dumb-init\\] setsid complete\\.\n\\[dumb-init\\] Received signa...us 0.\n\\[dumb-init\\] Forwarded signal 15 to children\\.\n\\[dumb-init\\] Child exited with status 0\\. Goodbye\\.\n$', '[dumb-init] setsid complete.\n[dumb-init] Child spawned with PID 12915.\n[dumb-init] Received signal 17.\n[dumb-init]... with exit status 0.\n[dumb-init] Forwarded signal 15 to children.\n[dumb-init] Child exited with status 0. Goodbye.\n')
21:49:40 E        +    where <function match at 0x2af9ace03c80> = re.match
21:49:40 
21:49:40 tests/cli_test.py:85: AssertionError

4 tests erros when validating build in rpm spec

when I activated testing the build with

PATH=.:$PATH py.test tests/

I got

4 failed, 169 passed in 3.06 seconds

here is the full build logs

https://paste.fedoraproject.org/414308/22152031/

Release binary instead of DEB

Hi, Firstly thanks for the awesome utility. Just quickly wanted to ask: Why release a DEB, when such a compact utility like this can be released straight away as a simple binary? wget, chmod and run would be nice right?

manylinux wheel?

Figure we might as well :)

Here's how I do this for libsass: https://github.com/dahlia/libsass-python/blob/e9b7421798c03f922e10ce440629520074efd90e/build_manylinux_wheels.py

[Question] How to generate zombie processes to test PID 1 problem.

I would like to gain some insight into the docker zombie process. Is there a simple way to produce a zombie in a docker container to see if it is reaped or not?

I am trying to create a zombie process that will persist even after the container is closed (that's what I understand the problem is in any case) so I can prove that using dumb-init as my ENTRYPOINT will fix the problem. However I haven't been able to show that the zombie process I created will stay no matter how I terminate my container or no matter how i terminate the parent of the zombie process.

I'm running nvidia/cuda:7.5-devel for the base image and docker v1.12.3. I've created a zombie in my container by having this script run as PID 1 in my dockerfile via CMD:

#!/bin/bash
# Spawn a zombie process whose parent is this script which is PID 1 in a container.
(sleep 1 & exec /bin/sleep 200000) &

tail -f /dev/null 

I started my test by launching 2 containers, 1 with dumb init as PID 1 (which launches my script) and 1 with the script as PID 1.

I can see the zombies here

:~/dockerfiles$ ps -ef | grep sleep
root     30599 30597  0 16:24 ?        00:00:00 /bin/sleep 200000
root     30600 30599  0 16:24 ?        00:00:00 [sleep] <defunct>
root     30746 30722  0 16:24 ?        00:00:00 /bin/sleep 200000
root     30747 30746  0 16:24 ?        00:00:00 [sleep] <defunct>

but when I try to kill the parents either outside of inside of the container, the zombies are reaped even in the container not running dumb-init.

I'm looking for a way to reproduce this PID1 problem so I can show that dumb-init is doing this job because so far I haven't been able to show that zombies will stick to the host OS after the containers are killed, on the contrary the zombie processes are immediately reaped when I kill their parent process even if the container is still running.

Question about ENTRYPOINT use

I'm writing up a base Dockerfile with dumb-init and was looking at some existing examples.

I've reached the point where I'm dealing with ENTRYPOINT and am a bit confused by something I've seen in someone else's Dockerfile that uses dumb-init.

They're using ENTRYPOINT ["dumb-init", "--"] - it's the second value there that I'm confused about ("--").

I've tried running mine with and without it, and they both work. I'm not sure if anything behaves differently. I've looked through dumb-init's source code to try and find anything about that, without success.

So is this a random typo from that author, or is anyone else using it and knows whether or not it does anything?

Using an existing entrypoint in your container

Is it possible to add an existing entrypoint script as tini is doing ?

ENTRYPOINT ["/tini", "--", "/docker-entrypoint.sh"]

Thanks

Allow specifying signal names for --rewrite

It would be nice to be able to specify --rewrite TERM:HUP rather than using signal numbers.

Unfortunately doesn't look like there is a list of signal names available in any common headers. There's sys_siglist (from string.h), but the names are like Hangup and Interrupt, rather than the short macro names that everyone uses.

From looking at the kill source, they define an array like this:

static const mapstruct sigtable[] = {
  {"ABRT",   SIGABRT},  /* IOT */
  {"ALRM",   SIGALRM},
  {"BUS",    SIGBUS},
  {"CHLD",   SIGCHLD},  /* CLD */
  {"CONT",   SIGCONT},
  {"FPE",    SIGFPE},
  {"HUP",    SIGHUP},
  {"ILL",    SIGILL},
  {"INT",    SIGINT},
  {"KILL",   SIGKILL},
  {"PIPE",   SIGPIPE},
  {"POLL",   SIGPOLL},  /* IO */
  {"PROF",   SIGPROF},
  {"PWR",    SIGPWR},
  {"QUIT",   SIGQUIT},
  {"SEGV",   SIGSEGV},
  {"STOP",   SIGSTOP},
  {"SYS",    SIGSYS},   /* UNUSED */
  {"TERM",   SIGTERM},
  {"TRAP",   SIGTRAP},
  {"TSTP",   SIGTSTP},
  {"TTIN",   SIGTTIN},
  {"TTOU",   SIGTTOU},
  {"URG",    SIGURG},
  {"USR1",   SIGUSR1},
  {"USR2",   SIGUSR2},
  {"VTALRM", SIGVTALRM},
  {"WINCH",  SIGWINCH},
  {"XCPU",   SIGXCPU},
  {"XFSZ",   SIGXFSZ}
};

It's not the absolute worst (they're still using the signal macros, so they don't need to worry about signal numbers not matching the ordering in the file (or it being recompiled on a different architecture with different numbers)).

Is it worth it to do something like this in dumb-init? (or is there a better way?)

compare with coreutils timeout

The timeout program has almost the same problem description as dumb-init: start a process tree, then kill the whole tree. If you look at their code though, the implementation is quite different.

Before public release, we should do some due diligence and figure out if their implementation decisions are better than ours, or not, and why.

http://git.savannah.gnu.org/cgit/coreutils.git/tree/src/timeout.c

dumb-init doesn't uninstall

I'm not entirely sure how to reproduce this. It reproduces in yelp-main but not in an empty virtualenv.

$ rm virtualenv_run/bin/dumb-init
$ which dumb-init
(no output)

$ pip install dumb-init
Downloading/unpacking dumb-init
  Downloading dumb-init-1.0.0.tar.gz
  Running setup.py (path:/nail/home/buck/pg/master/virtualenv_run/build/dumb-init/setup.py) egg_info for package dumb-init

Installing collected packages: dumb-init
  Running setup.py install for dumb-init
    cc -c dumb-init.c -o build/temp.linux-x86_64-2.7/dumb-init.o
    cc build/temp.linux-x86_64-2.7/dumb-init.o -o build/scripts-2.7/dumb-init -static

  Could not find .egg-info directory in install record for dumb-init
Successfully installed dumb-init
Cleaning up...

$ which dumb-init
/nail/home/buck/pg/master/virtualenv_run/bin/dumb-init

$ pip uninstall dumb-init
Uninstalling dumb-init:
  /nail/home/buck/pg/master/virtualenv_run/lib/python2.7/site-packages/dumb_init-1.0.0-py2.7.egg-info
Proceed (y/n)? y
  Successfully uninstalled dumb-init

$ which dumb-init
/nail/home/buck/pg/master/virtualenv_run/bin/dumb-init

cli_test.{test_verbose,test_verbose_and_single_child} fail on some architectures due to signal number assertions

Signals have different numbers on different architectures, and these tests assert that SIGCHLD is 17 (but on mips, it is 18!).

Should be easy to fix by just formatting in signal.SIGCHLD to the assertion string.

The Debian mips build for dumb-init failed: https://buildd.debian.org/status/fetch.php?pkg=dumb-init&arch=mips&ver=1.1.2-1&stamp=1469941155&file=log

So did the mipsel build: https://buildd.debian.org/status/fetch.php?pkg=dumb-init&arch=mipsel&ver=1.1.2-1&stamp=1469941005&file=log

=================================== FAILURES ===================================
_______________________________ test_verbose[-v] _______________________________

flag = '-v'

    @pytest.mark.parametrize('flag', ['-v', '--verbose'])
    def test_verbose(flag):
        """dumb-init should print debug output when asked to."""
        proc = Popen(('dumb-init', flag, 'echo', 'oh,', 'hi'), stdout=PIPE, stderr=PIPE)
        stdout, stderr = proc.communicate()
        assert proc.returncode == 0
        assert stdout == b'oh, hi\n'

        # child/parent race to print output after the fork(), can't guarantee exact order
        assert re.search(b'(^|\n)\[dumb-init\] setsid complete\.\n', stderr), stderr  # child
>       assert re.search(  # parent
            (
                b'(^|\n)\[dumb-init\] Child spawned with PID [0-9]+\.\n'
                b'.*'  # child might print here
                b'\[dumb-init\] Received signal 17\.\n'
                b'\[dumb-init\] A child with PID [0-9]+ exited with exit status 0.\n'
                b'\[dumb-init\] Forwarded signal 15 to children\.\n'
                b'\[dumb-init\] Child exited with status 0\. Goodbye\.\n$'
            ),
            stderr,
            re.DOTALL,
        ), stderr
E       AssertionError: [dumb-init] setsid complete.
E         [dumb-init] Child spawned with PID 27262.
E         [dumb-init] Received signal 18.
E         [dumb-init] A child with PID 27262 exited with exit status 0.
E         [dumb-init] Forwarded signal 15 to children.
E         [dumb-init] Child exited with status 0. Goodbye.
E         
E       assert None
E        +  where None = <function search at 0x77249270>('(^|\n)\\[dumb-init\\] Child spawned with PID [0-9]+\\.\n.*\\[dumb-init\\] Received signal 17\\.\n\\[dumb-init\\] A ch...us 0.\n\\[dumb-init\\] Forwarded signal 15 to children\\.\n\\[dumb-init\\] Child exited with status 0\\. Goodbye\\.\n$', '[dumb-init] setsid complete.\n[dumb-init] Child spawned with PID 27262.\n[dumb-init] Received signal 18.\n[dumb-init]... with exit status 0.\n[dumb-init] Forwarded signal 15 to children.\n[dumb-init] Child exited with status 0. Goodbye.\n', 16)
E        +    where <function search at 0x77249270> = re.search
E        +    and   16 = re.DOTALL

Name change ?

Hi there,

any chance of a name change for dumb-init, e.g. to smart-init ? I find dumb-init pretty awesome and feel that it might profit from being named differently:-)

Kind Regards,
Beet

--version, --help, usage

We should support these minimal expected cli behaviors:

$ dumb-init
usage: dumb-init <your program here>

$ dumb-init --version
0.5.0

$ dumb-init --help
usage: dumb-init <your program here>

`dumb-init` can serve as PID1 in a minimal system. It any recieved signals to its child.
This is particularly useful in the context of docker; it makes ctrl-c and friends Just Work. 

For bonus credit, make -v set debug to 1.