nicolas-van/multirun

A minimalist init process designed for Docker

Passionate developer, loving dad.

multirun's Issues

Debian and Ubuntu packages

Hi! Thanks for this great work! I hope it will be spreaded widely as a great, simple alternative to supervisord!

I wonder if there are plans for Debian and Ubuntu packages, so it could be easily installed on non-alpine images?

Forward stdin

Thanks Nicolas for building multirun, it made building docker containers so much more simple compared to supervisord. While extending the docker images I build to some more I found an interesting use case that I believe still fits multirun:

Take for example you are launching e.g. two processes with multirun.

1. The first one is a CLI application that is packaged in docker because it needs very specific libraries.
2. The second one (and some more) are just helper daemons that need to run on the container (e.g. some socket services)

That works perfectly as expected. Very good work! But if you extend the CLI application to now also ask questions on the stdin from the user this does not work anymore as the stdin is not forwarded.

My idea would be to always forward stdin to the first process in the multirun list of commands to run. For anyone using multirun until now nothing changes. But for the few also wanting to use stdin they can now have one process responsible to handle the stdin. That's also important for application servers that launch an http server and provide a stdin console to control the server.

appearance in `ps` is weird

Runs fine, but appearance in ps is odd:

1000100+      5  0.0  0.0   4308   612 ?        S    17:48   0:00 multirun php-fpm -F httpd -D FOREGROUND tail --retry -f /var/log/php-fpm/www-error.log                     
1000100+      6  0.0  0.9 369116 38988 ?        Ss   17:48   0:00  \_ php-fpm: master process (/etc/php-fpm.conf)                                                            
1000100+    319  0.0  1.2 375356 50468 ?        S    17:50   0:00  |   \_ php-fpm: pool www                                                                                  
1000100+      7  0.0  0.2 265052 10468 ?        S    17:48   0:00  \_ httpd -D FOREGROUND                                                                                    
1000100+     11  0.0  0.0   4344   732 ?        S    17:48   0:00  |   \_ /usr/bin/cat                                                                                       
1000100+     28  0.0  0.0   4344   728 ?        S    17:48   0:00  |   \_ /usr/bin/cat                                                                                       
1000100+     29  0.0  0.1 264784  4428 ?        S    17:48   0:00  |   \_ httpd -D FOREGROUND                                                                                
1000100+     30  0.0  0.1 1781300 7976 ?        Sl   17:48   0:00  |   \_ httpd -D FOREGROUND                                                                                
1000100+     39  0.0  0.1 1781236 7976 ?        Sl   17:48   0:00  |   \_ httpd -D FOREGROUND                                                                                
1000100+     69  0.0  0.1 929268  7976 ?        Sl   17:48   0:00  |   \_ httpd -D FOREGROUND                                                                                
1000100+    247  0.0  0.2 1781236 8220 ?        Sl   17:49   0:00  |   \_ httpd -D FOREGROUND                                                                                
1000100+      8  0.0  0.0   4364   768 ?        S    17:48   0:00  \_ tail --retry -f /var/log/php-fpm/www-error.log

The actual command was:

multirun "php-fpm -F" "httpd -D FOREGROUND" "tail --retry -f /var/log/php-fpm/www-error.log"

Very minor but just wanted to note.

Prebuilt arm64 binaries

With more and more ARM computers now in use due to new Apple Silicon computers, it would be very nice to provide prebuilt amd64 and arm64 in github releases for easy docker container creation

Zombie processes

Does it handle zombie processes correctly like Tini (or any other init manager) ?

Some info on the subject for those who may be interested : krallin/tini#8 (comment)

edit: typo

NOISSUE: one or more of the provided commands ended abnormally

It would be nice to know which one of the commands failed / ended:

Affects: "multirun: one or more of the provided commands ended abnormally"

For example:
"multirun: command #1 ended abnormally"
"multirun: command #1, #3 ended abnormally"

Exception on run: Access is allowed from Event Dispatch Thread (EDT) only

Hello,

I am encountering this exception every time I run multirun Run Configuration.
Exception is thrown, but everything runs normally. So really just a cosmetic issue (I think).

Run Configuration:
docker-compose, spring boot, spring boot, npm

log:

Current thread: Thread[ApplicationImpl pooled thread 50,4,main] 43535799 (EventQueue.isDispatchThread()=false)
SystemEventQueueThread: Thread[AWT-EventQueue-0,6,main] 1660539600
com.intellij.openapi.diagnostic.RuntimeExceptionWithAttachments: Access is allowed from Event Dispatch Thread (EDT) only; see https://jb.gg/ij-platform-threading for details
Current thread: Thread[ApplicationImpl pooled thread 50,4,main] 43535799 (EventQueue.isDispatchThread()=false)
SystemEventQueueThread: Thread[AWT-EventQueue-0,6,main] 1660539600
at com.intellij.openapi.application.impl.ApplicationImpl.createThreadAccessException(ApplicationImpl.java:1083)
at com.intellij.openapi.application.impl.ApplicationImpl.throwThreadAccessException(ApplicationImpl.java:1078)
at com.intellij.openapi.application.impl.ApplicationImpl.assertIsDispatchThread(ApplicationImpl.java:1066)
at com.intellij.openapi.wm.impl.ToolWindowManagerImpl.registerToolWindow(ToolWindowManagerImpl.kt:1019)
at com.jetbrains.rdserver.toolWindow.BackendServerToolWindowManager.registerToolWindow(BackendServerToolWindowManager.kt:114)
at com.intellij.execution.ui.RunContentManagerImpl.registerToolWindow(RunContentManagerImpl.kt:155)
at com.intellij.execution.ui.RunContentManagerImpl.getOrCreateContentManagerForToolWindow(RunContentManagerImpl.kt:432)
at com.intellij.execution.ui.RunContentManagerImpl.getContentManagerForRunner(RunContentManagerImpl.kt:416)
at com.intellij.execution.ui.RunContentManagerImpl.getReuseContent(RunContentManagerImpl.kt:391)
at com.intellij.execution.impl.ExecutionManagerImpl.doStartRunProfile(ExecutionManagerImpl.kt:243)
at com.intellij.execution.impl.ExecutionManagerImpl.startRunProfile(ExecutionManagerImpl.kt:205)
at com.intellij.remoteServer.impl.runtime.DeployToServerRunner.execute(DeployToServerRunner.kt:20)
at com.intellij.docker.before.DockerBeforeRunRunner.execute(DockerBeforeRunRunner.kt:43)
at com.khmelyuk.multirun.MultirunRunnerState.runConfigurations(MultirunRunnerState.java:273)
at com.khmelyuk.multirun.MultirunRunnerState.lambda$execute$0(MultirunRunnerState.java:76)
at com.intellij.openapi.application.impl.ApplicationImpl$2.run(ApplicationImpl.java:272)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at com.intellij.util.concurrency.ContextCallable.call(ContextCallable.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at com.intellij.util.concurrency.ContextRunnable.run(ContextRunnable.java:24)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:702)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:699)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:699)
at java.base/java.lang.Thread.run(Thread.java:833)

CTRL + C or docker stop <container-name> does not kills all the processes

I run two processes in a container (Gunicorn and Nginx)
when I kill one of the processes in the container, others stop automatically.
logs:

time="2022-02-09T13:54:52+03:30" level=info msg="received terminated, shutting down"
time="2022-02-09T13:54:52+03:30" level=info msg="waiting for jobs to finish"
time="2022-02-09T13:54:52+03:30" level=info msg=exiting
[2022-02-09 13:54:52 +0330] [18] [INFO] Handling signal: term
[2022-02-09 10:24:52 +0000] [31] [INFO] Worker exiting (pid: 31)
[2022-02-09 10:24:52 +0000] [30] [INFO] Worker exiting (pid: 30)
[2022-02-09 10:24:52 +0000] [32] [INFO] Worker exiting (pid: 32)
[2022-02-09 13:54:52 +0330] [18] [INFO] Shutting down: Master
2022-02-09 13:54:52,981 WARN received SIGTERM indicating exit request
2022-02-09 13:54:52,981 WARN received SIGTERM indicating exit request
2022-02-09 13:54:52,981 INFO waiting for celery-worker_00 to die
2022-02-09 13:54:52,981 INFO waiting for celery-worker_00 to die

worker: Warm shutdown (MainProcess)

But when I try to stop the container by CTRL + C nothing happens.
Or when I try to stop the container with docker stop <container-name> command it takes about 10 seconds then the container stops but there is no log about sending a signal to the subprocess.

Request to modify multirun behavior regarding SIGTERM handling

Greetings! I'd like to express my appreciation for this fantastic tool.

Within my container environment, I'm utilizing Gunicorn and Celery orchestrated by multirun. Celery functions as a master process, spawning numerous worker processes based on its configurations. However, when multirun receives a SIGTERM signal, it propagates it to all processes, including those spawned by Celery but not directly managed by multirun.

The crux of the matter arises during the graceful shutdown of Celery. Ideally, we seek to terminate only the main Celery worker process, without affecting its child processes. Additional insights into this requirement can be found here: Celery FAQ.

Is there a way to disable this default behavior or tailor it for specific process groups during the startup of multirun?

Docker Image to use in multi-stage builds

You can use docker buildx to build an image from scratch on each architecture and upload to https://hub.docker.com/u/nicolasvan
So we can use these images as intermediaries stages in the final build process without relying on scripts to validate the architecture.
Example of Usage:

FROM debian:bullseye-slim 
...
RUN apt-get install php-fpm nginx
...
COPY --from=nicolasvan/multirun:1.1.3-glibc /multirun /usr/bin/multirun
CMD ["/usr/bin/multirun", "/usr/sbin/php-fpm",  "/usr/sbin/nginx"]

Examples files do build

Dockerfile to build multirun in Docker multi-arch

# syntax=docker/dockerfile:1
FROM debian:bullseye-slim as build
ENV DEBIAN_FRONTEND=noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN=true
WORKDIR /srv/multirun
RUN apt-get -qq update && \
    apt-get -qq install -y wget ca-certificates tar gzip cmake bats && \
    wget -q -O- https://github.com/nicolas-van/multirun/archive/refs/tags/1.1.3.tar.gz | tar -xz --strip-components 1 && \
    cmake -S . -B build && \
    cmake --build build && \
    bats tests.bats

FROM scratch
COPY --from=build /srv/multirun/build/multirun /multirun

And build with

#!/bin/bash
# Setup host if needed
docker run -it --rm --privileged tonistiigi/binfmt --install arm64
# Run Build
docker buildx build --push --platform linux/amd64    --tag nicolasvan/multirun:1.1.3-glibc .
docker buildx build --push --platform linux/arm64/v8 --tag nicolasvan/multirun:1.1.3-glibc .

You can create this build in Github Actions and change wget to a copy using content from checkout action

Published to Habitat

Hey there, thanks for building this!

I made a plan for building and published it to my favorite package manager, Chef Habitat

Anyone with the hab client installed could use the package like this:

$ hab pkg install --binlink jarvus/multirun
» Installing jarvus/multirun
☁ Determining latest version of jarvus/multirun in the 'stable' channel
↓ Downloading jarvus/multirun/0.3.2/20190925020410
☛ Verifying jarvus/multirun/0.3.2/20190925020410
→ Using core/glibc/2.27/20190115002733
→ Using core/linux-headers/4.17.12/20190115002705
✓ Installed jarvus/multirun/0.3.2/20190925020410
★ Install of jarvus/multirun/0.3.2/20190925020410 complete with 1 new packages installed.
» Binlinking multirun from jarvus/multirun/0.3.2/20190925020410 into /hab/bin
★ Binlinked multirun from jarvus/multirun/0.3.2/20190925020410 to /hab/bin/multirun

$ multirun -h
Usage: multirun <options> command...

or like this:

$ hab pkg install jarvus/multirun
» Installing jarvus/multirun
☁ Determining latest version of jarvus/multirun in the 'stable' channel
↓ Downloading jarvus/multirun/0.3.2/20190925020410
☛ Verifying jarvus/multirun/0.3.2/20190925020410
→ Using core/glibc/2.27/20190115002733
→ Using core/linux-headers/4.17.12/20190115002705
✓ Installed jarvus/multirun/0.3.2/20190925020410
★ Install of jarvus/multirun/0.3.2/20190925020410 complete with 1 new packages installed.

$ hab pkg exec jarvus/multirun multirun -h
Usage: multirun <options> command...

or like this:

FROM habitat/default-studio-x86_64-linux:0.85.0

RUN hab pkg install --binlink \
        jarvus/multirun \
        core/nginx \
        core/php

ENTRYPOINT ["multirun"]

If you're interested in publishing to Habitat yourself under your own origin name, just register your own origin and post what it is, and I'll open a PR for you adding the plan.sh in your repo root or under habitat/ and configure it for building right from your repo. Then after you linked the repo, the Habitat bldr would handle rebuilding the binary on any push to master or update to glibc getting marked stable