Passionate developer, loving dad.
nicolas-van / multirun Goto Github PK
View Code? Open in Web Editor NEWA minimalist init process designed for Docker
Home Page: https://nicolas-van.github.io/multirun/
License: MIT License
A minimalist init process designed for Docker
Home Page: https://nicolas-van.github.io/multirun/
License: MIT License
Passionate developer, loving dad.
Hi! Thanks for this great work! I hope it will be spreaded widely as a great, simple alternative to supervisord!
I wonder if there are plans for Debian and Ubuntu packages, so it could be easily installed on non-alpine images?
Thanks Nicolas for building multirun, it made building docker containers so much more simple compared to supervisord. While extending the docker images I build to some more I found an interesting use case that I believe still fits multirun:
Take for example you are launching e.g. two processes with multirun.
That works perfectly as expected. Very good work! But if you extend the CLI application to now also ask questions on the stdin from the user this does not work anymore as the stdin is not forwarded.
My idea would be to always forward stdin to the first process in the multirun list of commands to run. For anyone using multirun until now nothing changes. But for the few also wanting to use stdin they can now have one process responsible to handle the stdin. That's also important for application servers that launch an http server and provide a stdin console to control the server.
Runs fine, but appearance in ps
is odd:
1000100+ 5 0.0 0.0 4308 612 ? S 17:48 0:00 multirun php-fpm -F httpd -D FOREGROUND tail --retry -f /var/log/php-fpm/www-error.log
1000100+ 6 0.0 0.9 369116 38988 ? Ss 17:48 0:00 \_ php-fpm: master process (/etc/php-fpm.conf)
1000100+ 319 0.0 1.2 375356 50468 ? S 17:50 0:00 | \_ php-fpm: pool www
1000100+ 7 0.0 0.2 265052 10468 ? S 17:48 0:00 \_ httpd -D FOREGROUND
1000100+ 11 0.0 0.0 4344 732 ? S 17:48 0:00 | \_ /usr/bin/cat
1000100+ 28 0.0 0.0 4344 728 ? S 17:48 0:00 | \_ /usr/bin/cat
1000100+ 29 0.0 0.1 264784 4428 ? S 17:48 0:00 | \_ httpd -D FOREGROUND
1000100+ 30 0.0 0.1 1781300 7976 ? Sl 17:48 0:00 | \_ httpd -D FOREGROUND
1000100+ 39 0.0 0.1 1781236 7976 ? Sl 17:48 0:00 | \_ httpd -D FOREGROUND
1000100+ 69 0.0 0.1 929268 7976 ? Sl 17:48 0:00 | \_ httpd -D FOREGROUND
1000100+ 247 0.0 0.2 1781236 8220 ? Sl 17:49 0:00 | \_ httpd -D FOREGROUND
1000100+ 8 0.0 0.0 4364 768 ? S 17:48 0:00 \_ tail --retry -f /var/log/php-fpm/www-error.log
The actual command was:
multirun "php-fpm -F" "httpd -D FOREGROUND" "tail --retry -f /var/log/php-fpm/www-error.log"
Very minor but just wanted to note.
With more and more ARM computers now in use due to new Apple Silicon computers, it would be very nice to provide prebuilt amd64 and arm64 in github releases for easy docker container creation
Does it handle zombie processes correctly like Tini (or any other init manager) ?
Some info on the subject for those who may be interested : krallin/tini#8 (comment)
edit: typo
Hello,
I am encountering this exception every time I run multirun Run Configuration.
Exception is thrown, but everything runs normally. So really just a cosmetic issue (I think).
Run Configuration:
docker-compose, spring boot, spring boot, npm
log:
Current thread: Thread[ApplicationImpl pooled thread 50,4,main] 43535799 (EventQueue.isDispatchThread()=false)
SystemEventQueueThread: Thread[AWT-EventQueue-0,6,main] 1660539600
com.intellij.openapi.diagnostic.RuntimeExceptionWithAttachments: Access is allowed from Event Dispatch Thread (EDT) only; see https://jb.gg/ij-platform-threading for details
Current thread: Thread[ApplicationImpl pooled thread 50,4,main] 43535799 (EventQueue.isDispatchThread()=false)
SystemEventQueueThread: Thread[AWT-EventQueue-0,6,main] 1660539600
at com.intellij.openapi.application.impl.ApplicationImpl.createThreadAccessException(ApplicationImpl.java:1083)
at com.intellij.openapi.application.impl.ApplicationImpl.throwThreadAccessException(ApplicationImpl.java:1078)
at com.intellij.openapi.application.impl.ApplicationImpl.assertIsDispatchThread(ApplicationImpl.java:1066)
at com.intellij.openapi.wm.impl.ToolWindowManagerImpl.registerToolWindow(ToolWindowManagerImpl.kt:1019)
at com.jetbrains.rdserver.toolWindow.BackendServerToolWindowManager.registerToolWindow(BackendServerToolWindowManager.kt:114)
at com.intellij.execution.ui.RunContentManagerImpl.registerToolWindow(RunContentManagerImpl.kt:155)
at com.intellij.execution.ui.RunContentManagerImpl.getOrCreateContentManagerForToolWindow(RunContentManagerImpl.kt:432)
at com.intellij.execution.ui.RunContentManagerImpl.getContentManagerForRunner(RunContentManagerImpl.kt:416)
at com.intellij.execution.ui.RunContentManagerImpl.getReuseContent(RunContentManagerImpl.kt:391)
at com.intellij.execution.impl.ExecutionManagerImpl.doStartRunProfile(ExecutionManagerImpl.kt:243)
at com.intellij.execution.impl.ExecutionManagerImpl.startRunProfile(ExecutionManagerImpl.kt:205)
at com.intellij.remoteServer.impl.runtime.DeployToServerRunner.execute(DeployToServerRunner.kt:20)
at com.intellij.docker.before.DockerBeforeRunRunner.execute(DockerBeforeRunRunner.kt:43)
at com.khmelyuk.multirun.MultirunRunnerState.runConfigurations(MultirunRunnerState.java:273)
at com.khmelyuk.multirun.MultirunRunnerState.lambda$execute$0(MultirunRunnerState.java:76)
at com.intellij.openapi.application.impl.ApplicationImpl$2.run(ApplicationImpl.java:272)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at com.intellij.util.concurrency.ContextCallable.call(ContextCallable.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at com.intellij.util.concurrency.ContextRunnable.run(ContextRunnable.java:24)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:702)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:699)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:699)
at java.base/java.lang.Thread.run(Thread.java:833)
I run two processes in a container (Gunicorn and Nginx)
when I kill one of the processes in the container, others stop automatically.
logs:
time="2022-02-09T13:54:52+03:30" level=info msg="received terminated, shutting down"
time="2022-02-09T13:54:52+03:30" level=info msg="waiting for jobs to finish"
time="2022-02-09T13:54:52+03:30" level=info msg=exiting
[2022-02-09 13:54:52 +0330] [18] [INFO] Handling signal: term
[2022-02-09 10:24:52 +0000] [31] [INFO] Worker exiting (pid: 31)
[2022-02-09 10:24:52 +0000] [30] [INFO] Worker exiting (pid: 30)
[2022-02-09 10:24:52 +0000] [32] [INFO] Worker exiting (pid: 32)
[2022-02-09 13:54:52 +0330] [18] [INFO] Shutting down: Master
2022-02-09 13:54:52,981 WARN received SIGTERM indicating exit request
2022-02-09 13:54:52,981 WARN received SIGTERM indicating exit request
2022-02-09 13:54:52,981 INFO waiting for celery-worker_00 to die
2022-02-09 13:54:52,981 INFO waiting for celery-worker_00 to die
worker: Warm shutdown (MainProcess)
But when I try to stop the container by CTRL + C
nothing happens.
Or when I try to stop the container with docker stop <container-name>
command it takes about 10 seconds then the container stops but there is no log about sending a signal to the subprocess.
Greetings! I'd like to express my appreciation for this fantastic tool.
Within my container environment, I'm utilizing Gunicorn and Celery orchestrated by multirun. Celery functions as a master process, spawning numerous worker processes based on its configurations. However, when multirun receives a SIGTERM
signal, it propagates it to all processes, including those spawned by Celery but not directly managed by multirun.
The crux of the matter arises during the graceful shutdown of Celery. Ideally, we seek to terminate only the main Celery worker process, without affecting its child processes. Additional insights into this requirement can be found here: Celery FAQ.
Is there a way to disable this default behavior or tailor it for specific process groups during the startup of multirun?
You can use docker buildx to build an image from scratch on each architecture and upload to https://hub.docker.com/u/nicolasvan
So we can use these images as intermediaries stages in the final build process without relying on scripts to validate the architecture.
Example of Usage:
FROM debian:bullseye-slim
...
RUN apt-get install php-fpm nginx
...
COPY --from=nicolasvan/multirun:1.1.3-glibc /multirun /usr/bin/multirun
CMD ["/usr/bin/multirun", "/usr/sbin/php-fpm", "/usr/sbin/nginx"]
Dockerfile
to build multirun in Docker multi-arch
# syntax=docker/dockerfile:1
FROM debian:bullseye-slim as build
ENV DEBIAN_FRONTEND=noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN=true
WORKDIR /srv/multirun
RUN apt-get -qq update && \
apt-get -qq install -y wget ca-certificates tar gzip cmake bats && \
wget -q -O- https://github.com/nicolas-van/multirun/archive/refs/tags/1.1.3.tar.gz | tar -xz --strip-components 1 && \
cmake -S . -B build && \
cmake --build build && \
bats tests.bats
FROM scratch
COPY --from=build /srv/multirun/build/multirun /multirun
And build with
#!/bin/bash
# Setup host if needed
docker run -it --rm --privileged tonistiigi/binfmt --install arm64
# Run Build
docker buildx build --push --platform linux/amd64 --tag nicolasvan/multirun:1.1.3-glibc .
docker buildx build --push --platform linux/arm64/v8 --tag nicolasvan/multirun:1.1.3-glibc .
You can create this build in Github Actions and change wget to a copy using content from checkout action
Hey there, thanks for building this!
I made a plan for building and published it to my favorite package manager, Chef Habitat
Anyone with the hab
client installed could use the package like this:
$ hab pkg install --binlink jarvus/multirun
» Installing jarvus/multirun
☁ Determining latest version of jarvus/multirun in the 'stable' channel
↓ Downloading jarvus/multirun/0.3.2/20190925020410
☛ Verifying jarvus/multirun/0.3.2/20190925020410
→ Using core/glibc/2.27/20190115002733
→ Using core/linux-headers/4.17.12/20190115002705
✓ Installed jarvus/multirun/0.3.2/20190925020410
★ Install of jarvus/multirun/0.3.2/20190925020410 complete with 1 new packages installed.
» Binlinking multirun from jarvus/multirun/0.3.2/20190925020410 into /hab/bin
★ Binlinked multirun from jarvus/multirun/0.3.2/20190925020410 to /hab/bin/multirun
$ multirun -h
Usage: multirun <options> command...
or like this:
$ hab pkg install jarvus/multirun
» Installing jarvus/multirun
☁ Determining latest version of jarvus/multirun in the 'stable' channel
↓ Downloading jarvus/multirun/0.3.2/20190925020410
☛ Verifying jarvus/multirun/0.3.2/20190925020410
→ Using core/glibc/2.27/20190115002733
→ Using core/linux-headers/4.17.12/20190115002705
✓ Installed jarvus/multirun/0.3.2/20190925020410
★ Install of jarvus/multirun/0.3.2/20190925020410 complete with 1 new packages installed.
$ hab pkg exec jarvus/multirun multirun -h
Usage: multirun <options> command...
or like this:
FROM habitat/default-studio-x86_64-linux:0.85.0
RUN hab pkg install --binlink \
jarvus/multirun \
core/nginx \
core/php
ENTRYPOINT ["multirun"]
If you're interested in publishing to Habitat yourself under your own origin name, just register your own origin and post what it is, and I'll open a PR for you adding the plan.sh
in your repo root or under habitat/
and configure it for building right from your repo. Then after you linked the repo, the Habitat bldr would handle rebuilding the binary on any push to master
or update to glibc getting marked stable
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.