cdalvaro / docker-salt-master Goto Github PK
View Code? Open in Web Editor NEW๐ณ๐ง Dockerized Salt Master
Home Page: https://docs.saltproject.io
License: MIT License
๐ณ๐ง Dockerized Salt Master
Home Page: https://docs.saltproject.io
License: MIT License
Hi.
Have you experienced this issue ?
Describe the bug
When using Gitfs2 for remote git repo, i keep getting errors.
When doing it via cli, it works fine, and i can see, that it's using the same ssh key files.
To Reproduce
Steps to reproduce the behavior:
Create new github repo
Create ssh key, without password
put the keys in the keys folder, and name them accordingly
restart container, and watcht the logs.
test using cli to clone the repo, using ssh, and verify it works.
Expected behavior
No errors in logs, and able to access the gitfs repo
Additional info
The full log with the outputted error.
Version report (please complete the following information):
Additional context
gitfs.conf
gitfs_provider: pygit2
gitfs_privkey: /home/salt/data/keys/gitfs_ssh
gitfs_pubkey: /home/salt/data/keys/gitfs_ssh.pub
gitfs_remotes:
- [email protected]:rhjensen79/salt.git:
- root: salt
- base: master
Originally posted by abhi1693 March 8, 2024
I need to be able to fetch basic information about the master via the API such as version it's running. This is needed by 3rd party integrations to ensure we can begin the minion upgrade process when this version changes. One idea was to simply install a minion inside the master. I tried adding a minion.conf
in the config
directory with just master: localhost
and it started throwing errors about missing directories.
I was wondering if this is a hidden feature of this repo so that I can simply enable this via environment variable and start managing our master too?
As requested by @abhi1693, having a salt-minion
running inside the container can be useful in some circumstances. So, I'm going to add support for enabling a salt-minion
instance inside the container with a basic configuration to be paired with the master
instance.
The minion will be disabled by default and could be activated using the environment variable: SALT_MINION_ENABLED
with value True
.
Additionally, the minion will be configurable using a config directory.
I'll add more details when the development begins.
As a new user I just edit compose file and bring up a container. But unfortunately forget about exposing api port in docker compose. That's why I faced with ssl error when trying to connect to api.
Should be added at:
docker-salt-master/docker-compose.yml
Line 12 in 581808b
Of course it can be added as commented row, because it is not enabled by default. But I think it should be. That's just an example.
@cdalvaro what do you think about this ?
Gitfs tests fail at their first attempt.
As per the documentation, there is a way for telling the master to sync custom types on minions' start. Please refer to: https://docs.saltproject.io/en/latest/topics/reactor/index.html#minion-start-reactor
It would be good to have a config option and directory mapping for configuring the reactor, much like other options are (mounting the keys, roots, etc).
An alternative could be enhancing the config system so a master config template (or zero or more small config files like Apache's conf.d/* files, for example) can be used. This way, future or yet unsupported options could be covered easily.
Follow the following link: https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#adding-a-description-to-multi-arch-images to properly set the description when using multi-arch images.
Describe the bug
When i try to start the container using the docker-compose file, I get permission denied errors
I also tried starting the container with this command, but same output.
docker run --name salt_stack -it --rm \
--publish 4505:4505 --publish 4506:4506 \
--env "PUID=$(id -u)" --env "PGID=$(id -g)" \
--volume $(pwd)/roots/:/home/salt/data/srv/ \
--volume $(pwd)/keys/:/home/salt/data/keys/ \
ghcr.io/cdalvaro/docker-salt-master:latest
To Reproduce
Clone repo and do docker-compose up
Expected behavior
The container should generate keys etc and start.
Additional info
[+] Building 0.0s (0/0)
[+] Running 1/0
โ Container salt_master Created 0.0s
Attaching to salt_master
salt_master | [INFO] - Configuring directories ...
salt_master | '/srv' -> '/home/salt/data/srv'
salt_master | '/var/log/salt' -> '/home/salt/data/logs/salt'
salt_master | [INFO] - Configuring logrotate ...
salt_master | [INFO] - Configuring container timezone ...
salt_master | [INFO] - Setting TimeZone -> Europe/Madrid ...
salt_master | [INFO] - Configuring salt-master service ...
salt_master | [INFO] - Configuring 3rd-party salt-formulas ...
salt_master | [INFO] - Setting up salt keys ...
salt_master | [INFO] - ==> Setting up master keys ...
salt_master | [INFO] - Generating master keys ...
salt_master | Error: [Errno 13] Permission denied: '/home/salt/data/keys/master.pem'
salt_master | [INFO] - ==> Setting up GPG keys ...
salt_master | [INFO] - Could not find GPG keys. GPG setup skipped.
salt_master | [INFO] - Setting up salt keys permissions ...
salt_master | chmod: changing permissions of '/home/salt/data/keys/master.pem': Permission denied
salt_master exited with code 1
salt_master | chown: changing ownership of '/home/salt/data/keys/master.pem': Permission denied
salt_master exited with code 1
salt_master | chown: changing ownership of '/home/salt/data/keys/master.pem': Permission denied
salt_master exited with code 1
salt_master | chown: changing ownership of '/home/salt/data/keys/master.pem': Permission denied
salt_master exited with code 1
salt_master | chown: changing ownership of '/home/salt/data/keys/master.pem': Permission denied
Version report (please complete the following information):
uname -a
]Darwin MBP 23.2.0 Darwin Kernel Version 23.2.0: Wed Nov 15 21:55:06 PST 2023; root:xnu-10002.61.3~2/RELEASE_ARM64_T6020 arm64
docker --version
]Docker version 25.0.2, build 29cf629
3006.6
]ghcr.io/cdalvaro/docker-salt-master:latest
Additional context
Add any other context about the problem here.
Describe the bug
I use 3001.1 image for up a master with api. When I done I try to connect via pepper(0.7.6) installed from pip. But faced with an error:
Error with request: <urlopen error [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1108)>
<urlopen error [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1108)>
Uncaught Pepper error (increase verbosity for the full traceback).
Also api.log has errors like this:
2020-10-02 15:23:45,895 [cherrypy.error :219 ][INFO ][181] [02/Oct/2020:15:23:45] ENGINE Listening for SIGTERM.
2020-10-02 15:23:45,896 [cherrypy.error :219 ][INFO ][181] [02/Oct/2020:15:23:45] ENGINE Listening for SIGHUP.
2020-10-02 15:23:45,896 [cherrypy.error :219 ][INFO ][181] [02/Oct/2020:15:23:45] ENGINE Listening for SIGUSR1.
2020-10-02 15:23:45,896 [cherrypy.error :219 ][INFO ][181] [02/Oct/2020:15:23:45] ENGINE Bus STARTING
2020-10-02 15:23:45,896 [cherrypy.error :219 ][INFO ][181] [02/Oct/2020:15:23:45] ENGINE Started monitor thread '_TimeoutMonitor'.
2020-10-02 15:23:46,003 [cherrypy.error :219 ][INFO ][181] [02/Oct/2020:15:23:46] ENGINE Serving on https://0.0.0.0:8000
2020-10-02 15:23:46,003 [cherrypy.error :219 ][INFO ][181] [02/Oct/2020:15:23:46] ENGINE Bus STARTED
2020-10-02 15:23:46,005 [cherrypy.error :219 ][ERROR ][181] [02/Oct/2020:15:23:46] ENGINE Error in HTTPServer.tick
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/cherrypy/wsgiserver/__init__.py", line 2021, in start
self.tick()
File "/usr/lib/python3/dist-packages/cherrypy/wsgiserver/__init__.py", line 2090, in tick
s, ssl_env = self.ssl_adapter.wrap(s)
File "/usr/lib/python3/dist-packages/cherrypy/wsgiserver/ssl_builtin.py", line 66, in wrap
s = self.context.wrap_socket(sock,do_handshake_on_connect=True,
File "/usr/lib/python3.8/ssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "/usr/lib/python3.8/ssl.py", line 1040, in _create
self.do_handshake()
File "/usr/lib/python3.8/ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
OSError: [Errno 0] Error
So I think we need to do something about it. Doesn't work out of the box or documentation needs clarification
That's is not critical for me at all.
I will try to figure out what's going on.
Version report (please complete the following information):
5.4.0-47-generic
]Docker version 19.03.13, build 4484c46d9d
]Describe the bug
In the 3006.2 release, the /home/salt directory was owned by the salt user:
root@f6964e308ee1:/home/salt# ls -la
total 32
drwxrwxr-x 1 salt salt 4096 Aug 10 20:10 .
drwxr-xr-x 1 root root 4096 Aug 10 20:09 ..
-rw-r--r-- 1 salt salt 220 Jan 6 2022 .bash_logout
-rw-r--r-- 1 salt salt 3771 Jan 6 2022 .bashrc
drwxrwxr-x 3 salt salt 4096 Aug 10 20:10 .cache
-rw-r--r-- 1 salt salt 849 Aug 10 20:09 .profile
drwxr-xr-x 7 root root 4096 Aug 10 20:10 data
root@f6964e308ee1:/home/salt# exit
However, in the 3006.3 release, this directory is now owned by root:
root@48671de9e0fb:/home/salt# ls -la
total 28
drwxr-x--- 1 root root 4096 Sep 8 11:46 .
drwxr-xr-x 1 root root 4096 Sep 8 11:46 ..
-rw-r--r-- 1 root root 220 Jan 6 2022 .bash_logout
-rw-r--r-- 1 root root 3771 Jan 6 2022 .bashrc
-rw-r--r-- 1 root root 849 Sep 8 11:46 .profile
drwxr-xr-x 7 root root 4096 Sep 8 11:46 data
This causes the system to fail to startup, since the salt user cannot cd to /home/salt:
2023-09-14 18:03:06,255 INFO supervisord started with pid 1
2023-09-14 18:03:07,257 INFO spawned: 'salt-master' with pid 99
2023-09-14 18:03:07,259 INFO spawned: 'cron' with pid 100
supervisor: couldn't chdir to /home/salt: EACCES
supervisor: child process was not spawned
2023-09-14 18:03:07,262 INFO exited: salt-master (exit status 127; not expected)
2023-09-14 18:03:08,264 INFO spawned: 'salt-master' with pid 101
2023-09-14 18:03:08,265 INFO success: cron entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
supervisor: couldn't chdir to /home/salt: EACCES
supervisor: child process was not spawned
2023-09-14 18:03:08,269 INFO exited: salt-master (exit status 127; not expected)
2023-09-14 18:03:10,273 INFO spawned: 'salt-master' with pid 102
supervisor: couldn't chdir to /home/salt: EACCES
supervisor: child process was not spawned
2023-09-14 18:03:10,279 INFO exited: salt-master (exit status 127; not expected)
^C2023-09-14 18:03:11,280 WARN received SIGINT indicating exit request
2023-09-14 18:03:11,281 INFO waiting for salt-master, cron to die
2023-09-14 18:03:11,281 INFO stopped: cron (terminated by SIGTERM)
To Reproduce
Steps to reproduce the behavior:
docker run --rm -ti cdalvaro/docker-salt-master:3006.3
Expected behavior
salt-master starts up
Additional info
The full log with the outputted error.
[INFO] - Configuring directories ...
'/srv' -> '/home/salt/data/srv'
'/var/log/salt' -> '/home/salt/data/logs/salt'
[INFO] - Configuring logrotate ...
[INFO] - Configuring container timezone ...
[INFO] - Setting TimeZone -> UTC ...
[INFO] - Configuring salt-master service ...
[INFO] - Configuring 3rd-party salt-formulas ...
[INFO] - Setting up salt keys ...
[INFO] - ==> Setting up master keys ...
[INFO] - Generating master keys ...
Error: Write access denied to "/home/salt/data/keys" for user "salt".
[INFO] - ==> Setting up GPG keys ...
[INFO] - Could not find GPG keys. GPG setup skipped.
[INFO] - Setting up salt keys permissions ...
Salt Version:
Salt: 3006.3
Python Version:
Python: 3.10.13 (main, Sep 6 2023, 02:11:27) [GCC 11.2.0]
Dependency Versions:
cffi: 1.14.6
cherrypy: 18.6.1
dateutil: 2.8.1
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 3.1.2
libgit2: 1.6.4
looseversion: 1.0.2
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.2
msgpack-pure: Not Installed
mysql-python: Not Installed
packaging: 22.0
pycparser: 2.21
pycrypto: Not Installed
pycryptodome: 3.9.8
pygit2: 1.12.2
python-gnupg: 0.4.8
PyYAML: 6.0.1
PyZMQ: 23.2.0
relenv: 0.13.10
smmap: Not Installed
timelib: 0.2.4
Tornado: 4.5.3
ZMQ: 4.3.4
System Versions:
dist: ubuntu 22.04.3 jammy
locale: utf-8
machine: x86_64
release: 6.5.2-arch1-1
system: Linux
version: Ubuntu 22.04.3 jammy
[INFO] - Starting supervisord ...
2023-09-14 18:04:53,145 INFO Included extra file "/etc/supervisor/conf.d/cron.conf" during parsing
2023-09-14 18:04:53,145 INFO Included extra file "/etc/supervisor/conf.d/salt-master.conf" during parsing
2023-09-14 18:04:53,145 INFO Set uid to user 0 succeeded
2023-09-14 18:04:53,148 INFO RPC interface 'supervisor' initialized
2023-09-14 18:04:53,148 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2023-09-14 18:04:53,148 INFO supervisord started with pid 1
2023-09-14 18:04:54,150 INFO spawned: 'salt-master' with pid 99
2023-09-14 18:04:54,151 INFO spawned: 'cron' with pid 100
supervisor: couldn't chdir to /home/salt: EACCES
supervisor: child process was not spawned
2023-09-14 18:04:54,154 INFO exited: salt-master (exit status 127; not expected)
2023-09-14 18:04:55,157 INFO spawned: 'salt-master' with pid 101
2023-09-14 18:04:55,158 INFO success: cron entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
supervisor: couldn't chdir to /home/salt: EACCES
supervisor: child process was not spawned
2023-09-14 18:04:55,163 INFO exited: salt-master (exit status 127; not expected)
2023-09-14 18:04:57,167 INFO spawned: 'salt-master' with pid 102
supervisor: couldn't chdir to /home/salt: EACCES
supervisor: child process was not spawned
2023-09-14 18:04:57,173 INFO exited: salt-master (exit status 127; not expected)
2023-09-14 18:05:00,178 INFO spawned: 'salt-master' with pid 103
supervisor: couldn't chdir to /home/salt: EACCES
supervisor: child process was not spawned
2023-09-14 18:05:00,183 INFO exited: salt-master (exit status 127; not expected)
2023-09-14 18:05:01,184 INFO gave up: salt-master entered FATAL state, too many start retries too quickly
Version report (please complete the following information):
Additional context
Add any other context about the problem here.
Originally posted by abhi1693 March 7, 2024
What is the best way to install additional dependencies into the container so that it persists across reboots?
As requested by @abhi1693, it's a good enhancement for this project to add support for additional dependencies.
The main idea is to be able to install python packages when the container starts before running salt-master
so modules that require extra packages can fulfill their requirements.
Two approaches can be taken:
Both approaches can be implemented. Since one method can be more suitable than the other depending each configuration.
I think we can use the environment variables PYTHON_PACKAGES_FILE
and PYTHON_PACKAGES
to specify the requirements.txt file and a list of packages, respectively.
Allowing both systems at the same time can be confusing, so the first one will take priority over the second one. So, when the env variable PYTHON_PACKAGES_FILE
is defined, PYTHON_PACKAGES
will be ignored.
PYTHON_PACKAGES
will be directly forwarded to the pip install
command. So, as long it respects pip install
format, all pip functionality will be supported.
As suggested by @jothibasu-kamaraj, salt-api
support is going to be added.
I had to downgrade docker-compose file to 3.3. Like the file below. I'm getting connection refused for the ports and /var/log/salt/master is logging the messages below, tell me if this looks like a bug, I can report it as so (using the template). I've also noticed that, there's no configuration in /home/salt/data/config/. I've added a master.conf with default ports but no luck. Thanks for any help.
version: '3.3'
volumes:
roots:
keys:
logs:
services:
master:
container_name: salt_master_engage1
image: ghcr.io/cdalvaro/docker-salt-master:latest
restart: always
volumes:
- "roots/:/home/salt/data/srv"
- "keys/:/home/salt/data/keys"
- "logs/:/home/salt/data/logs"
ports:
- "4505:4505"
- "4506:4506"
### salt-api port
# - "8000:8000"
healthcheck:
test: ["CMD", "/usr/local/sbin/healthcheck"]
#start_period: 30s
environment:
DEBUG: 'false'
TZ: America/Chicago
PUID: 1000
PGID: 1000
SALT_LOG_LEVEL: info
### salt-api settings
# SALT_API_SERVICE_ENABLED: 'True'
# SALT_API_USER: salt_api
# SALT_API_USER_PASS: 4wesome-Pass0rd
2022-10-31 10:12:57,968 [salt.modules.network:2143][ERROR ][12952] Exception while creating a ThreadPoolExecutor for resolving FQDNs: can't start new thread
2022-10-31 10:12:57,986 [salt.modules.network:2143][ERROR ][12951] Exception while creating a ThreadPoolExecutor for resolving FQDNs: can't start new thread
2022-10-31 10:12:57,995 [salt.modules.network:2143][ERROR ][12950] Exception while creating a ThreadPoolExecutor for resolving FQDNs: can't start new thread
2022-10-31 10:12:57,996 [salt.modules.network:2143][ERROR ][12949] Exception while creating a ThreadPoolExecutor for resolving FQDNs: can't start new thread
2022-10-31 10:12:58,007 [salt.modules.network:2143][ERROR ][12948] Exception while creating a ThreadPoolExecutor for resolving FQDNs: can't start new thread
2022-10-31 10:12:58,066 [salt.utils.process:998 ][ERROR ][13192] An un-handled exception from the multiprocessing process 'FileserverUpdate' was caught:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/salt/utils/process.py", line 993, in wrapped_run_func
return run_func()
File "/usr/local/lib/python3.10/dist-packages/salt/master.py", line 508, in run
self.update_threads[interval].start()
File "/usr/lib/python3.10/threading.py", line 935, in start
_start_new_thread(self._bootstrap, ())
RuntimeError: can't start new thread
Instead of using /etc/salt-docker/keys
as a source for master and minions keys, this directory will be used as the default pki/master
directory.
/etc/salt-docker/keys
directory will be able to be mounted as a Docker volume and will contain the master and minion keys.
In addition support for signed keys will be added.
In order to maintain logs tidy and easy to read, it is a good idea to add support for log rotation.
python${PYTHON_VERSION} python3-dev libpython3-dev
You're only specifying the minor version for python
, and not any of the related packages. They will instead use the default for the distribution, ignoring PYTHON_VERSION
.
You either need to specify the same version for all of them (and use e.g. the deadsnakes repo), or not bother with PYTHON_VERSION
and use the distro default for everything.
Install specified saltstack-formulas repos and make them available directly from the docker image.
Another interesting source of formulas could be the unofficial salt-formulas account.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.