bytemarkhosting/symbiosis

A hosting environment that works with you, not against you.

symbiosis's Issues

Offer alternative Symbiosis managed webservers

symbiosis-nginx would be loved by many.

Originally reported on Bytemark's Gitlab by @patch0 on 2016-02-18T10:33:21.000Z

Easier method of specifying email aliases

Hi,

While the current mechanism for specifying email aliases is very flexible, I managed to mess things up while creating some aliases by mistyping the local address in the aliases file.

Other hosting platforms allow you to define aliases for a mailbox, I was wondering if this might be a good addition to symbiosis.

I'd suggest adding an 'aliases' file in the Mailbox directory (where the password file is currently) that just contains a list of aliases for that mailbox.

Andy

Originally reported on Bytemark's Gitlab by @patch0 on 2013-10-03T10:24:04.000Z

Package up byteback into symbiosis-backups-byteback-{client,server}

Originally reported on Bytemark's Gitlab by @patch0 on 2015-09-09T09:59:22.000Z

Request: Write mysql root credentials to /root/.my.cnf when imaging.

(Note: This may be something for imager, or Stretch, but applied to Symbiosis images only)

It's never clear that the root, admin and mysql root@localhost users all have the same password in a newly imaged machine, which leads to users likely changing the root/admin passwords like they should, and not making note of the root@localhost password we set for mysql.

Simply writing the below to /root/.my.cnf (with relevant permissions) would make password recovery simpler, and allow the user to log in directly.

[client]
user=root
password="<example>"

There's a small outside risk to this, by keeping it in /root would negate most of this, and make things simpler for users.

Originally reported on Bytemark's Gitlab by @pcammish on 2017-04-13T13:49:00.308Z

Dovecot security update breaks symbiosis's dovecot integration

Debian bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=860049
Dovecot commit: dovecot/core@000030f

Reported by a friend who I convinced to use symbiosis-jessie a while ago. He reports that, after upgrading his machine to the latest version of dovecot from debian security, his authentication to dovecot stopped working. SMTP is still accepting emails, etc.

He reports the following in his logs:

symbiosis-email-dict-proxy: Caught too few arguments
dovecot: auth-worker(11354): Error: dict(user@domain,ip): Failed to lookup key shared/passdb/%u

It's too easy to break Exim by changing ssl certificate ownership.

Pretty much everything in /srv/ is owned by admin:admin, so it's tempting to run something like "chown -R admin:admin /srv". The problem is that Exim certificates lie in /srv//config/ssl/sets and Debian-exim (the user that runs Exim) is not a member of the admin group, so this is an awkward fact to learn and remember.

It might be better if the certificates were managed in /etc/ssl - from where they are currently, and tortuously symlinked.

Alternatively, if issue 38 https://gitlab.bytemark.co.uk/open-source/symbiosis/issues/38 is implemented, then I've made a suggestion for managing these certs.

Originally reported on Bytemark's Gitlab by @ieiloart on 2016-09-23T14:08:10.769Z

ClamAV can be a resource hog, and doesn't need to be running if it's not configured.

We've had a few Symbiosis users recently (with otherwise pretty quiet machines who are seeing issues with clamav hogging resources - CPU, disk and RAM.

This seems to be down to a memory leak in clamd, which after 150+ days of uptime chews up a significant amount of RAM, which can then cause freshclam to have problems allocating memory, leading to it cheaing up CPU time and disk space as it writes WARNING: [LibClamAV] mpool_malloc(): Can't allocate memory ([0-9]* bytes). to freshclam.log over and over until the disk is full, then starts consuming all the CPU time on the box.

Really, clamd doesn't need to even be running if its not configured to be used, and provides users a false sense of security if they see it running on the box but its not configured, however it may be worth an extra daily/weekly forced restart/reload of the service if it is being used, to clear out any memory leak issues.

Originally reported on Bytemark's Gitlab by @pcammish on 2016-12-14T12:17:25.897Z

Ability to add an email attachment file extension black/whitelist

Would be nice to be able to block certain attachments to email, in addition to the usual antivirus scanning

Originally reported on Bytemark's Gitlab by @patch0 on 2016-08-10T10:31:40.021Z

Package up grafana/influxdb as symbiosis-stats-server

Originally reported on Bytemark's Gitlab by @patch0 on 2015-09-09T09:57:41.000Z

Package up collectd as symbiosis-stats-client

Originally reported on Bytemark's Gitlab by @patch0 on 2015-09-09T09:58:10.000Z

symbiosis-ssl can generate SSL config for sites that have no certificate

symbiosis-ssl can generate SSL config for sites that have no certificate returned by Lets Encrypt. This can lead to invalid configuration, and Apache being unable to re-start.

This has been observed both in terms of missing certs that were never returned successfully from Lets Encrypt, or where symbiosis-ssl didn't have permission to write the certificate, but still wrote the SSL config.

Originally reported on Bytemark's Gitlab by @dedwards on 2016-08-10T11:32:30.849Z

Simple database creator

A typical usage scenario for Symbiosis is a user creating a new site, then uploading a PHP webapp and expecting it to work.

The problem is, once they've uploaded the PHP content, they need to create a MySQL database, which can only currently be done over the command-line or PHPMyadmin.

An interesting feature would allow a user to create a "database" file in the /srv/domain.tld/config directory, containing a user and password. Symbiosis would detect the presence of this file, and if a database named after the domain didn't already exist, would create a new database, and GRANT the right permissions to it allowing local access with the username/password in the file. The user could then install say, Wordpress with only SFTP.

Originally reported on Bytemark's Gitlab by @jiphex on 2016-08-10T11:39:18.395Z

Drop-in Varnish

It would be amazing if symbiosis allows users to 'drop in' varnish caching.

Originally reported on Bytemark's Gitlab by @patch0 on 2014-11-13T12:24:26.000Z

SNI config for dovecot and exim

We should allow dovecot and exim4 to use SNI to allow proper certificate verification.

Originally reported on Bytemark's Gitlab by @patch0 on 2015-10-19T16:46:40.000Z

monit: Use systemd timer to launch instead of cron

This requires a .timer as well as a .service file. It also means that sysvinit isn't really supported, unless a check is put in place to remove the cron job or otherwise disable it.

Originally reported on Bytemark's Gitlab by @patch0 on 2017-04-10T12:12:44.059Z

Using `/srv/<domains/mailboxes/<user>/forward` with an external destinations fails SPF and DKIM checks

As /srv/<domains/mailboxes/<user>/forward simply relays the mail, it often causes SPF and DKIM signed mail to fail. This is particularly bad as the more important the mail (banks etc) the more tight their SPK/DKIM will be, and the destination mail server may simply drop invalid mail, or in many cases (see SNDS) simply flag all mail coming in that route as spam.

This can be worked around by using a basic Sieve filter (created in a mail client or Roundcube, etc) instead of a basic forward, which generates a header in the final mailbox like this:

Delivered-To: [email protected]
Received: by 10.157.19.41 with SMTP id f38csp1124363ote;
        Fri, 24 Mar 2017 02:25:49 -0700 (PDT)
X-Received: by 10.28.157.150 with SMTP id g144mr1955711wme.89.1490347549347;
        Fri, 24 Mar 2017 02:25:49 -0700 (PDT)
Return-Path: <[email protected]>
Received: from carth.ebonhawk.example.uk0.bigv.io (under100words.com. [2001:41c8:51:7a3::163])
        by mx.google.com with ESMTPS id x4si1960991wmx.113.2017.03.24.02.25.48
        for <[email protected]>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Fri, 24 Mar 2017 02:25:49 -0700 (PDT)
Received-SPF: neutral (google.com: 2001:41c8:51:7a3::163 is neither permitted nor denied by best guess record for domain of [email protected]) client-ip=2001:41c8:51:7a3::163;
Authentication-Results: mx.google.com;
       dkim=pass [email protected];
       spf=neutral (google.com: 2001:41c8:51:7a3::163 is neither permitted nor denied by best guess record for domain of [email protected]) [email protected];
       dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=bytemark.co.uk
Received: from admin by carth.ebonhawk.example.uk0.bigv.io with local (Exim 4.84_2) (envelope-from <[email protected]>) id 1crLTg-0006yS-L8 for [email protected]; Fri, 24 Mar 2017 09:25:48 +0000
X-Sieve: Pigeonhole Sieve 0.4.2
X-Sieve-Redirected-From: [email protected]
Received: from yrk.mx.bytemark.co.uk ([2001:41c9:0:1019::81:84]) by carth.ebonhawk.example.uk0.bigv.io with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <[email protected]>) id 1crLTf-0006yJ-0J for [email protected]; Fri, 24 Mar 2017 09:25:48 +0000
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bytemark.co.uk; s=20131115; h=Content-Type:MIME-Version:Date:Message-ID:Subject:From:To; bh=0UTMmBvQCD1aDtGQ0LM/wKwl+36AyMeb9pEvAf/Lve8=; b=jCmTawHtj3wZvPA+DnmtoyYpJYmznfAbAR2ZZkF0WmkbB9MD0BdivSofH1ww4DoYM3tH3BtjFt02Bn0NpKpNeNX74c8l5lVQ6QB2uVc2dDBVhma/MLbceDvrjAsOXGnomgWIFPZNLzOAcXF/mS+5ctw7nRGg44nxxlXMKAymbzTvzOGiiyA0rX2DgKhrn3SDhBNNC2dW8AsiZ6lqs9jZz99bqJlMwzhGz709MfPfzdvnIKsnt9sBc/hL+KAu0lUH1l0ySSNeLqyU6CTU91B3ULnVkQoX9EMh6Rim3430thOQ0VAfon9kp3l7lVbaXfDOkcK2PLxRBYSqnmD5uVR3dQ==;
Received: from ratatoskr.bytemark.co.uk ([2001:41c9:0:1017::48] helo=ratatoskr.dh.bytemark.co.uk) by yrk.mx.bytemark.co.uk with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <[email protected]>) id 1crLTe-0006jU-Kd for [email protected]; Fri, 24 Mar 2017 09:25:46 +0000
Received: from [2001:41c8:3:1::186] by ratatoskr.dh.bytemark.co.uk with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <[email protected]>) id 1crLTe-0005hF-I3 for [email protected]; Fri, 24 Mar 2017 09:25:46 +0000
To: [email protected]
From: Paul <[email protected]>
Subject: Test forward
Organization: Bytemark Hosting
Message-ID: <[email protected]>
Date: Fri, 24 Mar 2017 09:25:44 +0000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="1kPfKMM3kgqtkMjBuqBIR3uSp4MpjjNV9"
Sender: Symbiosis Administrator <[email protected]>

This works with a simple sieve rule like:

# rule:[Redirect all mail.]
if true
{
  redirect "[email protected]";
  stop;
}

It would be great if the forward file generated the relevant sieve rule, and massively improve deliverability.

Originally reported on Bytemark's Gitlab by @pcammish on 2017-03-24T09:53:18.430Z

Offer alternative symbiosis managed databases

symbiosis-postgres, symbiosis-sqlite3, etc.

For postgres, perhaps a '/srv/SITE/config/postgres-password' file could automatically manage a role per site?

Originally reported on Bytemark's Gitlab by @patch0 on 2016-02-18T10:32:38.000Z

Event bus for configuration changes

A common problem in developing Symbiosis functionality is the difficulty in detecting configuration changes. Currently Symbiosis is configured using SFTP, but has no hooks into that system, so it's always going to be somewhat hobbled.

It would be good to have some sort of event bus to detect configuration changes, and act upon them at the time. This removes the need for polling which is wasteful and not very timely in some cases. Using a more event based approach would make configuration changes more or less instant, and maybe more reliable too.

One way of achieving this would be to have hooks into the SFTP subsystem. OpenSSH doesn't seem to have them, others do, ie https://www.npmjs.com/package/ssh

Whilst not advocating the addition of node into the stack, it would be good to have this functionality somehow, so it's open for debate.

Originally reported on Bytemark's Gitlab by @dedwards on 2016-08-10T11:39:11.969Z

ftp users for multiple sites

Currently symbiosis has the ability to create ftp users for individual sites. It would be nice if it also had the ability to create ftp users that have access to multiple sites, e.g. different subdomains on the same site.

You can work around it currently by creating the ftp user on each site, but that's very inefficient, especially for lots of sites.

Originally reported on Bytemark's Gitlab by @patch0 on 2015-12-03T16:32:27.000Z

symbiosis-firewall reflecting changes made directly to iptables

Another wishlist item:

Symbiosis-firewall should be able to read the current state of iptables and ip6tables and update the representation of the firewall in /etc/symbiosis/firewall.d/ accordingly, as well as being able to update the firewall based on the contents of that same folder.

This would, I think, work best if the part of the system that provides blacklisting functionality was split out from the part that represents the state of the firewall on the system. It would also let us play nicely with fail2ban, something that some users would thank us for (and which would let us rely on a well established tool that has a surrounding community and ecosystem).

Originally reported on Bytemark's Gitlab by @patch0 on 2014-11-14T15:04:49.000Z

Greylisting

Add a per-domain greylisting flag.

symbiosis-httpd-logger.go does not set ownership and permissions on default log file

When logging to the default log file, symbiosis-httpd-logger.go does not set ownership and permissions on the log file. Which therefore appears owned by root and with permissions 0600.

With websites where symbiosis-httpd-configure has generated specific configuration files, these log via default log files. So in this case, the log files, being owned by root, can't be manipulated by symbiosis-httpd-rotate-logs once it has dropped privs.

Stats default to On

I strongly believe Stats should be disabled OR the stats http page be password protected by default.

Originally reported on Bytemark's Gitlab by @mpoland on 2017-03-07T15:00:28.117Z

Testing in gitlab

At the moment the install / dist-upgrade / upgrade tests get weirdly-far in gitlab-ci then fails. Here's a quick summary of how the tests used to work on maker2 (as I understand it), and then later I'll go into detail on why the tests fail in gitlab-ci

The Current Situation

1. autotest sets up a VM using schroot magic i don't fully understand
2. the VM boots up with systemd and all that jazz, uses DHCP & SLAAC to configure its networking, and automatically runs all the scripts in the autotest folder, keeping all the output as a log file. Once done, it shuts down
3. autotest cracks open the VM's filesystem and reads the logfile. Somehow it detects failures and fails if there was a failure then it exits with a nonzero error code so that maker2 knows

Some Feelings About The Current Situation

Patrick said something about autotest using the console to talk to the tests, and something else much scarier about the VM sshing into the host to run something.

This doesn't work on gitlab-ci, and is also kinda hacky, for a few reasons.

• the scripts in the autotest folder aren't particularly focussed. In addition to actually running tests, they do these and probably others:
  • add an admin user
  • install all the packages needed by symbiosis from a big list of packages
  • install symbiosis
• opening up the filesystem of the VM so you can prod it is pretty gross

On the plus side it works, and it would only take a bit of effort to port the whole schroot setup over to gitlab-ci (but would have to run using a shell runner)

Why the tests fail in gitlab-ci

When gitlab-ci runs a container it starts bash in the context of the container. Effectively, bash is PID 1 for the container. There's no init-system to talk to to get stuff going. I believe the apt-get install step for some packages starts them using /etc/init.d (probably something about the package detecting a lack of systemd and putting a proper sysvinit script in) which would explain why a lot of the tests actually succeed. BUT SOME OF THEM FAIL, and we should really be doing a much more realistic test than running our symbiosis full-system tests in a docker container that isn't a full symbiosis system.

With that in mind:

A More Realistic Test Proposal

We're still going to want to run symbiosis in a VM, I think. To do a realistic full-install / dist-upgrade test we need to have a realistic system, which the docker container environment isn't. We need a systemd to talk to so we can schedule restarts, that sort of thing.

We will need some test-specific configurations (particularly repo URLs) too. And we'll need to be able to orchestrate the testing and fail the build when the tests fail.

We could create an image prior to the testing which would have a user account with passwordless sudo and a .ssh/authorized_keys . The private key would be kept in the secret variables section of the project on gitlab, and so would be presented to the gitlab-ci script as an env var.

In the gitlab-ci script we'd start the VM with qemu, as we do for bytemark/bytemark-packer-templates, then use ansible to copy over the tests, install the symbiosis packages, and run the tests. We could write our ansible playbook so that it captures the logs and copies them back to the runner and have the gitlab-ci script spit the logs out, then exit with ansible's exit code.

This would make our test output more readable and shorter, not be quite as weird the current autotest setup on maker2, probably not require also running a DHCP server.

The work we'd need to do:

• add an ansible layer to docker-images/layers
• rewrite the autotest/ scripts as ansible playbooks
• make a base VM image with the necessary networking & ssh setup

Thoughts @pcherry , @jcarter ?

Originally reported on Bytemark's Gitlab by @telyn on 2017-03-09T16:21:43.733Z

symbiosis-firewall: log why ip was blackisted

Currently into syslog, symbiosis-firewall enters:

Jan 22 07:50:01 symwheezy symbiosis-firewall-blacklist: adding xxx.xxx.xxx.xxx to blacklist for all ports

It would be nice to know why or which filter that IP address was blacklisted, as it stop the need to search through other logs to find out.

Originally reported on Bytemark's Gitlab by @patch0 on 2015-01-22T16:43:22.000Z

testing sites don't get DocumentRoot set

testing sites - e.g. example.com.testing.server.group.user.uk0.bigv.io don't have DocumentRoot set. Not being set by mod_rewrite

Originally reported on Bytemark's Gitlab by @patch0 on 2016-08-10T10:31:44.308Z

symbiosis-httpd-configure --diff-only option

Sometimes the configuration has been manually edited but it'd be nice to go back to the factory one, however the only way to see what would change other than checking manually, is to move the old hand-edited configuration out of the way, and then to run the symbiosis-httpd-configure (which reloads the site) and then compare the changes afterwards.

Would be nice if you could ask symbiosis-httpd-configure just to give you a diff of what would change if you asked it to take over the config for a particular site.

Originally reported on Bytemark's Gitlab by @jiphex on 2016-12-01T10:08:34.718Z

We need to test for failures when running under apparmor.

A user reported issues installing @symbiosis-mysql@, on an Ubuntu Trusty machine, which ultimately turned out to be caused by mysqld failing to start.

The specific problem was an apparmor policy which prevented mysql from reading our configuration-file:

 apparmor="DENIED"
 operation="open" 
 profile="/usr/sbin/mysqld"
 name="/etc/mysql/my.cnf.dpkg-symbiosis" 

I fixed this in ticket #606187 by updating the apparmor profile, but suspect there may be other lingering surprises for the future.

Originally reported on Bytemark's Gitlab by @skx on 2015-03-19T10:33:03.000Z

php-fpm

In brief I'm suggesting:

An option to enable php-fpm by touching /srv/domain.com/config/fpm

I'd like this to create the necessary apache2 config (seems generic enough) and perhaps merge config lines from /srv/domain.com/config/fpm into a custom php.ini for each pool (one per domain?).

I suspect this can be done without getting in the way of anyone doing a custom fpm setup for a site symbiosis isn't managing (for whatever reason they might have to do that).

Originally reported on Bytemark's Gitlab by @patch0 on 2014-11-13T13:33:06.000Z

Hook support for Symbiosis-SSL

It's common that the Lets Encrypt SSL certificates procured by Symbiosis for HTTPS might be used by other services, however there's currently no way for other applications configured by the user (e.g HAProxy, mail daemons, secondary servers etc) to know when a certificate has been updated other than by polling.

It would be nice if Symbiosis were to be able to run-parts on a specific directory whenever a domain's cert is rolled over so that other daemons could be notified. The environment should include information about the new certificates so that scripts could do whatever processing was necessary to use the new certificate.

New domains could come with a better mail config by default

When I'm setting up a new domain, it comes with a default mail setup which currently excludes a number of important-for-deliverability features, such as DKIM and SPF.

People who have read the documentation will notice this and add the flags, but people who don't, won't.

It'd be nice if the set-up-a-new-domain cronjob could touch various config items with default values ("default" in the dkim file, generate a dkim.key, touch spf, etc) if the config/ directory doesn't exist yet.

On a similar note, if symbiosis-xmpp is installed, the "xmpp" file could be added by default at the same time.

Obviously, users would have to have their "rm" of said file respected if they decided to change the default.

Originally reported on Bytemark's Gitlab by @patch0 on 2015-10-19T14:35:17.000Z

symbiosis-firewall pattern syntax/result checker

To avoid repeats of untested patterns getting pushed live, e.g.

https://projects.bytemark.co.uk/projects/symbiosis/repository/diff/ftpd/patterns.d?rev=ce82ebc6e83d&rev_to=0c1d4b9c91ae

Originally reported on Bytemark's Gitlab by @patch0 on 2014-10-20T10:30:41.000Z

Plaintext FTP should be disabled by default

/etc/pure-ftpd/conf/TLS currently appears to be set to 1 which means "Accept both normal sessions and SSL/TLS ones." - my opinion would be that for the next release, we should change this to 2, or even 3. Options are below.

              -Y tls behavior
              -Y 0 (default) disables SSL/TLS security mechanisms.
              -Y 1 Accept both normal sessions and SSL/TLS ones.
              -Y  2  refuses  connections  that  aren't using SSL/TLS security
              mechanisms, including anonymous ones.
              -Y 3 refuses connections  that  aren't  using  SSL/TLS  security
              mechanisms, and refuse cleartext data channels as well.
              The  server  must  have been compiled with SSL/TLS support and a
              valid certificate must be in place to accept encrypted sessions.

Originally reported on Bytemark's Gitlab by @jiphex on 2016-12-22T10:45:27.708Z

symbiosis-dns-generate uploads wrong data

If we have DNS files like so:

/srv/a.com/config/dns/a.com.txt (with some random DNS entries in it)

/srv/x.com/config/dns/x.com.txt
/srv/x.com/config/dns/a.com.txt (with different a.com.txt DNS entries in it)

symbiosis-dns-generate will copy the a.com.txt file from a.com, and then overwrite it with the entry from x.com . At a guess, this happens in alphabetical order and would be fine if things were reversed.

Could be a WONTFIX that relies on users being sensible, or we could eg, give precedence to a.com.txt from /srv/a.com over the version in /srv/x.com .

Originally reported on Bytemark's Gitlab by @patch0 on 2014-06-23T16:40:07.000Z

SSL symlinks broken on hostname change

When the hostname of a system running Symbiosis is changed the symbolic links for the self signed certificates in /etc/ssl are broken.

The symbolic links in /etc/ssl continue to point at /srv/original-server-name/....

This then prevents the Apache service from starting as the SSL files are missing/invalid.

Originally reported on Bytemark's Gitlab by @patch0 on 2016-08-10T10:31:43.046Z

File based chown utility

As a way of letting people who need writable upload directories for wordpress and the like without having to use ssh, we should have something along the following lines:

A file like:
/srv/domain/config/chown

Containing:

/relative/path/from/domain/directory - maybe from /public
user to own directory
group to own directory

eg:

/my/uploads/dir www-data www-data

A cronjob or similar can then run chown on each specified path for each domain that has things specified. It should be limited to relative paths within the domain it applies to.

NB: for the sake of keeping memory fresh, this is to attempt to further the idea of being able to do everything by editing files in FTP in an easily documented way.

Originally reported on Bytemark's Gitlab by @patch0 on 2013-06-21T10:20:45.000Z

symbiosis-httpd-configure needs forcing for SSL

I recently completely reinstalled my website with WordPress from a clean slate on Symbiosis, I then added all the necessary requirements to enable SSL with Let's Encrypt again, once I added the htaccess file and updated WordPress to use HTTPS, I was getting certificate errors as it was using the default self signed certificate, I then ran symbiosis-httpd-configure --force which fixed the issue.

Webmail: offer Rainloop as an installable alternative

https://xmob.me/article/rainloop_symbiosis

Originally reported on Bytemark's Gitlab by @patch0 on 2015-07-09T13:35:32.000Z

Authoritative DNS server support for non-Bytemark Symbiosis

Currently, symbiosis generates tinydns-data records and, when within bytemark, uploads them to upload.ns.bytemark.co.uk for DNS to be served. This works fine for us, but is less useful for domains that aren't Bytemark DNS, or machines running symbiosis outside of Bytemark's network.

It should be possible to set a flag that signals "run an authoritative nameserver with my DNS data". This could just be tinydns, or alternatively, we could use powerdns with the tinydns-data backend.

The latter would allow the VM to distribute its records to other nameservers (as a master, via AXFR) for redundancy; and we could also enable DNSSEC functionality this way, by way of pdnssec integration.

Originally reported on Bytemark's Gitlab by @patch0 on 2013-09-12T10:49:47.000Z

symbiosis-httpd-logger.go applies non-permission mode bits to log files

If logging to a directory that has the setgid bit turned on, the logger applies the directory mode to all new files, modulo turning off any execute permission. This means that log files get created with the setgid bit set, though of course not being executable this has no effect.

Except it does have an effect.

gzip: /srv//public/logs/ssl_error.log.2 is set-group-ID on execution - ignored
Failed to compress /srv/testconference.accu.org/public/logs/ssl_error.log.2

Package d-push for stretch?

d-push was removed from debian before stretch because it wasn't php7 compatible in time. It is now compatible with PHP7, so we could choose to package it ourselves.

We should determine if anyone actually uses d-push to sync emails to their mobile device and how important it is to them - i.e. whether or not they can just use IMAP instead.
I'd be very surprised if anyone was unable to use IMAP. We're talking early 2000s MS stuff here I'd expect.

Originally reported on Bytemark's Gitlab by @telyn on 2017-03-07T16:07:57.897Z

Permissions to not encourage admin-usage.

Although we do not expect users to edit the various templates which symbiosis ships with this is an option.

The general approach with Symbiosis is that the admin-user is the user that should be used. Unfortunately the file-permissions don't make that easy:

 admin@symb:~$ ls -l /etc/symbiosis/
total 28
drwxr-xr-x  2 root  root  4096 Dec 15 12:23 apache.d
drwxr-xr-x  5 root  root  4096 Nov 21 12:18 backup.d
drwxr-xr-x  2 root  root  4096 Nov 21 12:18 dns.d
drwxr-xr-x  8 admin admin 4096 Nov 21 12:14 firewall
drwxr-xr-x  2 root  root  4096 Dec 15 12:23 monit.d
drwxr-xr-x 11 root  root  4096 Dec 15 12:23 test.d
drwxr-xr-x  2 root  root  4096 Dec 15 12:23 xmpp.d
admin@symb:~$ 

On a pristine installation of symbiosis, from our imager, only the firewall directory is correctly writeable. This means that any user who wants to change the Apache template has to become root, for example.

Originally reported on Bytemark's Gitlab by @skx on 2015-02-16T17:20:30.000Z

Piwik analytics per site

Using piwik instead of webalizer would be much nicer!

Originally reported on Bytemark's Gitlab by @patch0 on 2014-10-10T11:10:35.000Z

Package up rsyslog into symbiosis-syslog-{client,server}

Originally reported on Bytemark's Gitlab by @patch0 on 2015-09-09T10:00:08.000Z

Using incrond instead of crontab for some (more) jobs

Some jobs (create-sites etc) run on a periodic basis, and I wonder if they shouldn't just be triggered by filesystem events - eg, creation/modification/deletion of monitored files.

I could see an argument against this - in the form of minimising the number of runs of say, symbiosis-httpd-configure when creating a new domain - but over time that would give quicker results for the user, prevent them from having to wonder when the next run is, and make the system feel more interactive.

Originally reported on Bytemark's Gitlab by @patch0 on 2014-11-13T13:37:45.000Z

exim4 blacklist by hostname

The second deny in /etc/exim4/symbiosis.d/10-acl/50-acl-check-rcpt/65-deny-blacklisted-hosts looks like it should be using +blacklisted_hosts_by_hostname (and the first might benefit from !+private_addresses).

  # Similarly, we will deny the message if the host IP is on a local blacklist
  deny    hosts         = +blacklisted_hosts_by_ip
          message       = The IP or hostname used when connecting is locally blacklisted.
          log_message   = IP locally blacklisted
  
  # Similarly, we will deny the message if the hostname is on a local blacklist (and not in an private range)
  deny    hosts         = !+private_addresses : +blacklisted_hosts_by_ip
          message       = The IP or hostname used when connecting is locally blacklisted.
          log_message   = Hostname locally blacklisted

  # OK, now we check whether the sender address is on a local blacklist
  deny    senders       = +blacklisted_senders
          message       = Your email address is locally blacklisted.
          log_message   = Sender locally blacklisted

Disclaimer: I'm in the early stages of finding my way around exim. ;)

File based chown utility

Symbiosis builds for arm

Hi folks,

It'd be super nice to be able to install Symbiosis on arm boxes, but at the moment symbiosis-common relies on ruby-linux-netlink which isn't available for arm. I've only looked briefly, but it could even be the case that it should be set to 'all' rather than 'amd64'.

Thanks.

Originally reported on Bytemark's Gitlab by @patch0 on 2015-12-19T17:48:54.000Z

Develop a Symbiosis: community edition

…that strips out Bytemark specific stuff.

Discuss :)

Originally reported on Bytemark's Gitlab by @patch0 on 2014-11-27T16:32:26.000Z

-s parameter to Apache logger does not sync output sometimes

We have a site conference.accu.org using a slightly modified Apache config. Logging using the default generated settings, and has '-s' enabled. But output lines are not appearing in the log file until some time (typically half an hour) after they are logged.

    ErrorLog   "|| /usr/sbin/symbiosis-httpd-logger -s -u 1002 -g 1001 /srv/conference.accu.org/public/logs/ssl_error.log"