sovereign/sovereign

A set of Ansible playbooks to build and maintain your own private cloud: email, calendar, contacts, file sync, IRC bouncer, VPN, and more.

sovereign's Issues

Two-factor Auth

Automated setup of a nice (free) two-factor auth platform like Duo Security would be cool.

https://www.duosecurity.com/vpn
https://www.duosecurity.com/unix

Use Docker to contain each application/service

Rather than installing each service directly on the machine, we should dockerize some or all of them. This would allow the services to better coexist with other customizations people make on their server.

This may require switching to Ubuntu as the supported distro.

Private key needed at the top of file referenced by SSLCertFile in znc configuration

In the SSL task under common roles, the following lines generate a combined certificate that is referenced in the template znc configuration file:

- name: Create a combined version of the public cert with intermediate and root CAs
  shell: cat /etc/ssl/certs/wildcard_public_cert.crt /etc/ssl/certs/wildcard_ca.pem >
    /etc/ssl/certs/wildcard_combined.pem creates=/etc/ssl/certs/wildcard_combined.pem

The znc documentation for setting up SSL suggests that SSLCertFile needs to reference a file that leads with a private key, followed by your certificate and the certificate bundle.

I had to execute the following to get znc to play nice with my Comodo wildcard certificate:

$ sudo cat /etc/ssl/private/wildcard_private.key /etc/ssl/certs/wildcard_combined.pem > /tmp/znc.pem
$ sudo mv /tmp/znc.pem /var/lib/znc
$ sudo chown znc:znc /var/lib/znc/znc.pem

From here I altered SSLCertFile to point to /var/lib/znc/znc.pem instead of /etc/ssl/certs/wildcard_combined.pem.

Anyone else hit anything similar? Is this worth a pull request against the znc tasks?

Occasional 'undefined variables' errors when running playbook in non-testing mode

I seem to be having an issue with the following line in site.yml:

- vars/{{ 'testing' if testing else 'user' }}.yml

I will occasionally get an 'undefined variables' error when I am NOT running in any sort of testing mode. The following is but one example but I've seen a similar one while running the enfs tasks as well:

TASK: [webmail | Configure the Apache HTTP server for roundcube] ************** 
fatal: [000.000.000.000] => {'msg': "One or more undefined variables: 'domain' is undefined", 'failed': True}
fatal: [000.000.000.000] => {'msg': "One or more undefined variables: 'domain' is undefined", 'failed': True}

FATAL: all hosts have already failed -- aborting

PLAY RECAP ******************************************************************** 
       to retry, use: --limit @/Users/Rob/site.retry

000.000.000.000            : ok=2    changed=1    unreachable=1    failed=0

I am able to solve these errors if I comment out the aforementioned offending line (and add - vars/user.yml) such that site.yml looks like this:

- vars/defaults.yml
- vars/user.yml
# - vars/{{ 'testing' if testing else 'user' }}.yml

Does anyone have any ideas on why this is happening?

Apache -> Nginx?

Several people have inquired about using Nginx, as it's lighter-weight than Apache. I'm open to the idea.

This Nginx config has some nice features, particularly when it comes to SSL, and could be used as a base template.

OpenVPN client and CA certificates expire after a month

The Server certificate remains valid for an extended period of time, but the rest of the certificates expire only 30 days after they were first generated. Flags need to be added to the certificate generation commands to prevent this from happening, and the number of days that a certificate will remain valid should be a user-configurable setting.

If self-signing, what takes place of wildcard_ca.pem?

I'm trying to write up how one generates a key and self-signs it. What should take the place of wildcard_ca.pem? Should it be just another copy of the .crt (again, in the event that one is self-signing). Or does one have to create a CA with its own private key?

SORBS rejected an email from my parents' gmail account

Sep 26 18:56:43 vps1 postfix/smtpd[28379]: 
    NOQUEUE: reject: RCPT from mail-ie0-f172.google.com[209.85.223.172]:
    554 5.7.1 Service unavailable; Client host [209.85.223.172] blocked
    using dnsbl.sorbs.net; Currently Sending Spam
    See: http://www.sorbs.net/lookup.shtml?209.85.223.172;
    from=<[email protected]> to=<[email protected]>
    proto=ESMTP helo=<mail-ie0-f172.google.com>

That's a legit gmail server. Out of all the spams blocked by RBLs, SORBS has rejected one for me, and it was this false positive!

# grep "blocked using" /var/log/mail.log* | wc -l
273
# grep "blocked using dnsbl.sorbs.net" /var/log/mail.log* | wc -l
1

For now I'm going to take it out of my configuration.

An article on the dspam wiki suggests using policyd-weight so that no single RBL causes a rejection:

I am not trusting one single RBL. Never ever would I do that!

From policyd-weight's homepage:

Postfix' built-in checks can be too tough for poorly configured clients: one hit, and the mail gets rejected.

That sounds good to me. What does everyone else think?

Add a role to summarize what was done.

After the install, we could display a message telling what is installed, what still remains to do (setting owncloud and znc...), urls to new web services (roundcube, owncloud, blog), and even a DNS config summary.

RSS?

This is sweet.

Would it be valuable to include Tiny Tiny RSS or another server-side RSS aggregator?

http://tt-rss.org/redmine/projects/tt-rss/wiki

Do you have an example of client config for the vpn?

I'm struggling slightly with what to do next with the files that end up in /tmp

bug in monit config for tomcat

sovereign / roles / monitoring / files / etc_monit_conf.d_tomcat
contains this block:

  start program = "/etc/init.d/tomcat6 start"
  as uid6 tomcat gid tomcat6
  stop program = "/etc/init.d/tomcat6 stop"
  as uid6 tomcat gid tomcat6

which should be:

  start program = "/etc/init.d/tomcat6 start"
  as uid tomcat6 gid tomcat6
  stop program = "/etc/init.d/tomcat6 stop"
  as uid tomcat6 gid tomcat6

XMPP Support

eJabberd server with postgres auth would fit in nicely

Nonstandard SSH port and ferm

If one of your first steps in setting up a new server is to change the ssh port, the ferm configuration will not anticipate you and you'll eventually find yourself locked out of your server.

• A workaround, if you can get root access to your server, is to edit /etc/ferm/ferm.conf and add your port to the proto tcp dport list. Then load your new rules: sudo ferm ferm.conf
• Or, don't change your SSH port!
• A more permanent fix might be to parameterize (or detect?) this and add when ferm is installed. I don't have this fix; still trying to get my first install going! :)

My email is webscale

Can you please add a webmail client that uses NODE.js and email server backed by MongoDB? This way when my email needs scaling I can just add servers.

TIA

Adding server's IP and DNS to etc_opendkim_TrustedHosts.j2

I'm having trouble making sense of this.

What should I be doing in roles/mailserver/templates/etc_opendkim_TrustedHosts.j2?

Contents of the file:

127.0.0.1                                                                   
# TODO add your server's IP and DNS hosts                                   
{% for domain in mail_virtual_domains %}                                    
{{ domain.name }}                                                           
{% endfor %}                                                                
{% for domain in mail_virtual_domains %}                                    
mail.{{ domain.name }}                                                      
{% endfor %} 

Could you provide an example of what this might look like with the info added?

Git hosting

Use https://github.com/sitaramc/gitolite or http://gitlab.org or something. Probably GitLab.

Firefox sync server

See Mozilla's guide to running your own sync server.

Wildcard cert requirement

It looks like wildcard certs are quite expensive (hundreds of dollars per year). Is it absolutely required for installing Sovereign? If so, can you explain a little more about how you self-signed or point to a tutorial? A quick Google search brought up this tutorial but it seems a little too easy for something that's so expensive from a provider.

Music jukebox?

I wrote one a few years ago: https://github.com/naggie/vosbox

It has a search engine and can crawl a server for all music, allowing streaming and playlist management/sharing.

Cheers
Naggie

tarsnap sha check

Would you like a PR that checks the SHA256 of the tarsnap download?

first five.five minutes on a server...

I asked the author about pulling some of this post:

http://lattejed.com/first-five-and-a-half-minutes-on-a-server-with-ansible

into a role for this project. I expect to replicate whats in Step 3. Getting Started in the readme for Sovereign and these steps from the blog post:

• Update & upgrade the system via apt-get
• Install fail2ban to block ssh brute-force attempts
• Create a new user so we don't have to use root
• Copy over our pub key
• Lock down sudo
• Lock down ssh to prevent root & password login
• Configure unattended security upgrades

It would ease new user setup and address #92

I've got time to work on it this weekend and can send a pull request if that's the kind of thing you're in to for this project. Lemme know.

encfs security

Hello,

I'm not sure I understand the point of encfs, or perhaps how it is implemented:

If the machine is considered trusted, then what's the point of encryption (not considering possible loss here like a laptop because it is always remote anyway)?

If the machine is not trusted, it cannot be allowed access to the decryption key, and thus cannot decrypt email, etc.

Is everything here done in-browser/mail client on the fly? I guess not, since I don't know whether such technology exists today, and even so, it probably wouldn't be compatible with search...

Can you please explain either how encfs is used (if I misunderstood something) or its utility (if understood correctly)?

Thanks in advance!

Hardcoded postmaster_address pointing to @syntax.cc in multiple files

I think these are leftovers from @al3x local setup:

./roles/mailserver/files/etc_dovecot_conf.d_20-imap.conf: postmaster_address = [email protected]
./roles/mailserver/templates/etc_dovecot_conf.d_15-lda.conf.j2:postmaster_address = [email protected]

Open source Pocket/Instapaper clone

Poche just went 1.0.0.

Comment every variable

Add a comment to each variable in vars/defaults.yml and vars/user.yml to explain what it does and how to use it.

Privacy patch for postfix

Thoughts on including the header replacement for postfix to strip out the client's sending IP?
https://we.riseup.net/debian/mail#getting-your-postfix-anonymized

apt-get install postfix-pcre

grep smtp_header_checks /etc/postfix/main.cf

smtp_header_checks = pcre:/etc/postfix/maps/smtp_header_checks.pcre

cat /etc/postfix/maps/smtp_header_checks.pcre

/^\s_(Received: from)[^\n](.)/ REPLACE $1 [127.0.0.1](localhost [127.0.0.1])$2
/^\s_User-Agent/ IGNORE
/^\s_X-Enigmail/ IGNORE
/^\s_X-Mailer/ IGNORE
/^\s*X-Originating-IP/ IGNORE

SMTP + TLS

Hello,

I have performed a fresh install of the distribution on a debian 7 system.

I do not manage to connect to SMTP with the command

$ openssl s_client -connect mail.domain.com:465

Log message is the following

Sep 18 16:57:03 ks33 postfix/smtps/smtpd[9030]: warning: cannot get RSA private key from file /etc/ssl/private/wildcard_private.key: disabling TLS support
Sep 18 16:57:03 ks33 postfix/smtps/smtpd[9030]: warning: TLS library problem: 9030:error:0B080074:x509 certificate routines:X509_check_private_key:key values mismatch:x509_cmp.c:330:
Sep 18 16:57:03 ks33 postfix/smtps/smtpd[9030]: warning: hostname 80-83-52-2.csm-inst.ch does not resolve to address 80.83.52.2: Name or service not known
Sep 18 16:57:03 ks33 postfix/smtps/smtpd[9030]: connect from unknown[80.83.52.2]
Sep 18 16:57:03 ks33 postfix/smtps/smtpd[9030]: warning: Wrapper-mode request dropped from unknown[80.83.52.2] for service smtps. TLS context initialization failed. For details see earlier warnings in your logs.
Sep 18 16:57:03 ks33 postfix/smtps/smtpd[9030]: disconnect from unknown[80.83.52.2]

The wildcard_private.key certificate seem validate

$ openssl rsa -in /etc/ssl/private/wildcard_private.key -check -noout
RSA key ok

I wonder if problem is with permission for vmail/postfix users:

$ ls -l /etc/ssl/private/
total 8
-rw-r----- 1 root ssl-cert 1704 Sep 18 13:47 ssl-cert-snakeoil.key
-rw-r--r-- 1 root root     1679 Sep 18 13:51 wildcard_private.key

I have already install sovereign distrib on this server 2 weeks ago. Worked fine. I just want to update everything with last modifications. I fully reinstall debian (disk format) but now, I do not manage to get it work

Any hint

Sincerely

Bertrand

ufw fails to enable cleanly

Following @ariddell's contribution, I switched my server to use ufw instead of ferm, and ran into this problem:

TASK: [Check status of ufw] ***************************************************
ok: [vps1.lukecyca.com]

TASK: [Enable ufw] ************************************************************
failed: [vps1.lukecyca.com] => {"changed": true, "cmd": ["ufw", "--force", "enable"],
"delta": "0:00:00.720280", "end": "2013-10-27 21:17:00.421354", "item": "", "rc": 1,
"start": "2013-10-27 21:16:59.701074"}
stderr: ERROR: problem running ufw-init

I logged in and ran it manually, and got a different error:

root@vps1:~# ufw --force enable
ERROR: Could not load logging rules

The firewall is running and operating properly, but the enable command fails to mark itself as enabled in /etc/ufw/ufw.conf, so it fails to start at boot (which I think is a problem).

Upon further investigation, it seems there is a bug whereby ufw logging is incompatible with our version of iptables. A fix has been committed, but it's not fixed in the version we're using (0.31.1-2).

If we turn logging off, this bug goes away and the enable command works fully, but I'm not sure that's a good solution.

Question: Dovecot/IMAP user creation

I am setting up a server with the following config:

mail_server_hostname: example.com
mail_mysql_username: mailuser
mail_mysql_password: password
mail_mysql_database: mailserver
mail_virtual_domains:
  - name: example.com
    pk_id: 1
    primary_user: bill
mail_virtual_users:
  - address: [email protected]
    password_hash: HASH
    domain_pk_id: 1
  - address: [email protected]
    password_hash: HASH
    domain_pk_id: 1

When I run ansible, the users are added to the mysql database(virtual_users) correctly, but only the "primary" user has a directory created under /decrypted-mail/example.com/

eg /decrypted-mail/example.com/bill

When I try and login using [email protected] the following appears in /var/log/mail.log

mail.warn:Sep  3 08:32:30 (none) dovecot: imap([email protected]): Error: user [email protected]: Initialization failed: Namespace '': mkdir(/decrypted-mail/example.com/steve) failed: Permission denied (euid=5000(vmail) egid=5000(vmail) missing +w perm: /decrypted-mail/example.com, dir owned by 0:0 mode=0755)
mail.warn:Sep  3 08:32:30 (none) dovecot: imap([email protected]): Error: Invalid user settings. Refer to server log for more information.

Should that directory be created by ansible during setup? Or should it be created by dovecot when the user first logs in? If the latter, then do we need to modify the permissions so it can create the directory?

Cheers,

Jordan

Alternative cal/cardDav service

http://baikal-server.com/ comes to mind.

dovecot.index.log permission error

I have a bunch of these errors littering /var/log/mail.err:

Sep 25 00:20:39 vps1 dovecot: imap([email protected]): 
    Error: link(/decrypted/lukecyca.com/me/dovecot.index.log,
    /decrypted/lukecyca.com/me/dovecot.index.log.2) failed: 
    Operation not permitted

Postgres instead of MySQL

It would also be great to have the option to use postgreSQL instead of MySQL

Tarsnap and security

The default configuration stores tarsnap's key in /root/tarsnap.key and backs up /decrypted_mail to tarsnap.

Unless I misunderstand, this seems to mean that an attacker who gains physical access to the drive gains access to the unencrypted tarsnap key and can then gain access to tarsnap backups containing unencrypted mail.

The solution then seems to be one of two options:

1. back up encrypted-mail, not decrypted-mail

Pros:

• Defends against any theoretical weakness in tarsnap

Cons:

• Encrypted data doesn't deduplicate as well.
• Attacker still gains access to all backups, possibly useful for an attack.

2. move key to /decrypted-mail
   Pros:

• Key is never unencrypted-at-rest on the drive
• Attacker can't gain access to backups

Cons:

• Trusts tarsnap

3. move key to /decrypted-mail and back up /encrypted-mail
   Pros:

• Most secure option

Cons:

• Encrypted data doesn't deduplicate well

---

I'm personally going with 2 for now. Happy to submit a pull request for this one; thanks for all the great work on Sovereign!

fail2ban emails

I'm getting about a dozen emails per day telling me that a particular IP has been banned. I don't see much use in these, so I think we should stop sending them.

Testing.yml password for [email protected] email account?

When testing/using the development environment (VM), what is the email password for the "[email protected]" user? I see from the testing.yml file, the password line reads:

password_hash: "$6$.f8oDqN1cDE/$Iyk8.scbwZCxw5pf9Flcvu.VYk9Jk77y/UaM0DyIcw9wouNqifXr3xV1fQPPNgBIM3BEEabAxePtC5Y/iX5vH1" #foo

I'm not sure what the comment #foo refers to? It's not the password I don't think.

Am I supposed to change this myself?

Thanks!

"No closing quotation" error in Vagrant.

I'm at commit #7240e65644 When running inside vagrant, I get the following error:

vagrant@ansible:/vagrant$ sudo ansible-playbook -i ./hosts site.yml

Traceback (most recent call last):
  File "/usr/bin/ansible-playbook", line 209, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/usr/bin/ansible-playbook", line 84, in main
    inventory = ansible.inventory.Inventory(options.inventory)
  File "/usr/lib/python2.7/dist-packages/ansible/inventory/__init__.py", line 91, in __init__
    self.parser = InventoryParser(filename=host_list)
  File "/usr/lib/python2.7/dist-packages/ansible/inventory/ini.py", line 40, in __init__
    self._parse()
  File "/usr/lib/python2.7/dist-packages/ansible/inventory/ini.py", line 44, in _parse
    self._parse_base_groups()
  File "/usr/lib/python2.7/dist-packages/ansible/inventory/ini.py", line 80, in _parse_base_groups
    tokens = shlex.split(line)
  File "/usr/lib/python2.7/shlex.py", line 279, in split
    return list(lex)
  File "/usr/lib/python2.7/shlex.py", line 269, in next
    token = self.get_token()
  File "/usr/lib/python2.7/shlex.py", line 96, in get_token
    raw = self.read_token()
  File "/usr/lib/python2.7/shlex.py", line 172, in read_token
    raise ValueError, "No closing quotation"
ValueError: No closing quotation

Export Tarsnap install in a dedicated role

I don't use tarsnap, so I must comment "- tarsnap.yml" out in roles/common/tasks/main.yml putting it in a dedicated role would give a more consistent behaviour, I'd just comment it in site.yml

Ansible 1.4 is required?

Commit f64ad69 introducted the "failed_when" module variable which in turn has been added in ansible 1.4.

This causes an issue when running the playbooks with installations prior to 1.4 (for example, at the time I'm writing this, Homebrew is still sitting on ansible 1.3.3).

Support for Firefox Sync (formerly Mozilla Weave)

http://docs.services.mozilla.com/howtos/run-sync.html

Enable Roundcube's managesieve plugin

I've been trying to enable the managesieve plugin of Roundcube which provides a decent GUI for managing server-side filters or rules. So far I've been unsuccessful as I get an "Unable to connect to server" error message in Roundcube no matter what I try. This post sums up the issue pretty nicely: http://www.sunfinedata.com/tutorial/enable-the-managesieve-of-dovecot-and-use-it-in-roundcubemail/, though it did not solve the problem.

This should probably be tagged as an 'enhancement' request. If anyone can get this working or help me figure this out I'd be grateful!

Easy config using group_vars instead of role-bound vars

We could gather all vars in a single file, and even use string replacement to make config as simple as:

https://gist.github.com/lisael/f9af1d96b1323c56ec8c

this allow the user to keep all control over her config but also provides a sensible default config with minimal pain.

New cal/card server and web client

Server: http://trac.calendarserver.org
Client: http://agendav.org

Or there are many others to choose from:

• https://code.google.com/p/sabredav/
• http://baikal-server.com
• http://radicale.org
• http://en.wikipedia.org/wiki/Comparison_of_CalDAV_and_CardDAV_implementations

dspam config has 64-bit library hardcoded

I installed on 32-bit debian, dspam can't find

StorageDriver /usr/lib/x86_64-linux-gnu/dspam/libhash_drv.so

because it is in

/usr/lib/i386-linux-gnu/dspam/libhash_drv.so

which leads to postfix rejecting incoming mail.

Either the README should state 64-bit as a prerequisite or ansible could configure this path depending on the platform (if it can do it, I don't know it very well).

Dnsmasq does not restart properly in the vpn role

OpenVPN should restart before dnsmasq so the 10.8.0.0 interface that dnsmasq tries to bind to is available. Currently dnsmasq attempts to restart first, which causes the task to fail.

Fails on znc install?

It doesn't seem to like the --enable-python option:

TASK: [Build and install znc] ************************************************* 
failed: [37.139.30.53] => {"changed": true, "cmd": ["./configure", "--enable-python", ";", "make", ";", "make", "install"], "delta": "0:00:00.004606", "end": "2013-08-24 00:41:13.029741", "item": "", "rc": 2, "start": "2013-08-24 00:41:13.025135"}
stderr: ./configure: --enable-python: invalid option
Usage:  ./configure [GNU long option] [option] ...
    ./configure [GNU long option] [option] script-file ...

Enable unattended-upgrades

Automatically installs security patches:
https://help.ubuntu.com/community/AutomaticSecurityUpdates

WebMail

Supply/configure RoundCube Webmail would be nice. I, and others I know, tend to use webmail more than native apps. RoundCube, IMO, functions a lot like Gmail.

http://roundcube.net

More modular firewall either with ferm or with ufw

It would be nice to have a more modular firewall, i.e., if you're not using XMPP you shouldn't have to have that port open in the ferm.conf (in roles/common)

This can be done with ferm and its include directive but ufw seems (to me at least) very easy to use and well-maintained.

Would there be any openness to a pull request?

Extract PKCS11* env variables as variables

I'm not familiar with setting up openvpn, but I've stumbled upon this while working on #39: https://github.com/al3x/sovereign/blob/1e9a560ff6351fadbb4196d3a8ca06a130e04b95/roles/vpn/templates/etc_openvpn_easy-rsa_2.0_vars.j2#L71-L72

export PKCS11_MODULE_PATH=changeme
export PKCS11_PIN=1234

I think those should be extracted as variables, especially now that we can have "mandatory" attributes, it's too easy to miss and it probably has an impact on security.