stevearm/vault

Sync data between phones, laptops, and the web

Vault

Vault is a distributed information keeping system that runs on top of CouchDB. It is a Java daemon that syncs between multiple vaults.

Install

Installing Vault

Vault can run standalone as a pure CouchApp

1. Install CouchDB
2. Go to Futon's replicator page
3. Replicate from http://stevearm.iriscouch.org/vault-release to local vault
4. Replicate from http://stevearm.iriscouch.org/vaultdb-release to local vaultdb
5. You've now got an unconfigured local vault. Go to the install page to set it up

Installing Sentinel

Sentinel is a java daemon that syncs vaults automatically in the background for you.

1. Create a folder somewhere (c:/vault)
2. Download the newest vault.jar from the releases page into that folder
3. Create a CouchDB user with admin rights for Vault to use
4. Edit CouchDB's local.ini file:
   • Ensure the [couchdb] section has a uuid property (this should already exist on any modern version of CouchDB. If it doesn't, set it to the vaultId value in http://localhost:5984/vault/id)
   • Add a [vault] section at the bottom and add the following keys
     • username (required)
     • password (required)
   • Add a line to the os_daemons section of local.ini
     • Windows: vault = java -cp c:/vault/vault.jar com.horsefire.vault.CouchMain c:/vault
     • Linux: vault = java -cp /opt/vault/vault.jar com.horsefire.vault.CouchMain /opt/vault

Once you've added that line, CouchDB will keep Sentinel running, and Sentinel will read everything it needs from CouchDB's config using the api, and keep syncing.

Licences

Vault is licenced under Apache Licence 2.0. It contains libraries licenced under:

• Apache Licence 2.0 (Gson, Guava, Joda-Time, JCommander, LightCouch)
• Gnu Lesser General Public Licence LGPL (Logback)
• MIT Licences (AngularJS, AngularUI, Bootstrap, SLF4J, QR Code Generator, cornercouch)

As the LGPL libraries were not modified in any way, they can be released under non-GPL licences.

Default port

Apparently port 5995 is unassigned. If/when I get discovery protocols working and/or need a port for the sentinel to listen on, use this.

User Workflow

A user adds new vaults, installs/adds new apps, and updates connection information from a CouchApp directly against the vault db.

Responsibilities

The sentinel needs to perform the following responsibilities, and needs to do it using only the data in the Data Structure section below.

Ensures local data is correct

Vault should make sure the following are always true:

• There is publicly readable database called vault
• The vault db has an entry id with this vault's id
• There is non-publicly readable database called vaultdb
• The vaultdb db has an entry for this CouchDB's uuid
• The vaultdb entry for this CouchDB should have an accurate username, password, signature, and sentinel version

Syncs with remote vaults (unimplemented)

• Periodically sync with other vaults
  1. Iterate through all vaults with an addressable block, in priority order
  2. Check the signature. If it does not match, skip vault
  3. Sync each database my vault has that the remote vault should have

Trigger workers for installed vault apps (unimplemented)

• Trigger workers of installed apps
  1. Listen to the _changes feed for each installed app, watching for a changed to the worker entry
  2. If the "triggered" time is before the "started", do "run the worker" (see below)
  3. Update the "started" time to now
  4. Start the worker
  5. When the worker ends, update the "worker_finished" to now

Data Structure

Vault db entry for each vault

• id: vault_id
• type: vault
• name: string
• signature: object
• dbs: array of database_name
• addressable: object (this only exists if vault is externally accessable) ** host: string ** port: int ** priority: int ** enabled: boolean

Vault db entry for each app

• id: random
• type: app
• name: string
• db: database_name
• ui: entry point (if _design/ui/index.html then: "ui/index.html")
• worker: id for tracking and triggering worker times ** optional, but if specified, look for worker.jar in ui, and run with: --db --host --port --username --password

Worker trigger db entry for each app

• triggered: timestamp (when someone requested the worker to run)
• started: timestamp (last time started)
• finished: timestamp (last time finished)

vault's Issues

Add coveralls testing

Add coveralls testing

Allow upgrades from UI

Provide a page to upgrade the vault and vaultdb design docs from anywhere (default to stevearm.iriscouch.com as the sentinel does).

Show version of sentinel on UI

This is a bug (sentinel updates the version in the database to "UNKNOWN-VERSION") and an enhancement (the UI doesn't display the sentinel version at all)

Create install page to setup vault/id

Create a simple install.html page that sets up/validates vault/id.

1. Load only angular, none of the rest of vault
2. Load id if it exists
3. Load the signature from /
4. If signature has uuid compare it to the one in id.id (or use it if id had no value)
5. If id specifies a vaultDbName, validate that it exists
6. If id is missing, or has no vaultDbName, or wrong name, provide choice of db names

Save login credentials in browser

Split out from #13.

Chrome only saves creds after the POST is done (real browser POST), and it'll only save them from a form in the base page, not dynamically added.

Support HTTPS

Currently, both the sentinel and the couchapp (in the links to other vaults) use http. Add a field to the addressable field in the vault document called ssl (true/false) and have the sentinel and the links in the couchapp obey it.

Browsers should have up-to-date certs (for the links) and Java gets it's certs from either the JVM or the OS (not sure) so they should be reasonably up-to-date as well.

Install should lock down dbs if possible

The sentinel, when creating the vault and vaultdb databases, should set their security policies as well, instead of just complaining if they're wrong.

No way to trigger a sentinel run on demand

The UI should have a way to trigger sentinel to run 'now' so that sentinel will notice and run one sooner.

We can just add a "requested sync" timestamp to the vault's entry in vaultdb. Sentinel would then have to either poll couch all the time, or subscribe to the changes feed.

We will also need a way to see what the last run was, but this shouldn't got in a synced db, or there will be tons of churn in the network.

Very bad handling if vault/id/vaultDb is set wrong

While trying to manually update the id documents on the vaults that have no sentinel, I ran into a bunch of errors.

Test both:

• id document has no vaultDbName defined (this can be avoided by upgrading validate_doc_update.js to not allow this case)
• id document has an invalid vaultDbName defined

Improve log messages for non-admin accounts

When vault is running with a non-admin account (can't create new databases), it fails with strange 404 errors when trying to bootstrap the vault or vaultdb accounts. See if there's a way to verify the privileges of the current account at the start, and log something descriptive and helpful.

Login page sucks

The login page:

• doesn't respond to ENTER
• isn't aligned properly
• doesn't prompt me to remember my password (because it doesn't do a real page transition?)

About page sentinel run breaks if no sentinel

Page displays a bunch of NaN for last sentinel run on Vaults without sentinels

Allow manual setting of vault/id on non-sentinel vaults

Vaults without a sentinel (such as iriscouch), won't have anything creating or maintaining the id entry in vault database. Provide a way to create and/or set this from the UI

False error "Something went wrong during full sync"

On-demand syncing shows an error, even though the ajax call looks fine. Probably checking the wrong response information

Create info footer

Create a footer at the bottom of the page with:

• Current release number
• Link to check for new version
• "Built with glyphicons" link

Trigger replication to db of my choice

Somewhere in the UI, allow an on-demand replication for whatever vault I want

Create sentinel info page

Create a spot somewhere on the UI to show the sentinel's last run, and any important messages from it. Store this info on the non-synced vault database.

sentinel syncs all of vault, so 'id' is thrashing

Sentinel syncs the vault database, but also makes sure that vault's 'id' entry always references the current vault, so it thrashes.

Stop normal replication and just do replication of the single design doc id by specifying that id