cloudpeers/tlfs

The Local-First SDK

tlfs's Issues

api for subscribing to added/removed documents

enabling a flag in an array of structs removes other fields

research mls

useful properties we care about

• confidentiality
• authenticity
• forward secrecy
• deniability
• post compromise secrecy

generate or get keypair on startup

dos resistant positional identifiers

make sure we are BEC

properties of BEC are:

• self-update
• eventual update (#53)
• convergence
• atomicity (#46)
• authenticity
• causal consistency
• invariant preservation

authenticity

leaving this for after the mvp. we have a system of vouching that works well enough for the mvp

use signatures to perform acl checks is now possible. for this we need to take a bit more care to ensure convergence in the presence of revocations.

iterate over table keys

I guess this will be easier once the radix tree lands

sum types

byzantine fault tolerance

by reusing dots byzantine nodes can cause havoc

oplog

• convergence root read permission / causally apply
• convergence partial read permission
• convergence revocation (replay) / key rotation
• convergence accross different schemas
• prune oplog and registry based on safe present
• unjoin scheduling

Remove length prefix in path

For some types (most, see below), the length is always the same. For other types, having a length prefix messes with the sort order. So it would be good if we could get rid of the mandatory length prefix and suffix in paths.

Prepreq for fixing #72

pub enum Segment {
    /// Document identifier.
    Doc(DocId), // fixed size
    /// Peer identifier.
    Peer(PeerId), // fixed size
    /// Randomness used to ensure path uniqueness.
    Nonce(u64), // fixed size
    /// Boolean primitive.
    Bool(bool),  // fixed size
    /// Unsigned integer primitive.
    U64(u64),  // fixed size
    /// Signed integer primitive.
    I64(i64),  // fixed size
    /// Utf8 string primitive.
    Str(String), // variable size, length prefix?
    /// Policy statement.
    Policy(Policy),  // fixed size?
    /// Path identifier.
    Dot(Dot), // fixed size
    /// Positional identifier.
    Position(Fraction), // variable size, inline varint encoding
    /// Signature primitive.
    Sig(Signature), // fixed size
}

custom storage backend

`Fraction` sorts incorrectly inside path

Fraction was created to insert at arbitrary points into an ORArray (#59). As the len is encoded inside the path segment prior to the bytes, this breaks the ordering inside the sled tree, which the ORArray implemention relies on.

Before the ORArray can really be used, this must be fixed.

mdns + invitation protocol

for the mvp we need some way of discovering local peers and adding them to the document. just giving them permission isn't enough without some mechanism to notify them?

text crdt

• complete feature / add some tests
• cursor api
• note taking app demo

broadcast and unjoin scheduling

• use libp2p-broadcast to send messages on apply/unjoin to all connected peers subscribed to a document
• on connect and periodically, perform an unjoin with another connected peer

collection of papers to read after shipping tlfs

• A Generic Undo Support for State-Based CRDTs
• Merkle Search Trees: Efficient State-Based CRDTs in Open Networks
• webauthn: https://webauthn.guide/

transform integration test

zstd transport wrapper

document change subscription api

Make permission just a single value

The only situation where a set with more than 1 entry can happen is if somebody changes the value for a dot, which is an adversarial scenario.

I am quite confident that if somebody can edit the dot store he can cause all kinds of issues. So we don't consider this at this time and deal with it later in a more principled way.

related: #29

tlfs website

• documentation
  • getting started (flutter/react todoapp)
  • api reference (dart/js/rust)
  • topic based guides

astro looks quite interesting

build showcase app

compress radix trees

libp2p nym transport

atomicity

support for atomic transactions (read committed transaction isolation)

generate api using ffi-gen

fix unjoin

mime blobs

• build shopping list app with picture tasks

prune tombstone set based on safe present

write docs on how/why it all works

convergence

in a non byzantine setting we should always converge

related to #40, #35

initial networking

fix join after unjoin: no permission to write

implement transform_dotstore and transform_crdt

sequence crdt

initial js api and demo app

depends on #28

cli tool for precompiling lenses and schemas into a file

Permissions as tree

When implementing join, I can do many things using very fast radix tree ops. E.g. removing expired from the store can be done with a single tree op instead of a loop.

E.g. this loop

        for buf in causal.store.iter() {
            let path = buf.as_path();
            if !self.expired.contains_prefix(path) && !causal.expired.contains_prefix(path) {
                if !self.can(peer, Permission::Write, path)? {
                    tracing::info!("join: peer is unauthorized to insert {}", path);
                    continue;
                }
                self.store.insert(&path);
            }
        }

can be turned into this:

        let mut store = causal.store.clone();
        store.tree_mut().remove_prefix_with(&self.expired.tree());
        store.tree_mut().remove_prefix_with(causal.expired.tree());
        for buf in store.iter() {
            let path = buf.as_path();
            if !self.can(peer, Permission::Write, path)? {
                tracing::info!("join: peer is unauthorized to insert {}", path);
                continue;
            }
            self.store.insert(&path);
        }

However, operations that involve permissions at present still involve querying permissions for every path.

It would be quite useful if we had a way to produce a tree for a permission and a peer, and then perform bulk operations for that tree. This would not change except when permissions are changed.

E.g.

let writeable = self.writeable(peer, Permission::Write);
store.tree_mut().retain_prefix_with(&writeable);

windows build of tlfs fails

spurious unauthorized

right after creating a document spurious unauthorized erros may be thrown

multidevice demo

initial dart API and demo app

• setup ci
• edit doc title
• edit todo title
• ui tests
• multiuser demo

use radix trees

flat dot store

encrypted crdt

reduce trust in cloudpeer by storing and performing join/unjoin on an encrypted crdt. this requires a deterministic encryption scheme with the disadvantage of suceptibility to statistical analysis. This can be mitigated by mapping low entropy path segments to high entropy random path segments. A suitable cipher would a nonce reuse resistant cipher like aes-gcm-siv.

libp2p nfc transport

compromise recovery

recover from accidental or malicious data addition/removal

causal consistency

updates are applied in causal order

related to #40