Giter Site home page Giter Site logo

skiplabs / skdb Goto Github PK

View Code? Open in Web Editor NEW
101.0 4.0 9.0 452.98 MB

SKDB is an embedded SQL database that stays in sync.

Home Page: https://skdb.io

License: MIT License

Shell 19.55% Dockerfile 0.27% Makefile 2.76% LLVM 0.58% C 25.95% C++ 1.55% Python 5.67% Emacs Lisp 2.20% TypeScript 28.38% CSS 0.39% HTML 0.07% JavaScript 3.33% Kotlin 7.31% Lua 1.38% Perl 0.61%
database reactive sql

skdb's Introduction

skdb

This repository contains the source code for SKDB and its associated components.

Dependencies

You will need at least the following installed:

  • LLVM 15.
  • wasm-ld 15 (this is part of LLVM, but your distribution might split it into a separate package: your wasm-ld version must match your LLVM version).
  • typescript 5.2 or later.

License

SKDB is MIT licensed.

skdb's People

Contributors

beauby avatar bennostein avatar gregsexton avatar jberdine avatar leinadsepol avatar ltratt avatar mbouaziz avatar pikatchu avatar schickling avatar skiplabsdaniel avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

skdb's Issues

Cannot insert a generated id using a select

I want to do this:

INSERT INTO pk_single_row_hist SELECT local_sequence_number(), datetime(), client, value FROM pk_single_row;

But I cannot.

Error: line 1, character 0:
local_sequence_number expects a string literal

This is true of id() and some other things that seem to behave differently if they're an expression in a select's col expression list or an insert's value list.

Parse error when aliasing a sub-select

Trying to create a view:

CREATE VIRTUAL VIEW control_plane_create_db_outstanding_requests AS
SELECT reqs.id, reqs.skdb_author, reqs.skdb_access, reqs.db_name, res.success
FROM control_plane_create_db_requests reqs
LEFT OUTER JOIN (SELECT * FROM control_plane_create_db_results WHERE success IS NULL OR success < 1) AS res
ON reqs.id = res.id;

Getting the error:

     CREATE VIRTUAL VIEW control_plane_create_db_outstanding_requests AS
     SELECT reqs.id, reqs.skdb_author, reqs.skdb_access, reqs.db_name, res.success
     FROM control_plane_create_db_requests reqs
     LEFT OUTER JOIN (SELECT * FROM control_plane_create_db_results WHERE success IS NULL OR success < 1) AS res
                                                                                                          ^
                                                                                                          |
                                                                                                           ----- ERROR
Error: line 20, character 106:
Unexpected token AS: expected SEMICOLON or EOF

`read-only` `INSERT` forbidden?

I was a bit surprised that I can't insert a row with skdb_access set to read-only:

INSERT INTO messages (username, message, skdb_access) VALUES ('user2', 'msg3', 'read-only');                                                                                     
Could not eval query. Try `.help`                                                                                                                                                       
SkRuntimeExit [Error]: INSERT INTO messages (username, message, skdb_access) VALUES ('user2', 'msg3', 'read-only');                                                                     
^                                                                                                                                                                                       
|                                                                                                                                                                                       
 ----- ERROR                                                                                                                                                                            
Error: line 1, character 0:                                                                                                                                                             
cannot insert with skdb_access set to read-only (user f8b7WiJln5Scgj-T5inTJcn7Fi)                                                                                                       
...

What are the valid values of skdb_access? Are they: user ID; group ID; read-write; and that's it? Or are there are other valid predefined groups one can use?

Clarifying CLI flags

At the moment if you do npx skdb --help (or just npx skdb) you get this help string:

Usage: node_modules/.bin/skdb-cli [--help]
    [--db <db>]  -- Name of database to connect to.
    [--host <host>]  -- URI to connect to. Default: wss://api.skiplabs.io
    [--access-key <key>]  -- Access key to use. Default: first specified in credentials file.
    [--dev]  -- Connect to the local dev server, use this to access credentials.
    [--json-output]  -- Rows are output as JSON objects. Default: false.
    [--pipe-separated-output]  -- SQL output is separated by a pipe. Default: false.
    [--simple-output]  -- No cute terminal usage. E.g. no readline.
    [--add-cred]  -- Add a credential to /home/user/.skdb/credentials. Pass the private key b64 on stdin.
    [--create-db <db>]  -- Create database.
    [--create-user]  -- Create user.
    [--remote-repl]  -- Interactively execute SQL queries against the server.
    [--local-repl]  -- Interactively execute SQL queries against a local db.
    [--schema]  -- Dump the database schema as SQL DML.
    [--table-schema <table>]  -- Dump the table schema as SQL DML.
    [--view-schema <view>]  -- Dump the view schema as SQL DML.

Anything passed on stdin will be evaluated as sql by the remote db.

It's a bit difficult from this to work out exactly what combinations of flags are valid. For example consider --create-user:

$ npx skdb --create-user
# prints out the same help string as earlier
$ npx skdb --create-user --dev
# prints out the same help string as earlier
$ npx skdb --create-user --db chat-app
Could not find credentials for chat-app in /home/user/.skdb/credentials
$ npx skdb --create-user --dev --db chat-app
# Creates a user!

The eventual sequence of flags does make sense, but it's hard to guess from the string. I wonder if we can make a more helpful help string? I'm not sure exactly what all the valid combinations are, but I think it's something like:

[--dev | --host <host> --access-key <key>]
[--json-output | --pipe-separated-output | --simple-output]
--add-cred | --create-db <db> | --create-user | --remote-repl | --local-repl | --schema | --table-schema <table> | --view-schema <view>

Warning: I might have got the combinations very wrong!

Evaluating a select where in (select...) query does not seem to work

root@ws://localhost:3586/skiplabs> select * from task_events;
┌─────────┬───────────────────────────────┬──────────────────────────────┬────────────┬───────────────────────┬─────────────┬──────────────┐
│ (index) │              eid              │             tid              │   event    │           t           │ skdb_author │ skdb_access  │
├─────────┼───────────────────────────────┼──────────────────────────────┼────────────┼───────────────────────┼─────────────┼──────────────┤
│    0    │ 'f901qBX3mFn7cKmNYxufzsCxKp4' │ 'f90149mur-5jz9xT8AJa9LB4N9' │ 'created'  │ '2024-02-21 17:02:39' │   'root'    │ 'read-write' │
│    1    │ 'f901t_YPdQOXA9AI24hAlm6nv8E' │ 'f90149mur-5jz9xT8AJa9LB4N9' │ 'assigned' │ '2024-02-21 17:02:53' │   'root'    │ 'read-write' │
└─────────┴───────────────────────────────┴──────────────────────────────┴────────────┴───────────────────────┴─────────────┴──────────────┘
root@ws://localhost:3586/skiplabs> SELECT max(eid) AS eid FROM task_events WHERE event = 'assigned' GROUP BY tid;
┌─────────┬───────────────────────────────┐
│ (index) │              eid              │
├─────────┼───────────────────────────────┤
│    0    │ 'f901t_YPdQOXA9AI24hAlm6nv8E' │
└─────────┴───────────────────────────────┘
root@ws://localhost:3586/skiplabs> select * from task_events where eid in (SELECT max(eid) AS eid FROM task_events WHERE event = 'assigned' GROUP BY tid);

Relevant schema:

CREATE TABLE task_events (
  eid TEXT PRIMARY KEY,
  tid TEXT NOT NULL,
  event TEXT NOT NULL,
  t TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
  skdb_author TEXT NOT NULL,
  skdb_access TEXT NOT NULL
);

[skdb] repl crash on qualified select-star (`select t.* from ...`)

I'll take a look at this once I'm done with current tasks, just filing an issue and setting aside for now as it's unrelated.

Repro (same input evaluates correctly when piped in, fails when entered at the repl)

$ make build/skdb && echo "create table t (x integer, id integer primary key); create table tt (y integer, id integer primary key); insert into t values (100,1);insert into tt values (42,1); select t.* from t inner join tt on t.id = tt.id;" | build/skdb --always-allow-joins
make: 'build/skdb' is up to date.
100|1

$ build/skdb --always-allow-joins
skdb> create table t (x integer, id integer primary key); create table tt (y integer, id integer primary key); insert into t values (100,1);insert into tt values (42,1);
skdb> select t.* from t inner join tt on t.id = tt.id;
*** Uncaught exception of type KeyNotFound: Key not found
*** Stack trace:
File std:/skdb/prelude/src/skstore/Context.sk, line 1710
	???()
File std:/skdb/prelude/src/core/language/Exception.sk, line 126
	???()
File std:/skdb/prelude/src/core/language/Exception.sk, line 121
	sk.vtry.22()
File std:/skdb/prelude/src/skstore/Context.sk, line 1700
	SKStore.withRegionVoid<_>()
File skdb:/skdb/sql/src/Main.sk, line 386
	sk.SKDB_main__Closure1__call()
File skdb:/skdb/sql/src/Main.sk, line 409
	skip_main()
[1]    58374 abort      build/skdb --always-allow-joins

Adding a member to a group leaks this information to all users.

      const grp = await skdb.createGroup();
      await grp.setMemberPermission(someUser, "r");

Results in

select * from skdb_group_permissions;
┌─────────┬───────────────────────────────┬───────────────────────────────┬─────────────┬──────────────┐
│ (index) │            groupID            │            userID             │ permissions │ skdb_access  │
├─────────┼───────────────────────────────┼───────────────────────────────┼─────────────┼──────────────┤
│    0    │ '3ZVVNSpKUkylYNO0ZIGm7Ey5oI1' │ 'fA96m2waGvUIAw-dyA9Z3I75IJC' │      4      │ 'read-write' │

The read-write permission means that all users can see that this user is in the group with permission 4. We should only be sharing this information with the user and group admins.

This info could be used to figure out e.g. a user's friend list or worse depending on the app.

Auth errors when creating multiple dev users at the CLI.

Repro:

  1. run a devserver with docker run -it -p 3586:8080 skiplabs/skdb-dev-server:quickstart
  2. npx skdb --create-user alice --db repro --dev runs successfully, outputs credentials to stdout
  3. npx skdb --create-user bob --db repro --dev fails with the following:
node:internal/process/esm_loader:48
      internalBinding('errors').triggerUncaughtException(
                                ^
Authorization error
Thrown at:
    at loadESM (node:internal/process/esm_loader:48:33)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)


Node.js v20.7.0

Pattern starting with % don't work

This doesn't work:

echo "select * from (select '' as path) where path like '%foo';" | skdb

But the following does:

echo "select * from (select '' as path) where path like '%foo%';" | skdb

Server "request tail" is broken

The "request tail" command of the SKDB replication protocol is currently broken. The call to skdb.canMirror() is missing a --data argument, but even then, it invokes skdb can-mirror <table_name> "<requested schema>", whereas <table_name> should be the actual schema of the table.

The reason this went unnoticed is probably that clients only use the "request tail batch" command, the implementation of which does not call canMirror.

Two possible fixes: ditch the canMirror() call in "request tail", or ditch the "request tail" command altogether, and only support "request tail batch".

Limit use of `newObstack` to implementation of `withRegion` functions

Also rename newObstack to e.g. unsafeNewObstack.

Care must be taken when creating a new obstack since if mutable values from the old obstack are used by computations run in the new obstack, it is possible to create old-to-young pointers, and the runtime system does not handle them safely.

See discussion: #126

Cannot insert expressions

skdb> select 'foo' || 'bar';
┌──────────┐
│  col<0>  │
├──────────┤
│ "foobar" │
└──────────┘
skdb> create table test (x text);
skdb> insert into test values ('foo');
skdb> insert into test values ('foo' || 'bar');
insert into test values ('foo' || 'bar'); 
^
|
 ----- ERROR
Error: line 1, character 0:
Arbitrary expressions not supported for INSERT

But this as a workaround does work.

skdb> insert into test select 'foo' || 'bar';
skdb> select * from test;
┌──────────┐
│    x     │
├──────────┤
│  "foo"   │
│ "foobar" │
└──────────┘

Server using most recently created user?

If I start the dev server and then the CLI I am logged in as root:

$ npx skdb --dev --db chat-app --remote-repl
root@ws://localhost:3586/chat-app>

If I then create a user with the CLI I get credentials I can use but (and I think this is sensible) nothing is put in ~/.skdb/credentials:

$ npx skdb --dev --create-user --db chat-app
Successfully created user:  {
  f8a0ksyieWZ5aWIU9iLrXGMqsV4: '...'
}
$ cat ~/.skdb/credentials
{}

However now when I start the CLI I am logged in as that recently created user:

$ npx skdb --dev --db chat-app --remote-repl
f8a0ksyieWZ5aWIU9iLrXGMqsV4@ws://localhost:3586/chat-app>

This surprises me because I have an empty credentials file and --help says:

$ npx skdb --help
Usage: node_modules/.bin/skdb-cli [--help]                      
...   
    [--access-key <key>]  -- Access key to use. Default: first specified in credentials file.

Indeed, I can't see any file anywhere on my file system which contains the string "f8a0ksyieWZ5aWIU9iLrXGMqsV4". Interestingly, if I create more users with --create-user I am still logged in as "f8a0ksyieWZ5aWIU9iLrXGMqsV4".

My best guess is that the server has the following logic:

  1. If no users have been created, use root as the access key.
  2. If one or more users have been created, use the first of those as the access key.

This isn't necessarily wrong or bad behaviour, but I'm also not sure if it's what's intended or not. And, perhaps, I'm completely missing out on something and there is state locally on my machine that I'm not aware of!

Creating an SKDB instance as a worker causes JS to hang

Repro:

import { createSkdb } from "skdb";
const db = await createSkdb({ asWorker: true });

If you change asWorker to false the node process will exit cleanly, otherwise it hangs.

The skdb type has a closeConnection method. We should probably change this to close() which should close any conns, other resources, and the worker if needed.

optimize 32 bit memcpy for unaligned destinations

The current 32 bit implementation of memcpy performs word-by-word copies tacitly assuming that the source and destination are word-aligned. For the use of memcpy to concatenate strings, the destination need not be aligned. It may be faster to use an implementation that maintaines a "carry" word across loop iterations so that word-width reads and writes are both performed, and the word to write is calculated as a bitwise combination of the carry word from the previous iteration and the word read in the current iteration.

If SKIP_String_concat2 or SKIP_String_concatN shows up while profiling, this optimization could be worthwhile.

We're not wrapping tables in SKDBTable when using a worker??

Not sure the title is correct, but I don't know what else is going on.

I run an app using skdb-react and the dev console and it doesn't load the users when run as a worker, but it does when not run as a worker.

I get this error in the console.

dev-console.js:105 Uncaught (in promise) TypeError: userRows.onlyColumn is not a function
    at dev-console.js:105:35

Soak is broken with new parser

make test-soak results in the following. Doesn't support naming expressions? cc @beauby

Running expectation checks for checkpoint { client: 1, latest_id: 30 }
file:///skfs/build/package/skdb/dist/sk_types.mjs:195
                let error = new SkRuntimeExit(exitCode, message?.trim());
                            ^

SkRuntimeExit [Error]: select client, value >= @latest_id as check
                                      ^
                                      |
                                       ----- ERROR
Error: line 1, character 38:
Unexpected token CHECK: expected IDENTIFIER
Internal error: select client, value >= @latest_id as check
                                      ^
                                      |
                                       ----- ERROR
Error: line 1, character 38:
Unexpected token CHECK: expected IDENTIFIER
    at Utils.main (file:///skfs/build/package/skdb/dist/sk_types.mjs:195:29)
    at SKDBSyncImpl.main [as runLocal] (file:///skfs/build/package/skdb/dist/skdb_skdb.mjs:208:33)
    at file:///skfs/build/package/skdb/dist/skdb_database.mjs:78:29
    at SKDBSyncImpl.runner (file:///skfs/build/package/skdb/dist/skdb_skdb.mjs:199:30)
    at SKDBSyncImpl.exec (file:///skfs/build/package/skdb/dist/skdb_database.mjs:76:25)
    at SKDBImpl.exec (file:///skfs/build/package/skdb/dist/skdb_database.mjs:197:46)
    at check_expectation (file:///skfs/sql/server/test/soak_client.mjs:154:30)
    at check_expectations (file:///skfs/sql/server/test/soak_client.mjs:204:3)
    at file:///skfs/sql/server/test/soak_client.mjs:356:9
    at file:///skfs/build/package/skdb/dist/skdb_skdb.mjs:238:21 {
  code: 0,
  cause: SkException [Error]: Unexpected token CHECK: expected IDENTIFIER
      at Utils.ethrow (file:///skfs/build/package/skdb/dist/sk_types.mjs:328:23)
      at LinksImpl.js_throw (file:///skfs/build/package/skdb/dist/sk_runtime.mjs:30:53)
      at toWasm.js_throw (file:///skfs/build/package/skdb/dist/sk_runtime.mjs:128:58)
      at SKIP_throw (wasm://wasm/00dc0932:wasm-function[2247]:0x34921)
      at SQLParser.Parser__errorUnexpectedToken (wasm://wasm/00dc0932:wasm-function[4707]:0xbd0a6)
      at sk.SQLParser_Parser__parseSelectResult (wasm://wasm/00dc0932:wasm-function[5409]:0xf2cf3)
      at sk.SQLParser_Parser__parseSelectResults (wasm://wasm/00dc0932:wasm-function[5407]:0xf27af)
      at sk.SQLParser_Parser__parseSelectCoreQuery (wasm://wasm/00dc0932:wasm-function[3694]:0x8035a)
      at sk.SQLParser_Parser__parseSelectCore__Closure0__call (wasm://wasm/00dc0932:wasm-function[3693]:0x802c2)
      at SQLParser.Parser__parse_one_of_keywords (wasm://wasm/00dc0932:wasm-function[4714]:0xbd644) {
    cause: SkRuntimeExit [Error]: select client, value >= @latest_id as check
                                          ^
                                          |
                                           ----- ERROR
    Error: line 1, character 38:
    Unexpected token CHECK: expected IDENTIFIER
    
        at Utils.exit [as SKIP_throw_cruntime] (file:///skfs/build/package/skdb/dist/sk_types.mjs:89:19)
        at toWasm.SKIP_throw_cruntime (file:///skfs/build/package/skdb/dist/sk_runtime.mjs:129:58)
        at SKIP_exit (wasm://wasm/00dc0932:wasm-function[4663]:0xb8480)
        at sk.skipExit.6 (wasm://wasm/00dc0932:wasm-function[3502]:0x76f58)
        at sk.SKDB_main__Closure0__call__Closure1__call (wasm://wasm/00dc0932:wasm-function[3495]:0x76a28)
        at sk.SKDB_main__Closure0__call (wasm://wasm/00dc0932:wasm-function[4116]:0x95443)
        at sk.vtry__Closure0__call.17 (wasm://wasm/00dc0932:wasm-function[3458]:0x74fcd)
        at SKIP_call0 (wasm://wasm/00dc0932:wasm-function[5777]:0x10814a)
        at Utils.call (file:///skfs/build/package/skdb/dist/sk_types.mjs:276:33)
        at Utils.etry [as SKIP_etry] (file:///skfs/build/package/skdb/dist/sk_types.mjs:293:29) {
      code: 1
    }
  }
}

Piping input to the CLI causes readline errors

A classic trick I use for setting up databases is piping commands into their CLI. This causes a readline issue at the moment:

echo "CREATE TABLE t3 ( x TEXT );" | npx skdb --dev --db chat_app --remote-repl
root@ws://localhost:3586/chat_app> CREATE TABLE t3 ( x TEXT );
node:internal/errors:497
    ErrorCaptureStackTrace(err);
    ^

Error [ERR_USE_AFTER_CLOSE]: readline was closed
    at new NodeError (node:internal/errors:406:5)
    at [kQuestion] (node:internal/readline/interface:397:13)
    at node:readline/promises:57:22
    at new Promise (<anonymous>)
    at Interface.question (node:readline/promises:34:12)
    at remoteRepl (file:///home/ltratt/.npm/_npx/cc5540d868b72d43/node_modules/skdb/dist/skdb-cli.mjs:295:32)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async file:///home/ltratt/.npm/_npx/cc5540d868b72d43/node_modules/skdb/dist/skdb-cli.mjs:458:5 {
  code: 'ERR_USE_AFTER_CLOSE'
}

Node.js v20.9.0

I thought that adding --simple-output might solve this, but the error is unchanged.

An alternative to fixing this might be to add a commands (or similarly named) switch which takes a file with commands as input?

Is this project open source?

Hi, I just saw this project mentioned in a tweet, so I don't have a lot of context. There's no license in the README which often indicates that the project doesn't intend to be open source. That's fine of course - each project needs to decide that for themselves based on their business model and/or other factors.

That said, if this project is in fact open source, then it'd definitely be a good idea to add a license. At the very least it'd help you get a few extra stars from people like me who need to reduce the chance of lock-in risk for critical parts of their infrastructure, even if using the company's associated cloud offerings.

Please feel free to close, of course, if the project isn't ready for a license, for whatever reason.

Cannot insert using select if using partial column selection

Given

CREATE TABLE pk_single_row_hist (
  id INTEGER PRIMARY KEY,
  t TEXT DEFAULT CURRENT_TIMESTAMP,
  client INTEGER,
  value INTEGER
);

When

INSERT INTO pk_single_row_hist (client, value) SELECT client, value FROM pk_single_row;

Then

Error: line 2, character 7:
Incompatible select size

[skc] Suspicious handling of `@synonym`

According to the error message (Did you mean uppercase, uppercase or uppercase?), the handling of @synonym seems broken.

  File "src/build_runner.sk", line 264, characters 26-47:
  The field upper is undefined. Did you mean uppercase, uppercase or uppercase?
  262 |       | Some(bs_unit) ->
  263 |         this.build_script_outputs[bs_unit].metadata.each((k, v) ->
  264 |           cmd.env(`DEP_${dep.pkg.name().upper()}_${k}`, v)
      |                          ^^^^^^^^^^^^^^^^^^^^^^
  265 |         )

for

# prelude/src/core/primitives/String.sk
[...]
 444 │ @synonym("toUpperCase")                                                                                                 
 445 │ @synonym("toUpper")                                                                                                     
 446 │ @synonym("upper")                                                                                                       
 447 │ @synonym("uppercased")                                                                                                  
 448 │ @synonym("upcase")                                                                                                      
 449 │ @synonym("uc")                                                                                                          
 450 │ fun uppercase(): String {                                                                                               
 451 │ │ String::fromChars(this.getIter().map(c → c.capitalize()).collect(Array))                                              
 452 │ }   
[...]

"Access key" or "user ID"?

The CLI talks about "access keys":

$ npx skdb --help
Usage: bin/skdb-cli [--help]                      
    [--access-key <key>]  -- Access key to use. Default: first specified in credentials file.

but skdb_users uses "userID":

root@ws://localhost:3586/chat-app> .schema
CREATE TABLE skdb_users (
  userID TEXT PRIMARY KEY,
  privateKey TEXT NOT NULL
);

I guess "user ID" is now the preferred terminology?

Segfault in withRegionFold under certain conditions

Not using line numbers because they might change over time.

Inside Context.sk, inside the function withRegionFold, the "next" call on the iterator segfaults. It only segfaults on one specific code path, the one coming from EagerDir.sk, inside the function "update", the last call to withRegionFold.

I added a temporary fix that consists in rewriting the iterator from:
it = dirty.iterator()

To:
it = dirty.iterator().collect(Array).iterator()

But we should investigate why this segfaults without it.

Under ARM M3 (apple)

In the following version of ubuntu:
Distributor ID: Ubuntu
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy

With the following version of clang:
Ubuntu clang version 15.0.7
Target: aarch64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

And the following version of llvm:
Ubuntu LLVM version 15.0.7
Optimized build.
Default target: aarch64-unknown-linux-gnu
Host CPU: (unknown)

Random user names have a non-random prefix?

It seems that --create-user creates users with a non-random prefix:

$ npx skdb --dev --create-user --db chat-app
Successfully created user:  {
  f7rXJgr_UkpaoGDYUuM0c3b8Sz2: '...'
}
$ npx skdb --dev --create-user --db chat-app
Successfully created user:  {
  f7rXKWj0XQN155xsrF0PgSSwqN0: '...'
}
$ npx skdb --dev --create-user --db chat-app
Successfully created user:  {
  f7rXW0KTJrjfjya0cCRZNFZ3Ht3: '...'
}

There might be a good reason for this, and it isn't a security issue, but it makes it quite hard as a human to differentiate one username from another: you have to skip the first 4 characters ("f7rX") before getting to the actually random part.

Referencing the current SKDB userID.

I've realised that I'm often having to insert the SKDB user ID I'm currently logged in as. In the CLI I have to spell it out each time (f8b7ObYn10n1kuppab1Ds4bti3 etc.). In TypeScript I do the parameters dance of "... userID = @userID", {userID: 'f8b7ObYn10n1kuppab1Ds4bti3'}.

Could we introduce an SQL function (or constant?) that references the current SKDB userID? It would make a lot of this stuff a lot easier. It would also make it more obvious (at least to me!) when I'm referencing the current user ID (which I do 99% of the time) or another user ID (which I've so far done very rarely). By making the latter pop out, it might help me more carefully audit when I'm doing something "cross user", which is probably the most likely place I'm going to introduce privacy mistakes.

[skargo] Syntax errors produce misleading error messages

When the compiler frontend produces errors, skargo build is warning of "No such file or directory" instead of reporting the skc error messages.

e.g. if I introduce a syntax error

diff --git a/sql/src/Main.sk b/sql/src/Main.sk
index e87e010c..77ef4d71 100644
--- a/sql/src/Main.sk
+++ b/sql/src/Main.sk
@@ -763,3 +763,4 @@ fun execToggleView(args: Cli.ParseResults, _options: SKDB.Options): void {
 }
 
 module end;
+syntax error

and run skc --check sql/src/Main.sk then I get the expected

File "sql/src/Main.sk", line 766, characters 1-6:
Expected declaration.
764 |
765 | module end;
766 | syntax error
    | ^^^^^^

but if I run (cd sql && skargo build --release) then I get the following

    Fresh std v0.1.0 [build-script-std] (/skdb/prelude)
    Fresh std v0.1.0 [lib] (/skdb/prelude)
    Fresh skbuild v0.1.0 [lib] (/skdb/skbuild)
    Fresh skdate v0.1.0 [build-script-skdate] (/skdb/skdate)
    Fresh skdate v0.1.0 [lib] (/skdb/skdate)
    Fresh cli v0.1.0 [lib] (/skdb/cli)
    Fresh skjson v0.1.0 [lib] (/skdb/skjson)
    Fresh sqlparser v0.1.0 [lib] (/skdb/sqlparser)
    Compiling skdb v0.1.0 [lib] (/skdb/sql)
error command exited with non-zero status

Caused by:
  process did not exit successfully: `skc` (exit status: 45)
  --- stdout

  --- stderr
  ERROR (open failed): No such file or directory
  Could not open file: skdb:/skdb/sql/src/Main.sk

Can of course be worked around by running skc --check when needed, but curious if others are seeing the same behaviour.

Questions: Mirroring "sophistication", Concurrent writes, React Native etc...

Hello and thank you for this exciting software.
I ve stumbled upon SKDB few days back and I cannot stop thinking about it.
Many questions popping up in my head. Let me try and spell them down here:

Take the code below:

await skdb.mirror(
    {
      table: "chess_moves",
      filterExpr: "game_state = 'started'"
    }
  );

1- What are the limits of what filterExpr can "express"? Can one use any SQL expression to pick what rows and columns to sync from the chess_moves table? Can the filtering part of the expression use multiple different tables (other than chess_moves ) at once (exp using joins ... etc)?

3- I am guessing SKDB is uses WASM & the upcoming OPFS to work on the browser (and not Persist through IndexedDB ...etc). Also, since it runs in the user's device, it needs not catter to concurent writes (same as SQLite). However, SKDB runs also on the Server. How does that work? Is it the exact same version on both Browser and Server? Does that mean SKDB on the server is not meant to handle large ammounts of concurent writes? any benchmarks? BTW: Electric-SQL for instance uses PostgreSQL on the server!

4- Since it relies on WASM, why not having it work in React Native? Maybe it's already done or planned?

5- How about the software licenses. In Browser, Server and React NAtive? Any possibly other options you have in the pipeline.

I would appreciate much if you could give me some feedback on the above.
Best wishes!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.