b4fun/ku

Toolkit for collecting and exploring logs using pipelined query lanaguage and sqlite.

ku's Issues

cache query by session

Reuse cli session table

Ku itself sends logs to the ku cli session table on each run. For repeated run on the same database, we should reuse this session

schema conversion

grpc api server

More opreators support

• extend
• summarize

data query

Slow data import

When the source log file contains large amount of logs, current insertion might be slow. We should consider batch insert to the table.

kql to sql compiler

• function calls, UDF
• #15
• computed column & alias (project a = x + 12)

duration / time literals

for ago:

querying select * from ku_cli_26GgJgKcmoovUpimJHKWyY_raw where ts > ago(3h) limit 100
react_devtools_backend.js:4026 
        
       RpcError: query failed: unrecognized token: "3h"
    at grpc-web-transport.js:123:23

PRQL editor auto completion

In memory usage

We should allow user to start up the tool without saving logs to disk.

Handle duplicate column names

Bad case:

haha
| extend p = 1 + 2, lines = 3 + 2
| extend q = p + lines
| take 100

q1 as (with q0 as (select 1 + 2 as p, 3 + 2 as lines, * from ku_cli_mcpxw3gjzWAA7F5KnFa3F_haha) select p + lines as q, * from q0) select * from q1 limit 100

This will yield:

q,p,lines,ts,lines:1

Where lines:1 is generated by sqlite, to distinct duplicated column names (ref). We should remove this column from the output. However, we need the "original column" information from the C API to filter this out.

cross tables / sessions usage

• join tables under the same session
• join tables across different sessions

Container usage

We should publish as container version and allow using in a container