A DSL for query and benchmark Time Series Database. This a brand new version. Original Reika can be found here
at15 / reika Goto Github PK
View Code? Open in Web Editor NEW[WIP] A DSL for query and benchmark TSDB
License: MIT License
[WIP] A DSL for query and benchmark TSDB
License: MIT License
A DSL for query and benchmark Time Series Database. This a brand new version. Original Reika can be found here
Before introducing list and records, it's better to first have variable up and running, which also requires the type checker (infer?) #24 working as well, if we have just pure number .... everything can be done at compile time ...
Syntax
a = 1
or int a = 1
, we should be able to infer some type
let
style, allow (not forced) to specify the type, but will infer right hand side and compare, error if what the user want on left side is not what he write on right sideSemantics
Untyped visitor
Same as the python one #51, the main issue for Go is its runtime lack supports of visitor, there is several PR for that antlr/antlr4#1841 (should be the latest on, updated on 2017/10/03)
One thing to note is, we may not need generated BaseVisitor from ANTLR, most DSL in Go are using handwritten parser and have a walk
function, we should be able to find a way to use the context w/o the base
literal
: BOOL # BoolLiteral
| INT # IntLiteral
| DOUBLE # DoubleLiteral
| STRING # StringLiteral
;
// TODO: [0:123]
list
: '[' literal (' ' literal)* ']'
;
term
: literal # LiteralTerm
| list # ListTerm
// FIXME: the record rule is wrong
| '{' ID ':' literal (',' ID ':' literal)* '}' # RecordTerm // TODO: allow record to have non-literal types? nested record etc.
| '(' term ')' # BracketsTerm
;
java -cp ./third_party/antlr-4.7-complete.jar:./build/libs/reika-0.0.1-SNAPSHOT.jar org.antlr.v4.gui.TestRig me.at15.reika.parser.Reika prog -gui
{a: 123, b: 124};
line 1:3 extraneous input ' ' expecting {BOOL, INT, DOUBLE, STRING}
line 1:8 extraneous input ' ' expecting ID
line 1:11 extraneous input ' ' expecting {BOOL, INT, DOUBLE, STRING}
It is used when traverse ANTLR generated parse tree, also mentioned in ANTLR3 book P113 External Tree Visitor
accept
method, although identical, need to be declared in every subclass of nodeWe need negative number, but if we put the sign into the token INT
and DOUBLE
, it would create trouble for 1 - 2
, it is no longer INT MINUS INT, but INT INT.
// what we want
irk> -1
-1 : float
irk > 1 - 2
-1 : float
Don't bother trying to handle them in the lexer, handle uniary-minus in the parser instead
The results of any tests run via the JUnit Platform Gradle plugin will not be included in the standard test report generated by Gradle; however, the test results can typically be aggregated by your CI server software. See the reportsDir property of the plugin
reportsDir file('build/test-results/junit-platform') // this is the default
Other code quality tools
Current spec is 0.1
=
?:=
?+
-
and postfix unary ++
--
Optional
;
, it is mentioned in ANTLR 4 P216 Fun with Python new linesI was trying take notes of the dotty video #35 and download games while on windows, but failed to checkout when clone
fatal: cannot create directory at 'doc/talk/cmps253-Re:Reika': Invalid argument
From at15/mini-impl#6
Note TAPL use nameless representation for function parameter (binding), we chose to use name, it is easier to implement but error pron, more discussion can be found in the original issue
From #6 , we got trouble when parsing lambda-calculus when function body is a function application
(lambda x. x) (lambda x. x x);
currently we have
term: ID # Var
| 'lambda' ID '.' term # Abs
| term term # App
| '(' term ')' # Brackets
;
the function body is x x
, but we didn't parse it correctly
Related #18
Actually I didn't really figured out parser lexer problem when building old reika, because it was so simple and I was mainly following grammar on the book that I didn't meet any problem
Current AST node class has several problems
type
property in the abstract class, but we also treat user specified type as a node, and that type is stored in property v
, which may cause mistake when writing visitor, further more, we may not need to store this as a AST nodetype.Checker
which generates new tree instead of updating existing one, which waste resource since we never use the old oneparser.prog
is not correctRelated #18
Since the syntax for all the binary operations are the same, it's a waste to specify them differently
e : e '*' e # Mul
| e '-' e # Minus
| INT
;
Currently, I have two way to solve this
e : e BINARY_OP e # BinaryOp;
BINARY_OP : '*' | '/' | '+' | '-';
which caused two troubles
-
is for both negative and minus #18in ANTLR4 P264 15.3 Parser Rules
you can reuse the same label on multiple alternatives to indicate that the parse-tree walker should trigger the same event for those alternatives
e : e '*' e # BinaryOp
| e '+' e # BinaryOp
| INT
;
but you can't get token defined in parser rule, the context for visitor does not have it
It is possible to draw graph in Java using its own UI library, but it's way better if we can generate graph in browser. We might want to split the runtime to separate project in the future, so using the compiled java file don't need to bring the compiler around (along with antl's icu deps #39 )
It's better to introduce type checker before we start introducing operator overload for +
, -
Ref
Since a main reason for using ANTR is to generate parser in different language, it's time to start using this feature with the simpler dialect RCL #49
Although for implementing the checker in TAPL we don't need a parser, but the doc in old Reika is not a step by step guide to tell how to use ANTLR from scratch, and it's for Java only while we are planning to support multiple language in the newer version
From at15/mini-impl#6 we start our implementation of TAPL's type checker from simple untyped arithmetic expressions (a.k.a calculator)
If we mix all the binary operators (+, -, *, /) together, we lost the precedence, and user need to explicitly add ()
to have multiply executed before plus. The reason we put them together is we want to avoid duplicated code in visitor when building AST, because the operator is the only difference
Currently we have (this is wrong)
term
: literal # TmLiteral
| '-' term # TmNegative
| term BINARY_OP term # TmBinaryOp
// | list # TmList
// | record # TmRecord
| '(' term ')' # TmBrackets
;
BINARY_OP
: '+'
| '-'
| '*'
| '/'
;
for 1 + 2 * 3
we have
*
+ 3
1 2
but we should have
+
1 *
2 3
TODO
Due to need from xephonhq/awesome-time-series-database#38 , we want a configuration language that we can point out which line in the configuration file is wrong, i.e. get an AST w/ position instead of clean object (if we want)
In the long term, RCL should be merged into Reika compiler
Ref
ANTLR works best with Java, old Reika tsdb-proxy-java/ql is using Gradle instead of maven, we will continue using Gradle, but I forgot how to use Gradle entirely, this issue is used to track the progress of having a gradle project using ANTLR
gradle build
, use doLast
solve thisWhen reading the book Language Implementation Patterns (ANTLR3), found a example of list, which seems to have indirect left recursion. But as I remember ANTLR4 handles LR better but still can't handle indirect left recursion (it is mentioned in the ANTLR4 book somewhere)
From ANTLR3 book P26, example text is [a, b, c]
, [a, [b, c], d]
grammar NestedNameList;
list : '[' elements ']' ;
elements : element (',' element)* ;
element: NAME | list ;
NAME : [a-zA-Z]+;
From at15/mini-impl#6
Before implement subtyping, let's first finish typing, this match to simplebool and Chapter 10 in TAPL
At first I defined abs
as type Node
and ASTBuilder works, but after switching to Abs
, it broke
public static class App extends Node {
public final Abs abs;
public final Node param;
guess it is due to (lambda x. x) (lambda x. x x);
, in (lambda x. x x)
there is a application and x
is a var instead of a abs, it can become abs if some one bind it, the var here can be anything ... and (lambda x. x x)
seems never terminate?
java.lang.ClassCastException: me.at15.tapl.untyped.UntypedAst$Var cannot be cast to me.at15.tapl.untyped.UntypedAst$Abs
We need to know how long we spend on each compile phase, also it would be good if we could count time in our test and REPL, after all ... these data are kind of like time series .... scalac's could be an example
Though not sure what our type system gonna be (3 days before the presentation)
let x = 1; let y = x + 1
1 + [1, 2, 3]
can workAlthough this repo is almost dead ... just use issues to keep notes
Graal can be used to implement language that runs on jvm (it has many cool features that I don't even understand the name of them ...)
&
-> &
homogenous
-> homogeneous
https://github.com/at15/papers-i-read/issues/89
Weld is a language and runtime for improving the performance of data-intensive applications. It optimizes across libraries and functions by expressing the core computations in libraries using a common intermediate representation, and optimizing across each framework
I think I should not rely on complete ANTLR jar, and only use the runtime jar
Similar to at15/snowbot#20 due on tomorrow ... end of finals week, Friday Dec 15. 6-20 pages ... so again ... it's report driven development
Introduction
Body
Conclusion
When using https://github.com/google/error-prone found the following warning, now I manually add it, but it need to be add again after every change of grammar, don't know if anyone mention it or is already fixed in upstream
@Override
public boolean sempred(RuleContext _localctx, int ruleIndex, int predIndex) {
I changed it manually for now ...
Currently we have two folder for test scripts, arith
and arith_typed
which are quite misleading, arith_typed
is actually the one with let binding. Since we are adding list a more ideal folder structure should be
primitve
- int_double_float_only
- negative
- binary_op
- let (previous arith_typed)
list
- binary_op_overload (1 + [1, 2, 3])
We need standard library if we want to more than just calculator, it can be used in both REPL (interpreted) and compiled (actually transpile, we convert to Java file and let javac do the hard work, and run it with our runtime in class path)
Currently Reika lies in https://github.com/xephonhq/tsdb-proxy-java , it should be a project of its own and imported in tsdb-proxy-java(go) as a library for query and benchmark. Also we can't just copy the code or use git subtree, clean up need to be done and existing git history is not that useful.
Tooling
Code
Extra
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.