dcaoyuan/chana

Avro Data Store based on Akka

chana's Issues

relationship between schema-registry and chana

can Chana used as a schema-registry like thishttps://github.com/confluentinc/schema-registry?
what is the relationship,if i want to use chana as schema-registry,what else i need do?

Type inference for select result set

Type inference for select result set, that will be expressed as an Avro record schema.

Serializer for java.util.Map with avro's GenericContainer value

Akka will apply Java serializer on java.util.Map, which then will encounter not serializable on GenericContainer value (which is not extends Serializable)

sbt run warn

osx 10.10.4
jdk oracle 1.8.51

[email protected]:/Users/edwin/workspace  $ cd chana 
[email protected]:/Users/edwin/workspace/chana  git:(master*) $ sbt run
[info] Loading project definition from /Users/edwin/Documents/workfolder/workspace/chana/project
[info] Compiling 2 Scala sources to /Users/edwin/Documents/workfolder/workspace/chana/project/target/scala-2.10/sbt-0.13/classes...
[info] Set current project to chana (in build file:/Users/edwin/Documents/workfolder/workspace/chana/)
[info] Compiling 9 Scala sources and 8 Java sources to /Users/edwin/Documents/workfolder/workspace/chana/avpath/target/scala-2.11/classes...
[info] 'compiler-interface' not yet compiled for Scala 2.11.6. Compiling...
[info]   Compilation completed in 8.678 s
[warn] there were two feature warnings; re-run with -feature for details
[warn] one warning found
[warn] 未与 -source 1.6 一起设置引导类路径
[info] /Users/edwin/Documents/workfolder/workspace/chana/avpath/target/scala-2.11/src_managed/test/java/compiled_avro/chana/avro/test/Account.java: 某些输入文件使用了未经检查或不安全的操作。
[info] /Users/edwin/Documents/workfolder/workspace/chana/avpath/target/scala-2.11/src_managed/test/java/compiled_avro/chana/avro/test/Account.java: 有关详细信息, 请使用 -Xlint:unchecked 重新编译。
[info] Compiling 69 Scala sources and 2 Java sources to /Users/edwin/Documents/workfolder/workspace/chana/target/scala-2.11/classes...
[warn] /Users/edwin/Documents/workfolder/workspace/chana/src/main/scala/chana/collection/ArrayList.scala:297: method insert overrides concrete, non-deprecated symbol(s):    insert
[warn]   override def insert(n: Int, elems: A*) {
[warn]                ^
[warn] /Users/edwin/Documents/workfolder/workspace/chana/src/main/scala/chana/timeseries/TStampedMapBasedList.scala:162: method + overrides concrete, non-deprecated symbol(s):    +
[warn]   override def +(elem: A): this.type = {
[warn]                ^
[warn] /Users/edwin/Documents/workfolder/workspace/chana/src/main/scala/chana/timeseries/TStampedMapBasedList.scala:168: method insertOne overrides concrete, non-deprecated symbol(s):    insertOne
[warn]   override def insertOne(n: Int, elem: A) {
[warn]                ^
[warn] /Users/edwin/Documents/workfolder/workspace/chana/src/main/scala/chana/timeseries/TStampedMapBasedList.scala:173: method insertAll overrides concrete, non-deprecated symbol(s):    insertAll
[warn]   override def insertAll(n: Int, elems: scala.collection.Traversable[A]) {
[warn]                ^
[warn] /Users/edwin/Documents/workfolder/workspace/chana/src/main/scala/chana/timeseries/TStampedMapBasedList.scala:247: method companion overrides concrete, non-deprecated symbol(s):    companion, companion, companion, companion, companion, companion, companion, companion, companion, companion, companion, companion, companion, companion
[warn]   override def companion: GenericCompanion[TStampedMapBasedList] = throw new UnsupportedOperationException()
[warn]                ^
[warn] /Users/edwin/Documents/workfolder/workspace/chana/src/main/scala/chana/timeseries/TStampedMapBasedList.scala:256: method partition overrides concrete, non-deprecated symbol(s):    partition
[warn]   override def partition(p: A => Boolean): (TStampedMapBasedList[A], TStampedMapBasedList[A]) = {
[warn]                ^
[warn] /Users/edwin/Documents/workfolder/workspace/chana/src/main/scala/chana/timeseries/TStamps.scala:634: method partition overrides concrete, non-deprecated symbol(s):    partition
[warn]     override def partition(p: Long => Boolean): (TStampsOnOccurred, TStampsOnOccurred) = {
[warn]                  ^
[warn] /Users/edwin/Documents/workfolder/workspace/chana/src/main/scala/chana/timeseries/TStamps.scala:873: method partition overrides concrete, non-deprecated symbol(s):    partition
[warn]     override def partition(p: Long => Boolean): (TStampsOnCalendar, TStampsOnCalendar) = {
[warn]                  ^
[warn] there were two feature warnings; re-run with -feature for details
[warn] 9 warnings found
[warn] 未与 -source 1.6 一起设置引导类路径
[info] Running chana.Chana 
2015-08-04T12:46:09.658 INFO [lt-dispatcher-3] a.e.s.Slf4jLogger - Slf4jLogger started
2015-08-04T12:46:09.832 INFO [lt-dispatcher-4] Remoting - Starting remoting
2015-08-04T12:46:10.046 INFO [lt-dispatcher-4] Remoting - Remoting started; listening on addresses :[akka.tcp://[email protected]:2551]
2015-08-04T12:46:10.049 INFO [lt-dispatcher-4] Remoting - Remoting now listens on addresses: [akka.tcp://[email protected]:2551]
2015-08-04T12:46:10.064 INFO [lt-dispatcher-4] Cluster(akka://ChanaSystem) - Cluster Node [akka.tcp://[email protected]:2551] - Starting up...
2015-08-04T12:46:10.065 INFO [lt-dispatcher-4] Cluster(akka://ChanaSystem) - Cluster Node [akka.tcp://[email protected]:2551] - Using a dedicated scheduler for cluster. Default scheduler can be used if configured with 'akka.scheduler.tick-duration' [10 ms] <=  'akka.cluster.scheduler.tick-duration' [9 ms].
2015-08-04T12:46:10.162 INFO [lt-dispatcher-4] Cluster(akka://ChanaSystem) - Cluster Node [akka.tcp://[email protected]:2551] - Registered cluster JMX MBean [akka:type=Cluster]
2015-08-04T12:46:10.163 INFO [lt-dispatcher-4] Cluster(akka://ChanaSystem) - Cluster Node [akka.tcp://[email protected]:2551] - Started up successfully
2015-08-04T12:46:11.288 INFO [lt-dispatcher-2] s.c.s.HttpListener - Bound to /127.0.0.1:8080
2015-08-04T12:46:11.290 INFO [lt-dispatcher-3] a.r.RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [akka.io.Tcp$Bound] from Actor[akka://ChanaSystem/user/IO-HTTP/listener-0#861758154] to Actor[akka://ChanaSystem/deadLetters] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2015-08-04T12:46:15.216 INFO [t-dispatcher-19] Cluster(akka://ChanaSystem) - Cluster Node [akka.tcp://[email protected]:2551] - Node [akka.tcp://[email protected]:2551] is JOINING, roles [entity]
2015-08-04T12:46:16.195 INFO [lt-dispatcher-3] Cluster(akka://ChanaSystem) - Cluster Node [akka.tcp://[email protected]:2551] - Leader is moving node [akka.tcp://[email protected]:2551] to [Up]

Try to push incremental delta to JPQL selecting aggregator.

Need to carefully design incremental reduce algorithm

Should each node maintain a collection for each type of entity that contains all current actors of this type of entity?

Should each node maintain a collection for each type of entity that contains all current actors of this type of entity?

Windows case insensitive

Windows is case insensitive, so some problem occur after cloning the code base on window as the Chana.scala and chana.scala are in the same path.

JPQL example vs traditional SQL

Compared with traditional SQL query. It seems that the query has status and the query result can be monitored continuously and the result is updated once some monitored values change.

Is there any differences when doing the query optimization on the Query plan? What is the special aspect for JPQL on Chana?

typo in README.RST

in line 343
Del schame
should be
Del schema

ChanaClusterSpec became difficult to pass with akka-2.4.8

It seems that ChanaClusterSpec's 'awaitAssert for script's http_post.apply("http://localhost:8081/personinfo/put/2/age", "888"), which was triggered by name updated' becoming difficult to pass after upgraded to akka-2.4.8 620dba6

Push to Sonatype

Hey guys, cool project! :)

Could this get pushed to Sonatype?

Use compact xpath for binlog

Use Field.pos instead of Field.name to present xpath

Add APIs that execute multiple operations synced one time (batched operation)

Compilation issue

Getting errors building. Anything I can check on my local environment?

Brians-MBP:chana brian$ sbt publishLocal publishM2
[info] Loading global plugins from /Users/brian/.sbt/0.13/plugins
[info] Loading project definition from /Users/brian/dev/chana/project
[info] Set current project to chana (in build file:/Users/brian/dev/chana/)
[info] Avro compiler using stringType=String
[info] Formatting 14 Scala sources {file:/Users/brian/dev/chana/}chana-avro(compile) ...
[info] Compiling Avro schema /Users/brian/dev/chana/chana-avro/src/test/resources/avsc/Record.avsc
[info] Packaging /Users/brian/dev/chana/chana-avro/target/scala-2.11/chana-avro_2.11-0.2.0-SNAPSHOT-sources.jar ...
[info] Packaging /Users/brian/dev/chana/target/scala-2.11/chana_2.11-0.2.0-SNAPSHOT-sources.jar ...
[info] Updating {file:/Users/brian/dev/chana/}chana-avro...
[info] Formatting 81 Scala sources {file:/Users/brian/dev/chana/}chana(compile) ...
[info] Done packaging.
[info] Done packaging.
[info] Resolving org.apache.avro#avro;1.7.7 ...
[info] Rats! got a .rats file /Users/brian/dev/chana/chana-avro/src/main/scala/chana/xpath/rats/XPathGrammar.rats
[info] Rats! got a .rats file /Users/brian/dev/chana/src/main/scala/chana/jpql/rats/JPQLGrammar.rats
[info] Resolving org.apache.commons#commons-compress;1.4.1 ...
[info] Running Rats! on /Users/brian/dev/chana/chana-avro/src/main/scala/chana/xpath/rats/XPathGrammar.rats, input from /Users/brian/dev/chana/chana-avro/src/main/scala/chana/xpath/rats and /Users/brian/dev/chana/chana-avro/target/sbt-rats, output to /Users/brian/dev/chana/chana-avro/target/sbt-rats/chana/xpath/rats
[info] Running Rats! on /Users/brian/dev/chana/src/main/scala/chana/jpql/rats/JPQLGrammar.rats, input from /Users/brian/dev/chana/src/main/scala/chana/jpql/rats and /Users/brian/dev/chana/target/sbt-rats, output to /Users/brian/dev/chana/target/sbt-rats/chana/jpql/rats
[info] Resolving com.fasterxml.jackson.core#jackson-core;2.6.3 ...
xtc.tree.VisitorException: Internal error while visiting node xtc.parser.OrderedChoice@7b674838 with visitor xtc.parser.Resolver@92a033c
    at xtc.parser.Resolver.visit(Resolver.java:1918)
    at xtc.parser.Resolver.visit(Resolver.java:1943)
    at xtc.parser.Resolver.visit(Resolver.java:1908)
    at xtc.parser.Resolver.visit(Resolver.java:1903)
    at xtc.parser.Analyzer.process(Analyzer.java:1278)
    at xtc.parser.Resolver.check(Resolver.java:1034)
    at xtc.parser.Resolver.visit(Resolver.java:1739)
    at xtc.parser.Rats.process(Rats.java:292)
    at xtc.util.Tool.run(Tool.java:548)
    at SBTRatsPluginPatched$.runRatsImpl(SBTRatsPlugin.scala:437)
    at SBTRatsPluginPatched$.runGeneratorsImpl(SBTRatsPlugin.scala:214)
    at SBTRatsPluginPatched$$anonfun$runGenerators$1$$anonfun$12.apply(SBTRatsPlugin.scala:153)
    at SBTRatsPluginPatched$$anonfun$runGenerators$1$$anonfun$12.apply(SBTRatsPlugin.scala:152)
    at sbt.FileFunction$$anonfun$cached$1.apply(Tracked.scala:235)
    at sbt.FileFunction$$anonfun$cached$1.apply(Tracked.scala:235)
    at sbt.FileFunction$$anonfun$cached$2$$anonfun$apply$3$$anonfun$apply$4.apply(Tracked.scala:249)
    at sbt.FileFunction$$anonfun$cached$2$$anonfun$apply$3$$anonfun$apply$4.apply(Tracked.scala:245)
    at sbt.Difference.apply(Tracked.scala:224)
    at sbt.Difference.apply(Tracked.scala:206)
    at sbt.FileFunction$$anonfun$cached$2$$anonfun$apply$3.apply(Tracked.scala:245)
    at sbt.FileFunction$$anonfun$cached$2$$anonfun$apply$3.apply(Tracked.scala:244)
    at sbt.Difference.apply(Tracked.scala:224)
    at sbt.Difference.apply(Tracked.scala:200)
    at sbt.FileFunction$$anonfun$cached$2.apply(Tracked.scala:244)
    at sbt.FileFunction$$anonfun$cached$2.apply(Tracked.scala:242)
    at SBTRatsPluginPatched$$anonfun$runGenerators$1.apply(SBTRatsPlugin.scala:158)
    at SBTRatsPluginPatched$$anonfun$runGenerators$1.apply(SBTRatsPlugin.scala:144)
    at scala.Function6$$anonfun$tupled$1.apply(Function6.scala:35)
    at scala.Function6$$anonfun$tupled$1.apply(Function6.scala:34)
    at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
    at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
    at sbt.std.Transform$$anon$4.work(System.scala:63)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
    at sbt.Execute.work(Execute.scala:235)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
    at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
[info] Resolving org.scala-lang.modules#scala-xml_2.11;1.0.1 ...
[info] Done updating.
[info] Updating {file:/Users/brian/dev/chana/}chana...
[info] Wrote /Users/brian/dev/chana/chana-avro/target/scala-2.11/chana-avro_2.11-0.2.0-SNAPSHOT.pom
[info] Resolving org.scala-lang.modules#scala-xml_2.11;1.0.1 ...
[info] Done updating.
[info] :: delivering :: com.wandoulabs.chana#chana-avro_2.11;0.2.0-SNAPSHOT :: 0.2.0-SNAPSHOT :: integration :: Sun Mar 13 16:42:03 MDT 2016
[info]  delivering ivy file to /Users/brian/dev/chana/chana-avro/target/scala-2.11/ivy-0.2.0-SNAPSHOT.xml
[info] Wrote /Users/brian/dev/chana/target/scala-2.11/chana_2.11-0.2.0-SNAPSHOT.pom
[info] :: delivering :: com.wandoulabs.chana#chana_2.11;0.2.0-SNAPSHOT :: 0.2.0-SNAPSHOT :: integration :: Sun Mar 13 16:42:03 MDT 2016
[info]  delivering ivy file to /Users/brian/dev/chana/target/scala-2.11/ivy-0.2.0-SNAPSHOT.xml
java.lang.RuntimeException: Rats!, can't find generated file /Users/brian/dev/chana/target/sbt-rats/chana/jpql/rats/JPQLGrammar.java
    at scala.sys.package$.error(package.scala:27)
    at SBTRatsPluginPatched$.runRatsImpl(SBTRatsPlugin.scala:478)
    at SBTRatsPluginPatched$.runGeneratorsImpl(SBTRatsPlugin.scala:214)
    at SBTRatsPluginPatched$$anonfun$runGenerators$1$$anonfun$12.apply(SBTRatsPlugin.scala:153)
    at SBTRatsPluginPatched$$anonfun$runGenerators$1$$anonfun$12.apply(SBTRatsPlugin.scala:152)
    at sbt.FileFunction$$anonfun$cached$1.apply(Tracked.scala:235)
    at sbt.FileFunction$$anonfun$cached$1.apply(Tracked.scala:235)
    at sbt.FileFunction$$anonfun$cached$2$$anonfun$apply$3$$anonfun$apply$4.apply(Tracked.scala:249)
    at sbt.FileFunction$$anonfun$cached$2$$anonfun$apply$3$$anonfun$apply$4.apply(Tracked.scala:245)
    at sbt.Difference.apply(Tracked.scala:224)
    at sbt.Difference.apply(Tracked.scala:206)
    at sbt.FileFunction$$anonfun$cached$2$$anonfun$apply$3.apply(Tracked.scala:245)
    at sbt.FileFunction$$anonfun$cached$2$$anonfun$apply$3.apply(Tracked.scala:244)
    at sbt.Difference.apply(Tracked.scala:224)
    at sbt.Difference.apply(Tracked.scala:200)
    at sbt.FileFunction$$anonfun$cached$2.apply(Tracked.scala:244)
    at sbt.FileFunction$$anonfun$cached$2.apply(Tracked.scala:242)
    at SBTRatsPluginPatched$$anonfun$runGenerators$1.apply(SBTRatsPlugin.scala:158)
    at SBTRatsPluginPatched$$anonfun$runGenerators$1.apply(SBTRatsPlugin.scala:144)
    at scala.Function6$$anonfun$tupled$1.apply(Function6.scala:35)
    at scala.Function6$$anonfun$tupled$1.apply(Function6.scala:34)
    at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
    at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
    at sbt.std.Transform$$anon$4.work(System.scala:63)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
    at sbt.Execute.work(Execute.scala:235)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
    at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
java.lang.RuntimeException: Rats! failed
    at scala.sys.package$.error(package.scala:27)
    at SBTRatsPluginPatched$.runRatsImpl(SBTRatsPlugin.scala:446)
    at SBTRatsPluginPatched$.runGeneratorsImpl(SBTRatsPlugin.scala:214)
    at SBTRatsPluginPatched$$anonfun$runGenerators$1$$anonfun$12.apply(SBTRatsPlugin.scala:153)
    at SBTRatsPluginPatched$$anonfun$runGenerators$1$$anonfun$12.apply(SBTRatsPlugin.scala:152)
    at sbt.FileFunction$$anonfun$cached$1.apply(Tracked.scala:235)
    at sbt.FileFunction$$anonfun$cached$1.apply(Tracked.scala:235)
    at sbt.FileFunction$$anonfun$cached$2$$anonfun$apply$3$$anonfun$apply$4.apply(Tracked.scala:249)
    at sbt.FileFunction$$anonfun$cached$2$$anonfun$apply$3$$anonfun$apply$4.apply(Tracked.scala:245)
    at sbt.Difference.apply(Tracked.scala:224)
    at sbt.Difference.apply(Tracked.scala:206)
    at sbt.FileFunction$$anonfun$cached$2$$anonfun$apply$3.apply(Tracked.scala:245)
    at sbt.FileFunction$$anonfun$cached$2$$anonfun$apply$3.apply(Tracked.scala:244)
    at sbt.Difference.apply(Tracked.scala:224)
    at sbt.Difference.apply(Tracked.scala:200)
    at sbt.FileFunction$$anonfun$cached$2.apply(Tracked.scala:244)
    at sbt.FileFunction$$anonfun$cached$2.apply(Tracked.scala:242)
    at SBTRatsPluginPatched$$anonfun$runGenerators$1.apply(SBTRatsPlugin.scala:158)
    at SBTRatsPluginPatched$$anonfun$runGenerators$1.apply(SBTRatsPlugin.scala:144)
    at scala.Function6$$anonfun$tupled$1.apply(Function6.scala:35)
    at scala.Function6$$anonfun$tupled$1.apply(Function6.scala:34)
    at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
    at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
    at sbt.std.Transform$$anon$4.work(System.scala:63)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
    at sbt.Execute.work(Execute.scala:235)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
    at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
[error] (chana/compile:managedSources) Rats!, can't find generated file /Users/brian/dev/chana/target/sbt-rats/chana/jpql/rats/JPQLGrammar.java
[error] (chana-avro/compile:managedSources) Rats! failed
[error] Total time: 5 s, completed Mar 13, 2016 4:42:03 PM

Write a simplifier XPath implementation for binlog only

Write a simplifier XPath implementation for binlog only

typo in readme

should be get/put REST result?