schemasafe/troy

Type-safe and Schema-safe Scala wrapper for Cassandra driver

schemasafe

Singleton types based queries.

troy's Issues

Insert Statement AST + Parser

https://cassandra.apache.org/doc/latest/cql/dml.html#insert

Setup CI/CD pipeline

Check free github integrations to run testcases, and ultimately publish jars to bintray

Parametrise Schema location

Currently Schema file location is hardcoded is "resources" folder of the Troy itself.

1. This should be parametrised. Allowing future plans, like schema versioning.
2. Allow Troy to read resources folder of user's project at Compile time.

Basic Select statement validation

Cassandra doesn't allow arbitrary queries to be executed.

Troy should show compile time error, if a query is know to fails at runtime
For example: quering on a field that is not part of Primary key, nor has an index

Depends On: #35

Parsing Literals with escaped quote fails

Any CQL statement with a string containing an escaped quote

Example:

SELECT entry_title, content FROM posts WHERE blog_title='John''s Blog'

Test Coverage

More testcases

There have been a lot of testcases commented out during refactoring. Needs to be updated

Update Troy-Meta to support new Schema Engine

TroyMeta should use the V monad introduced.

Insert Statement AST + Parser

https://cassandra.apache.org/doc/latest/cql/dml.html#insert

Select without where

There is a commented test cases contains a simple select statement, without where clause at all.

The parser and model already handles. Must be either bug in parser, or macro.

ALTER TABLE

https://cassandra.apache.org/doc/latest/cql/ddl.html#alter-table

Backward compatibility checks for Queries

withSchema block should accept an optional minimumVersion parameter. If specified, troy should validate the query against previous schema versions as well.

Schema Migrations

Schema migrations in Cassandra can be tricky due to limitations like

• Renaming columns is only limited to parts of primary key
• Changing column types must be to a compatible type (for example text to blob)
• Removing a column, then adding another with the same name but different type fails (Cassandra remembers previous schemas as well)

Thus, it makes sense Troy as well should be aware of the previous versions of the schema, allowing us to provide a couple of awesome features.

Schema Syntax

Schema to be written in multiple files, in a versioned manner (001.cql, 002.cql, etc).
Each file would contain evolution style schema modification, like ALTER TABLE, as well as dropping, and creating new tables.

For convenience, we can add an SBT command to show the current schema. Or even the schema at any given point in time.

Compatibility checks

By adding param like: withSchema("2+") { ... }, troy can check that code inside this block would work against multiple schema versions. Usefull for rolling updates where servers might talk to old schema, untill it gets migrated.

Migration Framework

Sometimes ALTER TABLE statements are not enough for doing migrations. Usually, tools like Spark can be used to go through all data in Cassandra and modify it.
Troy offer a module/framework to assist writing and running such scripts.

Fix parse float

Support User Data Types

Support MAP type

Parse Create Index

Troy should be able to parse CREATE INDEX statement in "schema.cql" files.

Note: Schema engine should also handle the statement, along with CREATE KEYSPACE, and CREATE TABLE. However, it is not required to store/react to it, (i.e Schema engine won't validate that queries has the correct indexes, not in this story)

Support LIKE operation for SASI indexed fields

Update Statement AST + Parser

https://cassandra.apache.org/doc/latest/cql/dml.html#update

Where clause's restrictions operands should be allowed to have different types

Currently any restriction operators used at Where clause, assumes the operands have the same type. Which works fine for "=", "<=", ">", etc ...
But this won't work for "contains", "in", ...

Example:

val q = withSchema { (tag: String) =>
  cql"SELECT id, tags FROM test.post_details where tags CONTAINS $tag;"
}

TODO: Define first which operators to be implemented in this story, and which to be left for later

Select *

The problem with select * is that Troy depends on the index of columns in select clause.
Example: select id, name, age allows Troy to know the types will be UUID, String, Int, and can parse the row to any matching case class.

To be able to Select * we either have to depend on name of fields rather than index, which requires a way to map Case classes' member names, to column names, which can be different. Also this would require reflection features of Macro, which is probably going to be deprecated in Dotty.

Another approach is to mimic the default column ordering returned by Cassandra, which seems to be consistent. However, this can give users of Troy hard time with compiling issues because Schema doesn't match the case class.

Third approach, is not supporting Select * with .as(...). Which is pretty much means don't support this feature at all, and show meaningful error message in the macro.

Testcases to ensure/show ability to setConsistency level on Statement

Apply alter table rename

Better support for Collections of Primitives

Due to limitation of Java generics, there is no Codec[int], only Codec[java.lang.Integer].
But also Datastax's driver won't let you define one from Scala (raises exception at Constructor).

Since, Scala compiler won't pick an implicit of M[java.lang.Integer] as M[Int]. This affected Troy implicit Codecs feature. As a solution, a hack was introduced for List and Set, to have a wrapper TroyCodec, that maps on every item on the collection to convert it from the Java Wrapper to the Scala Primitive.

But for Map, and Tuples, this hack would require order of magnitude more boilerplate. So currently they only support Wrapper classes, like Map[Integer, String]

See #8 and https://github.com/tabdulradi/troy/pull/17/files

Support "limit" clause in "Select" statements

Apply ALTER TABLE

Depends on #73

Automatic publish for snapshots by Travis

Continuation for #26
Currently we have Travis setup, and Bintray setup.

Bintray only accepts non-snapshots, so we need

• OSS.JFrog account
• Modify SBT config to switch to JFrog if isSnapshot (which is provided by Sbt)
• Add encrypted keys to travis.yaml to be done automatically

Update Select statement to CQL 3.4.3

New CQL syntax slightly different from previous implementation https://cassandra.apache.org/doc/latest/cql/index.html

Shouldn't affect any tests in Macro, only Parser, AST, and maybe Schema engine

Support Delete Statement

Depends on #33

"Alter Column Type" compatibility checks

There is basic validation defined here 0a129ae

This needs to cover all cases.

Major Refactoring for AST and Parser

As project is getting bigger, and attracting collaborators, we need to split AST, and parser files into multiple files/packages to avoid merge hell.

Delete Statement AST + Parser

https://cassandra.apache.org/doc/latest/cql/dml.html#delete

Deep type inference for Bindmarkers in a Term

Troy analyses queries to figure the type of bind markers (i.e ? and :variables)
So far we do only shallow searching for those blind markers, if a statement has a term, we check if the whole term is a bindmarker, then we figure out its type.

But we might encounter a complex term, for example, a Map literal with one or more of the values are bind markers.

Parser bug related to UUID literals

DELETE FROM test.posts WHERE author_id = 01234567-0123-0123-0123-0123456789ab;

Fails to parse

;' expected but-' found

Fix non-exhaustive pattern matching

[warn]/troy/troy-schema/src/main/scala/troy/schema/ColumnOps.scala:13: match may not be exhaustive.
[warn] It would fail on the following inputs: In, NotEquals
[warn]     def operandType(operator: Operator): Option[DataType] = operator match {
[warn]                                                             ^
[warn]/troy/troy-schema/src/main/scala/troy/schema/Schema.scala:87: match may not be exhaustive.
[warn] It would fail on the following inputs: Simple(_, _, Constant(_)), Simple(_, _, FunctionCall(_, _)), Simple(_, _, ListLiteral(_)), Simple(_, _, MapLiteral(_)), Simple(_, _, Named(_)), Simple(_, _, SetLiteral(_)), Simple(_, _, TupleLiteral(_)), Simple(_, _, TypeHint(_, _)), Simple(_, _, UdtLiteral(_))
[warn]     Result.flattenSeq(where.relations.map {

Where clause Validation

Troy should be able to validate where clause.

Notes that where clause is common between Select, Update, and Delete. But with slightly different rules in each case.

There is already a placeholder for this code
https://github.com/cassandra-scala/troy/pull/65/files#diff-b73eab8cad58acc28fa6a7152da0b27d
#36

Better handling for Options

Whenever there is a codec from T to Cassandra type, there should be a codec from Option[T] to same Cassandra type.

Scala Meta Frontend

Spike: Types compatibility and Codecs

Versioning + Publish to Bintray

Support Update Statement

Depends on #47

Support SET type

Resolve codecs inside Macro

Instead of emitting code that accepts implicit TroyCodec instances,
resolve them inside the macro first, aborting if the codec is not found.

This allows us to

1. Customise error message, by hiding internal CassandraDataType.Type classes, and show the types as lowercase "text", "map <int, text>", as it would appear inside CQLsh
2. Validate queries against multiple versions of the Schema, for every version, we can make sure Codecs exists

Revisit Term AST & Parser

Fully Implement Cassanadra Term and Literals.
Update Create Index, and Create Keyspace to use them

• constant
• literal
  • collection literal
  • udt literal
  • tuple literal
• function call
• type hint (test only)
• bind marker

Allow Troy to access resources folder at compile time

Read versioned Schema

Support Insert statements

Depends on #49

Collection columns should not be restricted by '=' relation

cql"SELECT id, tags FROM test.post_details where tags = $tags;"

Currently compiles, but failing at execution, showing:

"Collection column 'tags' (set) cannot be restricted by a '=' relation"

This error should be shown at compile time instead

Release 0.3

• Bump version, and publish locally
• Upgrade Examples
• Release to bintray
• Bump version to SNAPSHOT again.

Support LIST type