jasperfx/marten

.NET Transactional Document DB and Event Store on PostgreSQL

marten's Issues

DateTime support in Linq

Use LATERAL jsonb_to_record(blah, blah, blah) to optimize shallow Linq queries?

See this post. The naive way we're querying today involves runtime casts and can't possibly be efficient.

It's sounding like a big win, but I'm not sure how helpful it would be for deeply nested queries. Not sure how we'd exactly go about switching between using jsonb_to_record/LATERAL and using CASTS instead.

More research necessary.

Command line management tool?

I think we need some kind of facility to generate the database schema objects for document types. Think about this one more. Is it a console tool on top of the library support? Migrations?

Possible features would be:

1. Dump the full DDL for the entire schema
2. Generate patches for any places where the schema is different than the Marten configuration
3. Bulk data loading?
4. Wipe out Marten schema objects?
5. Load the Javascript projections?

Compiled Queries

So I'm kind of freaked out now that I've seen how many intermediate steps there are between a Linq expression and actually getting the SQL command out of that.

For performance, what if we had a way to express a commonly used query as a Linq statement once and get the SQL string w/ the right parameters built upfront so you can avoid that parsing hit over and over again.

I've got some vague ideas about doing this, but nothing definite yet

Asynchronous versions of everything in the IDocumentSession API

More work on this later, but assuming that we're going to the OWIN'ified async all the things, Node.js style event loop kind of world for maximum scalability

Support decimal type in linq expressions

Document Projections

Just a placeholder for now. Have some kind of model that allows you to do transforms to read side views in the database. Might finally get into PLV8 code inside of postgres. I'm thinking that we could do this in lieu of supporting truly crazy LINQ queries of the real model.

Support partial document updates?

I used this feature a tiny bit with Raven. It's actually possible w/ Postgresql 9.5 to do partial updates to JSON documents, but I think you'd have to express the change as JSON. Not sure it'd be worth doing a strong typed facade over that, but could be wrong. Maybe use .Net's Dynamic type

Full control over how the JsonNetSerializer is built and configured internally

Json.Net options. Aw, Json.Net, how I've missed all your binding conflicts. (Not).

DocumentSession is a Blob class

Not this minute, but pretty soon get into DocumentSession and start pulling stuff out to smaller, cohesive classes

Define indexes by document type in code

This is going to need a lot more thought. What I'd like is a way to define Marten index's in code for common queries that may:

1. Create extra fields and matching db indexes in the document tables to duplicate information in the son payloads specifically to make querying more performant
2. Somehow directs the Linq query support to use these extra fields
3. Adds something to the DB to duplicate that data upon inserts or updates

DocumentSession.Query(string) should check for the presence of a "Select" statement

Today, it's expecting a where or maybe a where + order by clause and partially building the sql by itself. For the case of needing to exactly fine tune the sql used, support taking in the whole query as well.

Support Select() from the MartenQueryable() ????

If you do session.Query().Where(blah blah blah).Select(x => x.Something) it goes pear shaped. You have to do an intermediate ToArray() or ToList() in between.

Think about this and see if anyone really wants projections directly over the top of the document tables. Might temporarily have it fetch and deserialize, then do the projection in memory. Because you know that no one would kill their perf by using that naively.

Documents that have a string id field

Paging in the querying. Do something similar to RavenDb's

Will need this sooner or later.

ProductionDocumentSchema

A production time version of IDocumentSchema that blows up when an unknown document type is encountered. Can't be building new database tables in production in most likelihood anyway.

Eliminate the temporary IDocument crutch

Worry about this later. Will be smarter later about determining how to get the id type and accessor/funq

IEnumerable.Contains() Linq support for primitive types

Names.Contains("foo")

Support Count() in the MartenQueryable()

Implement DocumentSession's IQueryExecutor.ExecuteScalar<T>(QueryModel queryModel) method

Use duplicated fields as searchable inside the Linq SQL generation

Documents that have long or int id's

Load documents by an enumerable of id's

Flesh out DocumentSession's Load() methods.

• For a string id
• For a numeric id
• For a Guid id

Consider More Proven Document Stores

Don't take this as knocking postgres, which I like in other contexts...but it seems that using something slightly less new (and in use in more projects or companies) would be worthwhile. There's always redis.

Optionally use Sequence for numeric Id property (and/or high/low implementation)

Implement an attribute to identify a Id property as [Identity] with one or more implementations:

• fetch next number from sequence when DocumentSession.Save(..) is called and assign to Id property
• use high-low algorithm at domain level to assign next number from range or fetch next range from sequence. Set Id property when DocumentSession.Save(..) is called

Flesh out the DocumentCleaner

Testing utility to clean out the database between tests

Support Last() and LastOrDefault() in Linq support

Laters...

Support doubles in linq expressions

Support more .Net primitive types

Need to add tests and postgres casts in MartenExpressionParser for:

• bit [ (n) ] --->Byte[]
• bit varying [ (n) ] --->Byte
• bytea --->Byte[]
• interval [ (p) ] --->TimeSpan

Optimize the Document.SaveChanges()

Maybe use data adapters in ADO.Net to submit all the changes in one batch? May not be that crazy. Another alternative would be to submit a batch of JSON and have a plv8 sproc in the database do the work of pushing the data to the right places.

Boolean types in Linq support

Test the Linq support with multiple Where() clauses

Support Single() or First() in the Linq support

I think this mostly means implementing DocumentSession's IQueryExecutor.ExecuteSingle<T>() method

• Single hit only one
• Single hit more than one
• Single miss
• First hit
• First miss
• SingleOrDefault hit only one
• SingleOrDefault hit more than one
• SingleOrDefault miss
• FirstOrDefault hit
• FirstOrDefault miss

Bulk loading support

Use the Binary COPY feature of npgsql.

Support string StartsWith() and EndsWith() searches in Linq support

We already support Contains, so now what about:

• StartsWith
• EndsWith

Searching on Nullable Types

Later.

Linq support to search within a child array

Seeing plenty of Any() queries on a child collection in our code at work.

Use enumeration values as Linq values

Might be nothing but a test to make sure that it works

Support ordering in the Linq query able

Helpers to retrieve the raw JSON

For web app scenarios, when all you wanna do is spit the raw json directly over the outgoing HTTP response, just get the raw json data as a string and spit it out over the wire.

Synchronous and Async. Might investigate some kind of lightweight way to strip out the type information that comes in from Json.Net or whatever the serializer is gonna be.

Why "json" and not "jsonb"

I'm curious why documents are being stored with the "json" data type, and not the "jsonb" datatype?

https://github.com/JasperFx/Marten/blob/02e104e6b8d567e3dd86c4bf5d8112975ee7bb20/src/Marten/Generation/Templates/DocumentTable.txt#L4

http://stackoverflow.com/a/22910602/627205

Store Hierarchy / subclassed documents in the same table

By default store every class in it's own table, except when on one of the parent classes an attribute [DocumentRoot] is declared.
or always stored subclassed documents into the root class..

DevelopmentDocumentSchema should not overwrite existing tables

Watch this. It does a drop and create table combo regardless today, but that's not going to work in real development.

Full text searching support on the documents?

More research on this one.

Support Take() and Skip() in the Linq support

Support searching for "IS NULL" in linq queries

Batched requests, deferred queries, futures

Network round trips are evil!

What if we had some way to send several requests up to the db at one time? More research necessary.

Linq support for nested properties

Haven't tried something like Where(x => x.User.Name == "foo"). Look at MartenExpressionParser.JsonLocator()

Nullable<T> type support in Linq provider

Optimistic locking with version tracking on documents

Test Queryable.Any()

Just need a test for it maybe.