okamiio / nomure Goto Github PK
View Code? Open in Web Editor NEWA proof-of-concept graph Database on top of FoundationDB
A proof-of-concept graph Database on top of FoundationDB
Sadly I'm don't know too much about string index/searching/match algorithms so I need to rely on another solutions for it, since is an important part of any database
Tantivy looks really promising since works like lucene so is plugable, I need to explore the futher posibilities
The FDB record layer implements a full text search indexing technique whichs I think I can use it natively, so I can have all in a single place
An issue describing short and long term features as well the roadmap
Property Values
Index
Node implementation
(relation.uid, "edge_name", node.uid) = node_relation_edge.uid
Edge implementation
Query support
where
{ year, month, day, seconds }
>
KeySelector.first_greater_than(key)
<
KeySelector.last_less_than(key)
==
get_by_function(property, value, function = Function.equal)
>=
KeySelector.first_greater_or_equal(key)
<=
KeySelector.last_less_or_equal(key)
and
or
pagination support (cursor based)
after
Transaction.get_range(after_cursor, end, limit)
before
Transaction.get_range(before_cursor, end, limit, reverse=True)
limit
first
Transaction.get_range(start, end, limit=first)
count
sorting
order_asc
index query
order_des
index query
reverse node
functions
min
atomic
max
atomic
Ztandart data compression
Compress long sized strings in order to save space, unnoticeable performance impact
Types check!
More tests
Network Protocol implementation
query optimization/planning, what most databases does.
Well as the title says, could be great having ecto support, the problem with Ecto is that it works with the SQL mindset which here does not make too much sense overall
Right now any data type index operation could face a duplicate key prefix which is quite inefficient
Let's take for example 2 record
user_a = %{name: "Artorias", deaths: 50}
user_b = %{name: "Chosen Undeath", deaths: 50}
the index result of the deaths
record would looks like
("user", "deaths", 50, user_a_node_uid) = ''
("user", "deaths", 50, user_b_node_uid) = ''
As you can see we have repeated the prefix "node_name", "deaths", 50
which is quite long by itself, containing bit_string data as node_name, property_name and property_value, so this proposal is to change it this way
("user", "deaths", 50, random_id) = [user_a_node_uid, user_b_node_uid]
we use the random_id
in order to "extend" the index, since FDB has a limitation on value size, so when it reach this size we split the index in order to add more uids into the result of the index
This way a single key prefix can contain a quite big amount of node uids
I don't think it will have any repercution, normally on any index you need to query them all in order to bring any query result
Right now Nomure is somewhat schema less, I do think adding types is a must for any database, tho is not an easy task and need a lot of coordination in order of making in runetime schema updates etc... in conjunction with FDB
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.