Make your ActiveRecords stored and searchable on Neo4j graph database, in order to make fast graph queries that MySQL would crawl while doing them.
Neoid to Neo4j is like Sunspot to Solr. You get the benefits of Neo4j speed while keeping your schema on your plain old RDBMS.
Neoid doesn't require JRuby. It's based on the great Neography gem which uses Neo4j's REST API.
Neoid offers querying Neo4j for IDs of objects and then fetch them from your RDBMS, or storing all desired data on Neo4j.
Add to your Gemfile and run the bundle
command to install it.
gem 'neoid'
Requires Ruby 1.9.2 or later.
In an initializer, such as config/initializers/01_neo4j.rb
:
ENV["NEO4J_URL"] ||= "http://localhost:7474"
neo4j_uri = URI.parse(ENV["NEO4J_URL"])
$neo = Neography::Rest.new(neo4j_uri.to_s)
Neography::Config.tap do |c|
c.server = neo4j_uri.host
c.port = neo4j_uri.port
if neo4j_uri.user && neo4j_uri.password
c.authentication = 'basic'
c.username = neo4j_uri.user
c.password = neo4j_uri.password
end
end
Neoid.db = $neo
01_
in the file name is in order to get this file loaded first, before the models (files are loaded alphabetically).
If you have a better idea (I bet you do!) please let me know.
For nodes, first include the Neoid::Node
module in your model:
class User < ActiveRecord::Base
include Neoid::Node
end
This will help to create a corresponding node on Neo4j when a user is created, delete it when a user is destroyed, and update it if needed.
Then, you can customize what fields will be saved on the node in Neo4j, by implementing to_neo
method:
class User < ActiveRecord::Base
include Neoid::Node
def to_neo
{
slug: slug,
display_name: display_name
}
end
end
You can use neo_properties_to_hash
, a helper method to make things shorter:
def to_neo
neo_properties_to_hash(%w(slug display_name))
end
Let's assume that a User
can Like
Movie
s:
# user.rb
class User < ActiveRecord::Base
include Neoid::Node
has_many :likes
has_many :movies, through: :likes
def to_neo
neo_properties_to_hash(%w(slug display_name))
end
end
# movie.rb
class Movie < ActiveRecord::Base
include Neoid::Node
has_many :likes
has_many :users, through: :likes
def to_neo
neo_properties_to_hash(%w(slug name))
end
end
# like.rb
class Like < ActiveRecord::Base
belongs_to :user
belongs_to :movie
end
Now let's make the Like
model a Neoid, by including the Neoid::Relationship
module, and define the relationship (start & end nodes and relationship type) options with neoidable
method:
class Like < ActiveRecord::Base
belongs_to :user
belongs_to :movie
include Neoid::Relationship
neoidable start_node: :user, end_node: :movie, type: :likes
end
Neoid adds neo_node
and neo_relationships
to nodes and relationships, respectively.
So you could do:
user = User.create!(display_name: "elado")
user.movies << Movie.create("Memento")
user.movies << Movie.create("Inception")
user.neo_node # => #<Neography::Node…>
user.neo_node.display_name # => "elado"
rel = user.likes.first.neo_relationship
rel.start_node # user.neo_node
rel.end_node # user.movies.first.neo_node
rel.rel_type # 'likes'
You can query with all Neography's API: traverse
, execute_query
for Cypher, and execute_script
for Gremlin.
These examples query Neo4j using Gremlin for IDs of objects, and then fetches them from ActiveRecord with an in
query.
Of course, you can store using the to_neo
all the data you need in Neo4j and avoid querying ActiveRecord.
Most popular categories
gremlin_query = <<-GREMLIN
m = [:]
g.v(0)
.out('movies_subref').out
.inE('likes')
.inV
.groupCount(m).iterate()
m.sort{-it.value}.collect{it.key.ar_id}
GREMLIN
movie_ids = Neoid.db.execute_script(gremlin_query)
Movie.where(id: movie_ids)
Assuming we have another Friendship
model which is a relationship with start/end nodes of user
and type of friends
,
Movies of user friends that the user doesn't have
user = User.find(1)
gremlin_query = <<-GREMLIN
u = g.idx('users_index')[[ar_id:'#{user.id}']][0].toList()[0]
movies = []
u
.out('likes').aggregate(movies).back(2)
.out('friends').out('likes')
.dedup
.except(movies).collect{it.ar_id}
GREMLIN
movie_ids = Neoid.db.execute_script(gremlin_query)
Movie.where(id: movie_ids)
[0].toList()[0]
is in order to get a pipeline object which we can actually query on.
Whenever the neo_node
on nodes or neo_relationship
on relationships is called, Neoid checks if there's a corresponding node/relationship in Neo4j. If not, it does the following:
- Ensures there's a sub reference node (read here about sub reference nodes)
- Creates a node based on the ActiveRecord, with the
id
attribute and all other attributes fromto_neo
- Creates a relationship between the sub reference node and the newly created node
- Adds the ActiveRecord
id
to a node index, pointing to the Neo4j node id, for fast lookup in the future
Then, when it needs to find it again, it just seeks the node index with that ActiveRecord id for its neo node id.
Like Nodes, it uses an index (relationship index) to look up a relationship by ActiveRecord id
- With the options passed in the
neoidable
, it fetches thestart_node
andend_node
- Then, it calls
neo_node
on both, in order to create the Neo4j nodes if they're not created yet, and creates the relationship with the type from the options. - Add the relationship to the relationship index.
Neoid tests run on a regular Neo4j database, on port 7574. You probably want to have it running on a different instance than your development one.
In order to do that:
Copy the Neo4j folder to a different location,
or
symlink bin
, lib
, plugins
, system
, copy conf
and create an empty data
folder.
Then, edit conf/neo4j-server.properties
and set the port (org.neo4j.server.webserver.port
) from 7474 to 7574 and run the server with bin/neo4j start
Download, install and configure neo4j-clean-remote-db-addon. For the test database, leave the default secret-key
key.
Please create a new issue if you run into any bugs. Contribute patches via pull requests. Write tests and make sure all tests pass.
after_update
to update a node/relationship.- Allow to disable sub reference nodes through options
- Execute queries/scripts from model and not Neography (e.g.
Movie.neo_gremlin(gremlin_query)
with query that outputs IDs, returns a list ofMovie
s) - Rake task to index all nodes and relatiohsips in Neo4j
Developed by @elado