Giter Site home page Giter Site logo

levelgraph-n3's Introduction

LevelGraph-N3

Logo

Build Status Coverage Status Dependency Status Sauce Labs Tests

LevelGraph-N3 is a plugin for LevelGraph that adds the ability to store, fetch and process N3 and turtle files.

Install

Node.js

Adding support for N3 to LevelGraph is easy:

$ npm install level levelgraph levelgraph-n3 --save

Then in your code:

var level = require('level'),
    levelgraph = require('levelgraph'),
    levelgraphN3 = require('levelgraph-n3'),
    db = levelgraphN3(levelgraph(level('yourdb')));

Browser

If you use browserify you can use this package in a browser just as in node.js. Please also take a look at Browserify section in LevelGraph package

Usage

We assume in following examples that you created database as explained above!

var db = levelgraphN3(levelgraph(level("yourdb")));

Importing n3 files

In code:

var fs = require("fs");

var stream = fs.createReadStream("./triples.n3")
               .pipe(db.n3.putStream());

stream.on("finish", function() {
  console.log("Import completed");
});

Alternatively, you can run the import CLI tool by running npm install, then:

./import.js path/to/n3/file(s)

with the following optional flags:

-o or --output followed by the desired DB path. If not specified, path will be at ./db.

-q or --quiet will silence status updates during the import process. Otherwise, progress information is displayed.

File extensions must be .n3 or .nt. Additionally, there is glob support, so for example *.nt will import all the matching n-triple files.

Get and Put

Storing an N3 file in the database is extremey easy:

var turtle = "@prefix c: <http://example.org/cartoons#>.\n" +
             "c:Tom a c:Cat.\n" +
             "c:Jerry a c:Mouse;\n" +
             "        c:smarterThan c:Tom;\n" +
             "        c:place \"fantasy\".";

db.n3.put(turtle, function(err) {
  // do something after the triple is inserted
});

Retrieving it through pattern-matching is extremely easy:

db.n3.get({ subject: "http://example.org/cartoons#Tom" }, function(err, turtle) {
  // turtle is "<http://example.org/cartoons#Tom> a <http://example.org/cartoons#Cat> .\n";
});

It even support a Stream interface:

var stream = db.n3.getStream({ subject: "http://example.org/cartoons#Tom" });
stream.on("data", function(data) {
  // data is "<http://example.org/cartoons#Tom> a <http://example.org/cartoons#Cat> .\n";
});
stream.on("end", done);

Exporting NTriples from LevelGraph

LevelGraph-N3 allows to export ntriples from a LevelGraph database. LevelGraph-N3 augments the a standard search method with a { n3: ... } option that specifies the subject, predicate and object of the created triples. It follows the same structure of the { materialized: ... } option (see https://github.com/levelgraph/levelgraph#searches).

Here is an example:

db.search([{
  subject: db.v("s"),
  predicate: "http://example.org/cartoons#smarterThan",
  object: db.v("o")
}], {
  n3: {
    subject: db.v("o"),
    predicate: "http://example.org/cartoons#dumberThan",
    object: db.v("s")
  }
}, function(err, turtle) {
  // turtle is "<http://example.org/cartoons#Tom> <http://example.org/cartoons#dumberThan> <http://example.org/cartoons#Jerry> .\n"
});

It also supported by the searchStream method.

Changes

CHANGELOG.md including migration info for breaking changes

Contributing to LevelGraph-N3

  • Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet
  • Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it
  • Fork the project
  • Start a feature/bugfix branch
  • Commit and push until you are happy with your contribution
  • Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
  • Please try not to mess with the Makefile and package.json. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.

LICENSE - "MIT License"

Copyright (c) 2013-2015 Matteo Collina (http://matteocollina.com)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

levelgraph-n3's People

Contributors

bigbluehat avatar jmealo avatar mcollina avatar rubenverborgh avatar staticskies avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

levelgraph-n3's Issues

CLI for importing a dataset

I think a good tool we are missing is a CLI that loads an n3 file in LevelGraph-N3, ideally with a progress bar.

db.n3.put hangs

I'd like to use levelgraph for a project that involves receiving and storing triples for search, but it seems like the put call for levelgraph-n3 is hanging. The triples are indeed stored as a subsequent search shows, but there's no apparent call on the continuation putResult below - "running?" is never output, and, in this usage, the HTTP PUT that kicks it off spins until it times out because res.send is also never called.

server.put('/resource/new', function newResource(req, res, next) {
    db.n3.put(req.body.n3, function putResult(err) {
        console.log('running?');
        if (typeof err === "undefined" || err === null) {
            res.send(201, {'success': true});
        } else {
            console.log(err);
            res.send(500, {'success': false});
        }
    });
    return next();
});

Am I using it wrong?

  • node: 0.10.x, 0.12.4
  • levelgraph-n3: 0.5.0
  • levelgraph: 0.10.5
  • level: 1.0.0

This also kicks out a Node warning about possible leaking due to too many listeners:

(node) warning: possible EventEmitter memory leak detected. 16 _drain listeners added. Use emitter.setMaxListeners() to increase limit.
Trace
    at Stream.addListener (events.js:179:15)
    at Stream.once (events.js:204:8)
    at write (.../lib/node_modules/levelgraph/node_modules/level-write-stream/index.js:22:20)
    at Stream.EndStream.ended (.../lib/node_modules/levelgraph/node_modules/level-write-stream/node_modules/end-stream/index.js:13:9)
    at Stream.handleWrite [as write] .../lib/node_modules/levelgraph/node_modules/level-write-stream/node_modules/end-stream/node_modules/write-stream/index.js:22:28)
    at write (.../lib/node_modules/levelgraph/node_modules/readable-stream/lib/_stream_readable.js:623:24)
    at flow (.../lib/node_modules/levelgraph/node_modules/readable-stream/lib/_stream_readable.js:632:7)
    at .../lib/node_modules/levelgraph/node_modules/readable-stream/lib/_stream_readable.js:600:7
    at process._tickDomainCallback (node.js:381:11)

though I've hackishly modified the EventEmitter prototype to increase the limit, and when it's high enough so this warning doesn't show, the put call still hangs.

clarifications

Hi mcollina,

A great extension to Levelgraph, thank you. It works great at storing my N3 dataset.

Can you explain the role the "N3" option plays with the Join. Is it required? Is it required on the last conditional in the Join list? I looked at your code and saw some concatenation of streams going on, but that did not clarify it's purpose. What are the rules of its use?

Also, in your Join example you reference a predicate "http://example.org/cartoons#dumberThan" in the N3 block. Since this was not originally "put" into the db, is it inferred automatically? I am confused by your magic.

Thanks in advance for your clarifications.
-ross

Version bump for LevelGraph v2 compatibility?

I'm using both this and levelgraph-jsonld together on LevelGraph Playground. However, this one's not quite ready for today's LevelGraph 2.x release.

NPM says:

npm ERR! peerinvalid The package [email protected] does not satisfy its siblings' peerDependencies requirements!
npm ERR! peerinvalid Peer [email protected] wants levelgraph@^2.0.0
npm ERR! peerinvalid Peer [email protected] wants levelgraph@>= 1.0 < 2.0

Maybe the package.json just needs a tweak?

/cc @jmatsushita

Thanks!
🎩

Supporting formulas and quantification

Hi,

I'm trying to use levelgraph-n3 for storing N3 rules, but I need they have formulas and quantification like this example: http://n3.restdesc.org/rules/
If I write a simple rule like this:

@Prefix ppl: http://example.org/people#.
@Prefix foaf: http://xmlns.com/foaf/0.1/.

{
ppl:Cindy foaf:knows ppl:John.
}
=>
{
ppl:John foaf:knows ppl:Cindy.
}.

I get a syntax error in '=>'. Is there any way to store rules with this format using levelgraph-n3?

Thank you very much !

refactor N3 augmenting LevelGraph search (former join)

Conversation started in #6 after s/join/search/
Possibly we may switch to keeping all N3 related features namespaced under db.n3 so possibly we need to move the way we augment db.search() into `db.n3.search()'

We should keep conversation here in sync with #1

problem with min build

With levelgraph-n3 0.3.2 installed via bower, using levelgraph-n3.min.js, I get the following error when calling n3Db.n3.put. Only happens with minified file.

SyntaxError: Invalid regular expression: /^((?:[A-Za-zÀ-ÖØ-öø-Ë¿Í°-ͽͿ-῿‌â€�â�°-â†�â°€-⿯ã€�-퟿豈-ï·�ï·°-�]|[í €-í­¿][í°€-í¿¿])(?:[\.\-0-9A-Z_a-z·À-ÖØ-öø-ͽͿ-῿‌â€�‿â�€â�°-â†�â°€-⿯ã€�-퟿豈-ï·�ï·°-�]|[í €-í­¿][í°€-í¿¿])*)?:(?=\s)/: Range out of order in character class
    at new RegExp (native)
    at Object.<anonymous> (http://localhost:8080/lib/levelgraph-n3/build/levelgraph-n3.min.js:4:16872)
    at s (http://localhost:8080/lib/levelgraph-n3/build/levelgraph-n3.min.js:1:518)
    at http://localhost:8080/lib/levelgraph-n3/build/levelgraph-n3.min.js:1:569
    at Object../N3Lexer.js (http://localhost:8080/lib/levelgraph-n3/build/levelgraph-n3.min.js:4:24350)
    at s (http://localhost:8080/lib/levelgraph-n3/build/levelgraph-n3.min.js:1:518)
    at http://localhost:8080/lib/levelgraph-n3/build/levelgraph-n3.min.js:1:569
    at Object../N3Parser.js (http://localhost:8080/lib/levelgraph-n3/build/levelgraph-n3.min.js:5:6636)
    at s (http://localhost:8080/lib/levelgraph-n3/build/levelgraph-n3.min.js:1:518)
    at http://localhost:8080/lib/levelgraph-n3/build/levelgraph-n3.min.js:1:569 

I will probably switch to browserify for build soon anyway.

PS I'm doing a dive with levelgraph http://edrex.github.io/levelgraph-dive/, might end up being a good external example.

tests with blank nodes

just fast

$ grep blank test/*
$ grep _: test/*

makes impression that we don't test cases with blank nodes 🃏

Installation failed in npm

$ npm install level levelgraph levelgraph-n3 --save
Build failure.
Could anyone help?

I am using
npm -v
7.23.0
node -v
v16.9.0

Should I downgrade to npm 2.1.3 to make it work?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.