nodesolidserver / node-solid-server Goto Github PK
View Code? Open in Web Editor NEWSolid server on top of the file-system in NodeJS
Home Page: https://solidproject.org/for-developers/pod-server
License: Other
Solid server on top of the file-system in NodeJS
Home Page: https://solidproject.org/for-developers/pod-server
License: Other
Along with authentication, RDF ACL authorization is an important feature for this server. Then we will be able to run it facing the open net rather than as a protected proxy.
The goal would be to make the file structure used on the server be compatible with gold, so that one could run node-ldp-httpd or gold on the same files and have redundancy of project, and no 'project lock-in'.
Also of course made sure node-rdf-acl works with any other express systems.
(although other conversation have been raised, I open an issue just to keep track of this)
Tests are too slow, we should use nock
to mock HTTP requests and speed up the tests, especially ACL
This (with rdfACL control) is one of the two most pressing things for this project at the moment.
@deiu notices that @magnetik had actually done a webid for node https://github.com/magnetik/node-webid so we forked it here in linkeddata and it remains to integrate it with the rest of the server. Any volunteers?
A nine syllable name for this project is pretty painful. Whenever it comes up in conversation I find myself unable to remember the name, and unsure my listeners would understand/parse/recognize/remember it.
I suggest ldnode or ld-node would be a good name, along the lines of ldphp and ldpy, and not far from gold (which is ldgo re-ordered).
timbl: > [email protected] postinstall /devel/github.com/linkeddata/ldnode/node_modules/webid
> cake build
sh: cake: command not found
This was trying to do an install from scratch of ldnode
Would it be possible to add the coffee-script dependency to the install?
Also information on how to generate one should be provided
The tests that we have in our folder are acceptance tests, in other words we are testing if the API works as expected. The same tests can be used for any other LDP implementation, e.g. gold.
So, my idea is that we could just take this tests out in a new package like ld-test
or superld
that tests a LDP. So that in this repo we can test more on unit testing.
This is just an idea, otherwise we could just leave things as they are, which are great!
The idea is to use routes instead of implementing it straight into the main toutes
// in server.js or whoever will implement it
app.use('/what_i_desire', routes);
// in our library
var express = require('express');
var router = express.Router();
/* GET home page. */
router.get('*', function(req, res) {
});
router.patch('*', function(req, res) {
});
module.exports = router;
In this way, we can make something that any user can plug and play in their environment
This is only available in the latest express
. Good idea to upgrade?
While it could be useful to hard code the test
folder, it should not be hard-coded in this way:
/devel/github.com/linkeddata/node-ldp-httpd/test/
since we are assuming the developer is storing code in this way :(
TypeError: undefined is not a function
at Object.exports.publishDelta (/devel/github.com/linkeddata/ldnode/subscription.js:147:5)
at /devel/github.com/linkeddata/ldnode/handlers/patch.js:123:34
at doPatch (/devel/github.com/linkeddata/ldnode/node_modules/rdflib/dist/rdflib.js:4340:9)
at $rdf.IndexedFormula.applyPatch (/devel/github.com/linkeddata/ldnode/node_modules/rdflib/dist/rdflib.js:4369:9)
at handler (/devel/github.com/linkeddata/ldnode/handlers/patch.js:108:22)
at Layer.handle as handle_request
at next (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/route.js:131:13)
at Route.dispatch (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/route.js:112:3)
at Layer.handle as handle_request
at /devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/index.js:277:22
stderr exert:
Maybe the debug()
function is undefined
This happens after a successful PATCH
has been serviced, in the code to publish the change subscriptions. Whether or not --no-live
is set in the command live
Tue, 14 Jul 2015 15:35:12 GMT ldnode:handlers PATCH -- applied OK (sync)
TypeError: undefined is not a function
at Object.exports.publishDelta (/devel/github.com/linkeddata/ldnode/subscription.js:147:5)
at /devel/github.com/linkeddata/ldnode/handlers/patch.js:123:34
at doPatch (/devel/github.com/linkeddata/ldnode/node_modules/rdflib/dist/rdflib.js:4340:9)
at $rdf.IndexedFormula.applyPatch (/devel/github.com/linkeddata/ldnode/node_modules/rdflib/dist/rdflib.js:4369:9)
at handler (/devel/github.com/linkeddata/ldnode/handlers/patch.js:108:22)
at Layer.handle [as handle_request] (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/layer.js:95:5)
at next (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/route.js:131:13)
at Route.dispatch (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/route.js:112:3)
at Layer.handle [as handle_request] (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/layer.js:95:5)
at /devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/index.js:277:22
This error I thought was fixed yesterday but has come back Monday morning 7/13.
ldnode:handlers PATCH -- text length: 342 +0ms
TypeError: Arguments to path.join must be strings
at path.js:360:15
at Array.filter (native)
at Object.exports.join (path.js:358:36)
at Object.uriToFilename (/devel/github.com/linkeddata/ldnode/fileStore.js:9:25)
at handler (/devel/github.com/linkeddata/ldnode/handlers/patch.js:18:25)
at Layer.handle as handle_request
at next (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/route.js:131:13)
at Route.dispatch (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/route.js:112:3)
at Layer.handle as handle_request
at /devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/index.js:277:22
This should be added in the repo, package.json and README
Link: <http://www.w3.org/ns/ldp#Resource>; rel="type", </timbl/2015/08/02/poll/forms.ttl.changes>; rel="changes", <https://timbl.com/timbl/2015/08/02/poll/forms.ttl.acl>; rel="acl", <https://timbl.com/timbl/2015/08/02/poll/forms.ttl.meta>; rel="describedBy"
should be
Link: <http://www.w3.org/ns/ldp#Resource>; rel="type", <forms.ttl.changes>; rel="changes", <forms.ttl.acl>; rel="acl", <forms.ttl.meta>; rel="describedBy"
I am not sure where that comes from
I need to be able to set up a custom return HTML page for HTTP errors, especially error 401 "Unauthenticated". In practice this is an important part of the on-boarding process, to lead someone who does not have a webid to grab one.
Apache does this with the ErrorDocument directive a la
ErrorDocument 404 /Lame_excuses/not_found.html
ErrorDocument 401 /Subscription/how_to_subscribe.html
Following the LinkedData 2nd meeting. There is no need to set up the baseUri.
This should be set up by looking at the host in the request.
Remember: Always use the base that you find in the request
Turns out there's a bug in the webid module:
TypeError: Cannot call method 'replace' of undefined
at new VerificationAgent (/devel/github.com/linkeddata/ldnode/node_modules/webid/index.js:21:23)
at loginHandler (/devel/github.com/linkeddata/ldnode/login.js:20:30)
at Layer.handle [as handle_request] (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/index.js:312:13)
at /devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/index.js:280:7
at param (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/index.js:349:14)
at param (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/index.js:365:14)
at Function.process_params (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/index.js:410:3)
at next (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/index.js:271:10)
at /devel/github.com/linkeddata/ldnode/index.js:125:24```
I think the only correctly implemented test is test/params
, see line
https://github.com/linkeddata/ldnode/blob/master/test/params.js#L70
supertest
allow us to just test an express app, without running it on a port.
Running each test on a port is not just time consuming, but also, whenever two tests run on the same port, sometimes we get errors on travis.
Usage of proxy should be added in command line and checked if it works
Same as any other suffix, suffixAcl
, etc..
ldnode does not currently support CORS headers. We need to have at least Access-Control-Allow-Origin
support.
There are two solution for this:
process.env.URI_BASE
)I would prefer the second one having a fallback on the default test
folder
Ldnode with multi-language support
Gold (or databox.me) returns a listing of this kind:
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
<>
a <http://www.w3.org/ns/ldp#BasicContainer>, <http://www.w3.org/ns/ldp#Container>, <http://www.w3.org/ns/posix/stat#Directory> ;
<http://www.w3.org/ns/ldp#contains> <Applications/>, <Family/>, <Friends/>, <Preferences/>, <Private/>, <Public/>, <Work/>, <profile/> ;
<http://www.w3.org/ns/posix/stat#mtime> "1435333390" ;
<http://www.w3.org/ns/posix/stat#size> "4096" .
<Applications/>
a <http://www.w3.org/ns/ldp#BasicContainer>, <http://www.w3.org/ns/ldp#Container>, <http://www.w3.org/ns/posix/stat#Directory> ;
<http://www.w3.org/ns/posix/stat#mtime> "1435333433" ;
<http://www.w3.org/ns/posix/stat#size> "4096" .
However ldnode returns this:
@prefix n0: </>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix st: <http://www.w3.org/ns/posix/stat#>.
@prefix keys: </keys/>.
@prefix res: </resources/>.
<#>
ldp:contains
</acl.js>,
</formats.js>,
</http.js>,
n0:keys,
</params.js>,
</patch.js>,
n0:resources;
st:mtime
1437300761000;
st:size
306.
</acl.js> st:mtime 1437139220000; st:size 42866 .
</formats.js> st:mtime 1437090751000; st:size 1982 .
</http.js> st:mtime 1437302617000; st:size 6750 .
keys: st:mtime 1436972622000; st:size 340 .
</params.js> st:mtime 1437089197000; st:size 3349 .
</patch.js> st:mtime 1437090751000; st:size 5410 .
res: st:mtime 1437302621000; st:size 986 .
Since warp looks for POSIX(Directory)
in here https://github.com/linkeddata/warp/blob/gh-pages/src/app/list/list.js#L188
Still present here https://github.com/linkeddata/ldnode/blob/master/handlers/get.js#L20
Running ldnode -v
no longer outputs debug information to stdout.
We need the travis config file
Most of the function in parse.js
implement what $rdf.parse
is for, correct?
If not, this is a very valuable too and it should be separated in a different lib linkeddata-parse
/ ldnode-parse
Code can be taken and adapted from gold: https://github.com/linkeddata/gold/blob/master/server.go#L884
This is the last bit that's missing from ldnode, which would make it fully compatible with warp.
.js
should be in a folder resources
and testfiles
should be a subdirectory of resources
(it is important for testing to have a subdir!), possibly called testContainer
. (done #80)test.js
and restructure tests (done #85)Underscore is an awesome library, but I was wondering if we really need it.
I was having a look at https://github.com/linkeddata/ldnode/blob/master/acl.js and I have seen that we are using _.each
, which is the equivalent of obj.forEach
that I believe it is also more readable.
Is there any reason for this particular choice?
SoLiD apps require the presence of two Link headers in order to find where the corresponding ACL and meta resources are. The SoLiD document has examples here: https://github.com/linkeddata/SoLiD/blob/master/README.md#the-head-method
debug trace, filtered:
ldnode:ACL Looking for policies in /Library/WebServer/Documents/TimBL/Automation/Library/Mashup/jquery.min.js.acl +0ms
ldnode:ACL Looking for policies in /Library/WebServer/Documents/TimBL/Automation/Library/Mashup/.acl +0ms
ldnode:ACL Looking for policies in /Library/WebServer/Documents/TimBL/Automation/.acl +0ms
ldnode:ACL Looking for policies in /Library/WebServer/Documents/.acl +0ms
ldnode:ACL Found policies in /Library/WebServer/Documents/.acl +0ms
It never found the one in /Library/WebServer/Documents/TimBL/Automation/Library/.acl
with
commit 30dc1b5
Author: nicola [email protected]
Date: Mon Jul 13 00:45:33 2015 +0200
It looks like dirname() being invoked twice ...
At the moment I am trying to run the tests but they all fail since they can't find the resources, hence the socket hang-up.
(I know they are under development! This is just to know how they work!)
I was wondering why every single option would be needed
app.use(session({
secret: 'node-ldp',
saveUninitialized: false,
resave: false
}));
I consider the secret: 'node-ldp'
by default a serious security issue
The Solid world requires the User: header to be sent back when a user is logged on with webid/TLS
Typical response headers from a GET authenitcated by webid currently:
status 200
status Text OK
connection keep-alive
content length 1367428
content type text/n3; charset=utf-8
etag W/"14dd84-P7TE3CNZwvMq1mvb6/zJ+w"
link http://www.w3.org/ns/ldp#Resource; rel='type', </TimBL/Automation/mother/state.n3.changes>; rel='changes'
ms author via SPARQL
updates via /TimBL/Automation/mother/state.n3.changes
x powered by Express
x response time 4.160ms
Assuming we want to start two ldnode running, editing the options' attribute will statically edit the options of each ldnode instance.
This problem does not arise in the current version of ldnode that has to be run from terminal, but it does in my fork where I am transforming ldnode into a library rather than a command line tool.
See my branch nicola/ldnode#routes for reference
This is caused by tls.getPeerCertificate() getting a wrong modulus
. This is the case of @deiu's webId: https://deiu.me/profile#me
Since package.json
contains the script starts
and the script test
, is the Makefile still needed?
I am a bit confused of the references to tabulator and /dev/WWW (which I assume they were probably originally used for quick & dirty testing)
Update
What I am seeing now is that in order to run the tests one has to run ldnode
with some specific settings and just by doing make
you start ldnode ready for test settings. However this should all be done by npm test
. The user doesn't want to see or do too many things when running the tests
Following this and many other sources, unless we are exporting a function as the only thing we export (instead of an object).
Mentioned here! #21
In some part of the code (e.g. https://github.com/linkeddata/ldnode/blob/master/acl.js) I have seen that we declare some exports.something
, and then we use them as exports.something
. I think that makes the code a bit unreadable.
I think declaring an export in the following way is more of a standard in the js community.
function Handler () {..
// and at the end of the file
exports.handler = Handler
PS: I was going to make some pull reqs, but I think we should first agree on this!
I have seen in this file: https://github.com/linkeddata/ldnode/blob/master/handlers/get.js that the functions get
, parseContainer
, parseLinkedData
, globHandler
, containerHandler
, fileHandler
are declared as anonymous functions and assigned to a variable.
Since they are called before they assigned, this may lead to a TypeError
or similar issues related to variable hoisting in javascript. http://jamesallardice.com/explaining-function-and-variable-hoisting-in-javascript/
So either we move up the variables, or we declare them, however I think declaring them is not a good idea either.
Instead of declaring them inside a function call (this means 1000 function calls 1000 declarations), we could instead just declare them outside and carry all the variables they need, I assume req
, res
being the only two. (This comment is valid for all the other files in /handlers that I have seen)
In case it is not specified, use process.cwd()
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.