spiral-project / daybed Goto Github PK
View Code? Open in Web Editor NEWForm validation and data storage API
Home Page: http://daybed.rtfd.org/
License: BSD 3-Clause "New" or "Revised" License
Form validation and data storage API
Home Page: http://daybed.rtfd.org/
License: BSD 3-Clause "New" or "Revised" License
Request method: POST
Request URL: http://localhost:8000/data/mushroomspot
Request payload: {"mushroom":"tete","area":"[20.390625,7.536764322084078]"}
ERROR [waitress][Dummy-1] Exception when serving /data/mushroomspot
Traceback (most recent call last):
File "env/local/lib/python2.7/site-packages/waitress/channel.py", line 329, in service
task.service()
File "env/local/lib/python2.7/site-packages/waitress/task.py", line 173, in service
self.execute()
File "env/local/lib/python2.7/site-packages/waitress/task.py", line 380, in execute
app_iter = self.channel.server.application(env, start_response)
File "env/local/lib/python2.7/site-packages/pyramid-1.4-py2.7.egg/pyramid/router.py", line 252, in __call__
return response(request.environ, start_response)
File "env/local/lib/python2.7/site-packages/WebOb-1.2.3-py2.7.egg/webob/response.py", line 1021, in __call__
start_response(self.status, headerlist)
File "env/local/lib/python2.7/site-packages/waitress/task.py", line 364, in start_response
'Header value %r is not a string in %r' % (v, (k, v))
AssertionError: Header value u'http://localhost:8000/data/59fcf69f72069e8bf8d917a155001179' is not a string in ('location', u'http://localhost:8000/data/59fcf69f72069e8bf8d917a155001179')
Lettuce features and scenariis may be a little bit of the track. I lack a good example of Web API testing.
However I really like the idea !
I wonder if we should actually be more readable in the scenarii file, for e.g :
When I send a bad model
When I send a incomplete model
And something like:
@step("When I send a (bad|incomplete) model")
def i_send_a_model(aspect):
if aspect == 'bad':
model = "{uierui:gjigjr}"
elif aspect == "incomplete":
model = '{"title" : "yeah"}'
...
...
I asked @stephane for a quick review :) Let's see!
currently we are returning a list of fields, we could also return more information about them.
That can store and validate any type of supported geometry value
Will be useful for remote applications after merging what lays in dev branch
A way to register available fields types.
It should :
type
field validatorchoices
, dateformat
)month<=12
, abs(lat)<=90
)Accept
HTTP directive may not be enough to format records.
For example, getting the records of a model which contain a geometry as JSON may have two formattings :
[ {...}, {...} ]
){ "type": "FeatureCollection", "features": [{...}, {...}]
), that integrates nicely in JS map widgets like Leaflet.Both are valid JSON. We might need a ?format=XXX
parameter to specify which layout it should have.
Vega is a declarative grammar for data visualizations
List of integer, string, etc. or list of "sub-model" for inline records
The documentation is outdated.
We have to document everything.
Browser ID is a really good idea to use daybed as as javascript UI backend.
Basically we can then use the email as the account id for models and data management.
https://github.com/jkbr/httpie
that would probably be prettier than curl
When running tests, the "daybed" database should already be created otherwise an error will be thrown. This should be done automatically to avoid additional user hassle when users are getting started.
After discussion with @AntoineCezar and debates in issue #9 it seems relevant to use JSON schema for model definitions, instead of our own formalism.
A basic investigation needs to address the following questions:
More generally, should JSON schema validation come as a Cornice extension or as internal Daybed mechanics ?
(previous title was "remove the notion of token")
Suggested by @AntoineCezar
Daybed should be as standard as possible in terms of REST and CRUD.
If any ACL feature (see #8) has to be implemented, it may be relevant to implement it in a separated frontend rather than in daybed core.
That could be useful to return a random element from a list of resources.
At model creation, it could be possible to define groups of fields:
SCHEMA = {
name: "My super event",
description: "",
fields: [
{ type: "string",
name: "title",
description: "blah",
},
{ name: "category",
type: "enum",
description: "blah",
choices: ['sport', 'culture']
},
{ name: "street",
type: "string",
},
{ name: "city",
type: "string",
},
],
groups: [
{ title: "Home",
description: "Your address helps us find you",
fields: ["street", "city"]
},
]
}
This would allow the UI to nicely layout the form for this model.
Since groups would be optional, they do not interfere with existing models, or those who don't need UI forms.
The format description we're using to define our validation models could be useful for others, so let's split this up! (maybe in the documentation ?)
We could envision a way to refer a model as a relationship :
Definition could look like:
{
"title": "Person",
"description": "An individual with a soul",
"fields": [
{
"name": "fullname",
"type": "string",
}, {
"name": "brothers",
"type": "many-to-many",
"model": "Person"
}
And an entry like :
{
"name": "Howard",
"brothers": [54321, 9876]
}
Validating this entry would mean checking that provided brother
ids exist.
I found myself trying to find a name for my definition even if I didn't needed one. That could be useful to have daybed deciding directly this kind of things for us. I like heroku which is choosing some names that "could" be real worlds, rather than using hashed of some sort.
when resources don't exist, we should return the right CORS headers. I'm not sure we're doing this ATM.
The actual token system bring behaviors that does'nt look good to me.
When I POST data I excpect to get the full object with additionnal information like for exemple a "created_at".
In a POST based create definition I recive a token and have to follow a redirect to get the full object.
So unless there is a better way to pass tokens I suggest droping them for the moment.
Idea by @AntoineCezar
Allow users to define their own validation function in the posted JSON model definition :
fields: [
{
type: "int",
name: "age",
description: "Your mother age",
validation: "if (value < 15) {
return 'Seems a bit young.';
}
else (value > 120) {
return 'Can't be so old.';
}
return null;",
},
Most appropriate language seems to be Javascript (since JSON).
Javascript syntax must be checked during model definition.
Validation function should be executed on model post. In my opinion, server side (thus in python) :
http://renesd.blogspot.fr/search/label/shitjs
https://bitbucket.org/pypy/lang-js/src/de89ec32a7dc/js/javascript-interpreter.txt
http://code.google.com/p/pyv8/ : extra dep :(
I'm on ArchLinux and the default python is python3.
Virtualenv should explicitly use python2 with the --python=python2 option.
It says "make setup_venv" instead of "make install"
With this approach ?
POST /model
)GET /model
)PUT /model/<UUID>
Means users should keep track of their uuids themselves, not fantastic.
Other idea (brainstorm)...
POST /model?lock=1234
). If no password, record is public.Actually, same approach could be applied to model definitions...
Will imply running daybed behind https...
POST invalid point data gives 500 error :
File "/home/mathieu/Code/daybed/src/daybed/validators.py", line 35, in schema_validator
return validator(request, schema)
File "/home/mathieu/Code/daybed/src/daybed/validators.py", line 20, in validator
for field, error in error.asdict().items():
File "/home/mathieu/Code/daybed/src/local/lib/python2.7/site-packages/colander/__init__.py", line 165, in asdict
keyname = exc._keyname()
File "/home/mathieu/Code/daybed/src/local/lib/python2.7/site-packages/colander/__init__.py", line 153, in _keyname
return str(self.node.name)
AttributeError: 'PointType' object has no attribute 'name'
http://www.elasticsearch.org does exactly what we need in terms of faceting (i.e. filters, drill down, ...)
Plus, it has a plugin to be fed from CouchDB : http://www.elasticsearch.org/guide/reference/river/couchdb.html
However I wonder how easy it could be deployed (ex: Heroku, Travis,...)
see Cornice error Cornices/cornice#55
PATCH allows us to only change some values inside an item, which is rather cool.
I want that :-) Less traffic, less headaches.
POST ``{"mushroom":"frfr","area":"111"}`
{"status": "error", "errors": [{"location": "body", "name": "", "description": "\"111\" is not iterable"}]}
Il faut valider avec https://github.com/SPORE/specifications/blob/master/spore_validation.rx en utilisant https://github.com/rjbs/rx
File "/home/mathieu/Code/daybed/src/daybed/validators.py", line 35, in schema_validator
return validator(request, schema)
File "/home/mathieu/Code/daybed/src/daybed/validators.py", line 20, in validator
for field, error in error.asdict().items():
File "/home/mathieu/Code/daybed/src/local/lib/python2.7/site-packages/colander/__init__.py", line 167, in asdict
errors['.'.join(keyparts)] = '; '.join(interpolate(msgs))
TypeError: sequence item 0: expected string, exceptions.ValueError found
http://en.wikipedia.org/wiki/XForms#ODKCollect
Looks like Android applications can be automatically built upon this spec...
Be careful though, could be bloated (Java inside).
For example (does not have to be all of them) :
Does not seem to be the case:
https://github.com/spiral-project/daybed/blob/master/daybed/views/data.py#L45-L49
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.