jpmckinney/validictory

🎓 deprecated general purpose python data validator

validictory

warning:	⚠️ As of 2018 this library is deprecated, please consider using jsonschema (https://pypi.python.org/pypi/jsonschema) instead.

A general purpose Python data validator.

Schema format based on JSON Schema Proposal (http://json-schema.org)

Contains code derived from jsonschema, by Ian Lewis and Yusuke Muraoka.

Usage

JSON documents and schema must first be loaded into a Python dictionary type before it can be validated.

Parsing a simple JSON document:

>>> import validictory
>>>
>>> validictory.validate("something", {"type":"string"})

Parsing a more complex JSON document:

>>> import json
>>> import validictory
>>>
>>> data = json.loads('["foo", {"bar":["baz", null, 1.0, 2]}]')
>>> schema = {
...   "type":"array",
...   "items":[
...     {"type":"string"},
...     {"type":"object",
...      "properties":{
...        "bar":{
...          "items":[
...            {"type":"string"},
...            {"type":"any"},
...            {"type":"number"},
...            {"type":"integer"}
...          ]
...        }
...      }
...    }
...   ]
... }
>>> validictory.validate(data,schema)

Catch ValueErrors to handle validation issues:

>>> import validictory
>>>
>>> try:
...     validictory.validate("something", {"type":"string","minLength":15})
... except ValueError, error:
...     print(error)
...
Length of value 'something' for field '_data' must be greater than or equal to 15

You can read more in the official documentation at Read the Docs.

validictory's Issues

Proposal: Support of minProperties and maxProperties

The json schema proposal includes two validations for the properties: minProperties and maxProperties. Using these validations we could define an object with several optional properties and, for example, define that it could have only one property.

A example of use is:

schema = {"type": "object",
"properties": {
"a": {"type":...},
"b": {"type":...},
"minProperties" = 1,
"maxProperties" = 1}}

valid:
{"a": ...}
{"b": ...}

Invalid:
{}
{"a": ..., "b":...}

Positional arguments missing error

I am trying to use validictory package in python 2.6 and the validator failed due to the following error.

File "build/bdist.linux-x86_64/egg/validictory/validator.py", line 275, in validate_items
'{}[{}]'.format(path, index))
ValueError: zero length field name in format

Suggestions online:
In Python 2.6, you need indices in the format specs, like this:
"{0} {1}\n".format(x,y)

Few questions

I am planning to use validictory for one of my projects. Main motivation is possible sharing of schema with frontend validators. I could not find quick answers for some of the questions.

• How to customize error messages?
• Some validation libraries include dict (or similar structure) keyed by field names so that one can look at names and populate error messages accordingly. Is it a reasonable expectation to have such thing in Validictory? Or it's already there and just not documented.
• How to add new custom validators?

Sorry for many questions. I can contribute to code if needed.

Custom error messages

A validictory.validate(a,b) can, for example, raise a FieldValidationError. I would like to have access to the value, the constraint (maxLength, minLength) and the field so I can customise my error messages.

So far, we have:

ipdb> err
FieldValidationError("Value u'aaaaaaaaaaaaaaa' for field '<obj>[0]' must have length greater than or equal to 16",)
ipdb> err.
err.args       err.fieldname  err.message    err.value      
ipdb> err.args
("Value u'aaaaaaaaaaaaaaa' for field '<obj>[0]' must have length greater than or equal to 16",)
ipdb> err.fieldname
'[list item]'
ipdb> err.message
"Value u'aaaaaaaaaaaaaaa' for field '<obj>[0]' must have length greater than or equal to 16"
ipdb> err.value
u'aaaaaaaaaaaaaaa'

I can't programmatically do much with [list item] when it really should have been id (the key of the property being validated) and I can't tell that it was actually a minLength error.

Can anything be done here?

Support for extends

I wonder if validictory supports "extends" or if there are any plans to add support for it?

Add 0.9.1 to ChangeLog

I noticed that 0.9.1 was released on 2013-05-23, but I didn't see any entries in the ChangeLog at http://validictory.readthedocs.org/en/latest/changelog.html

though I do see entries at: https://github.com/sunlightlabs/validictory/blob/e653bfbf3a0bacfb51e8976cfc3c5f7b974a2977/docs/changelog.rst

So perhaps ReadTheDocs just needs to be pinged and told to rebuild the docs?

Problem with additionalItems

First, thanks a lot for this tool, it's very nice!

However, it is my understanding that an array with additionalItems set to True should match each item against the schema. I'm either misunderstanding the usage (first time using validictory) or it's not working.

below is a test script and dataset;

---

jsontest.py

import simplejson
import validictory

f = open('test.json2', 'r')
data = simplejson.load(f)
schema = {
"type" : "array",
"items" : [
{"type" : "object",
"properties" : {
"a" : {"type" : "string", "required" : True},
"b" : {"type" : "array",
"items" : [
{"type" : "string", "required" : True},
], 'required' : True,
},
"c" : {"type" : "string", "required" : True},
}
}
],
'minItems' : 1,
'additionalItems' : True,
'required' : True
}
if not validictory.validate(data,schema):
print "Success!"
else:
print "Failure!"

---

test.json2

[
{"a" : "A",
"b" : ["B"],
"c" : "C"
},
{"a" : "D",
"b" : ["E"],
"c" : "F"
}
]

This validates as I would expect and prints "Success!".

If I remove an item from the first set (e.g. "a" : "A") it fails to validate and prints "Failure!"

But, if I remove an item from the 2nd data set (e.g. "a" : "D") it validates successfully.

I appreciate any feedback.

-Dave

P.S. I'm using version 0.8.3

Stale Package on PyPI

The source here is now version 0.8.0, but the PyPI package is back at 0.7.2 . Any chance of bringing PyPI up to date?

add long-Type to allowed datatypes of 'utc-millisec' formatted numbers

Currently, 'utc-millisec' formatted numbers are only allowed to be of type int and float.
So

{"created":1346691273926}

cannot be validated by a schema like:

{"type":"number","format":"utc-millisec"}

I propose the following patch to allow the (in my eyes valid) data to be validated:

diff --git a/validictory/validator.py b/validictory/validator.py
index 8e81d58..f17d7f1 100755
--- a/validictory/validator.py
+++ b/validictory/validator.py
@@ -44,7 +44,7 @@ validate_format_time = _generate_datetime_validator('time', '%H:%M:%S')


 def validate_format_utc_millisec(validator, fieldname, value, format_option):
-    if not isinstance(value, (int, float)):
+    if not isinstance(type(value), _int_types):
         raise ValidationError("Value %(value)r of field '%(fieldname)s' is "
                               "not a number" % locals())

JSON data contains a key named "description" : SchemaError

Version 0.9.3 of validictory.

When I run the code below I get the error validictory.validator.SchemaError: The description for field '_data' must be a string

There seems to exist a naming conflict between the built-in "description" key and the "description" key in my data. (see the code and stacktrace below.)

In the docs it states title and description These do no validation, but if provided must be strings or a ~validictory.SchemaError will be raised.

This is highly problematic, in that, much of the json data I'm being provided contains one or more "description" keys. How do I work around this?

The Code

import json
import validictory

data = json.loads(''' {"results": [{"description": "abc", "status": "edf"}, {"description": "t resul", "status": "t stat"}]} ''')

schema = {
    u'type': u'object',
    u'properties': {
        u'results': {
            u'type': u'array',
            u'items': {
                u'description': {u'type': u'string'},
                u'status': {u'type': u'string'},
            },
        }
    }
}

validictory.validate(data, schema)

Running the code results in the following stacktrace:

Traceback (most recent call last):
  File "~/json/debug_schema_error.py", line 19, in <module>
    validictory.validate(data, schema)
  File "C:\Python27\lib\site-packages\validictory\__init__.py", line 36, in validate
    return v.validate(data, schema)
  File "C:\Python27\lib\site-packages\validictory\validator.py", line 584, in validate
    self._validate(data, schema)
  File "C:\Python27\lib\site-packages\validictory\validator.py", line 587, in _validate
    self.__validate("_data", {"_data": data}, schema)
  File "C:\Python27\lib\site-packages\validictory\validator.py", line 618, in __validate
    newschema.get(schemaprop))
  File "C:\Python27\lib\site-packages\validictory\validator.py", line 242, in validate_properties
    properties.get(eachProp))
  File "C:\Python27\lib\site-packages\validictory\validator.py", line 618, in __validate
    newschema.get(schemaprop))
  File "C:\Python27\lib\site-packages\validictory\validator.py", line 279, in validate_items
    self._validate(eachItem, items)
  File "C:\Python27\lib\site-packages\validictory\validator.py", line 587, in _validate
    self.__validate("_data", {"_data": data}, schema)
  File "C:\Python27\lib\site-packages\validictory\validator.py", line 618, in __validate
    newschema.get(schemaprop))
  File "C:\Python27\lib\site-packages\validictory\validator.py", line 551, in validate_description
    % fieldname)
validictory.validator.SchemaError: The description for field '_data' must be a string

Process finished with exit code 1

additionalProperties causes a 'int' object not iterable error

    def test_nested(self):
        x =  {'rows' : [1, 2, 3]}



        schema = {'type' : 'object',
                  'properties' : {
                                  'rows' : { 'type' : 'array', 
                                             'items' : {'type' : 'object',
                                                        'properties' : {'_id' : {'type' : 'string'}},
                                                        'additionalProperties' : {'type' : 'number'}}}}}

       valedictory.validate(x, schema)

Gives

  File "/home/jonas/veritable/validictory/validictory/validator.py", line 267, in validate_additionalProperties
    for eachProperty in value:
TypeError: 'int' object is not iterable

Now this appears to be caused by additionalProperties in the schema being checked prior to the 'type' operator. I've forked validictory and think I have a work-around -- I'll submit a pull request soon in the next hr.

Thanks for making such a great project!

Question: reserved words like "type" in the json data

What's the recommended approach for json data you don't control where a fieldname is "type"? I'm finding this collides with the "type" fieldname Validictory already looks for to resolve object type.

Schema for non "object" types

I am trying to create a schema that will validate something like:

{"string": 1}

Or:

{1: "string"}

But I could not find any examples on how to achieve that, since "string" is just accepting any kind of string really, but it needs to make sure it has an integer as a value. I am sure there are other combinations that I am not looking into but again, I could not find anything in the docs regarding this specific scenario.

Is this actually possible?

Value...for field '_data' is not of type object

The following schema has been generated with jsonschema.net based on the given json data but the validation throws an error:

import json
import validictory

data = json.dumps({'bp': [{'category': 'bp',
'created': '2013-03-08T09:14:48.148000',
'day': '2013-03-11T00:00:00',
'id': 'dc049c0e-d19a-4e3e-93ea-66438a239712',
'unit': 'mmHg',
'value': 147.0,
'value2': 43.0}]})

schema = {
"type":"object",
"$schema": "http://json-schema.org/draft-03/schema",
"id": "http://jsonschema.net",
"required":False,
"properties":{
"bp": {
"type":"array",
"id": "http://jsonschema.net/bp",
"required":False,
"items":
{
"type":"object",
"id": "http://jsonschema.net/bp/0",
"required":False,
"properties":{
"category": {
"type":"string",
"id": "http://jsonschema.net/bp/0/category",
"default": "bp",
"required":False
},
"created": {
"type":"string",
"id": "http://jsonschema.net/bp/0/created",
"default": "2013-03-08T09:14:48.148000",
"required":False
},
"day": {
"type":"string",
"id": "http://jsonschema.net/bp/0/day",
"default": "2013-03-11T00:00:00",
"required":False
},
"id": {
"type":"string",
"id": "http://jsonschema.net/bp/0/id",
"default": "dc049c0e-d19a-4e3e-93ea-66438a239712",
"required":False
},
"unit": {
"type":"string",
"id": "http://jsonschema.net/bp/0/unit",
"default": "mmHg",
"required":False
},
"value2": {
"type":"number",
"id": "http://jsonschema.net/bp/0/value2",
"default":43,
"required":False
},
"value": {
"type":"number",
"id": "http://jsonschema.net/bp/0/value",
"default":147,
"required":False
}
}
}

    }
}

}

validictory.validate(data,schema)

Traceback (most recent call last):
File "val.py", line 79, in
validictory.validate(data,schema)
File "/home/ajung/sandboxes/mib.portal/lib/python2.7/site-packages/validictory/init.py", line 31, in validate
return v.validate(data, schema)
File "/home/ajung/sandboxes/mib.portal/lib/python2.7/site-packages/validictory/validator.py", line 559, in validate
self._validate(data, schema)
File "/home/ajung/sandboxes/mib.portal/lib/python2.7/site-packages/validictory/validator.py", line 562, in _validate
self.__validate("_data", {"_data": data}, schema)
File "/home/ajung/sandboxes/mib.portal/lib/python2.7/site-packages/validictory/validator.py", line 593, in __validate
newschema.get(schemaprop))
File "/home/ajung/sandboxes/mib.portal/lib/python2.7/site-packages/validictory/validator.py", line 205, in validate_type
value, fieldname, fieldtype=fieldtype)
File "/home/ajung/sandboxes/mib.portal/lib/python2.7/site-packages/validictory/validator.py", line 145, in _error
raise FieldValidationError(message, fieldname, value)
validictory.validator.FieldValidationError: Value '{"bp": [{"category": "bp", "created": "2013-03-08T09:14:48.148000", "value": 147.0, "day": "2013-03-11T00:00:00", "value2": 43.0, "id": "dc049c0e-d19a-4e3e-93ea-66438a239712", "unit": "mmHg"}]}' for field '_data' is not of type object

start checking coverage

add coveralls support

revisit documentation

• some better examples of how to use validictory (esp. since jsonschema has diverged a bit)
• ensure all authors are mentioned

Update package on PyPi

I just discovered that the package on PyPi is out of date when I tried to use the fail_fast kwarg on validate

handling of dates fails for year 0000

>>> import validictory
>>> object = {'mydate': '0000-10-03T15:35:05Z'}
>>> schema = {'type': 'object', 'properties': {'mydate': {'type': 'string', 'format': 'date-time'}}}
>>> validictory.validate(object,schema)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/matze/venv/local/lib/python2.7/site-packages/validictory/__init__.py", line 28, in validate
    return v.validate(data, schema)
  File "/home/matze/venv/local/lib/python2.7/site-packages/validictory/validator.py", line 531, in validate
    self._validate(data, schema)
  File "/home/matze/venv/local/lib/python2.7/site-packages/validictory/validator.py", line 534, in _validate
    self.__validate("_data", {"_data": data}, schema)
  File "/home/matze/venv/local/lib/python2.7/site-packages/validictory/validator.py", line 563, in __validate
    newschema.get(schemaprop))
  File "/home/matze/venv/local/lib/python2.7/site-packages/validictory/validator.py", line 191, in validate_properties
    properties.get(eachProp))
  File "/home/matze/venv/local/lib/python2.7/site-packages/validictory/validator.py", line 563, in __validate
    newschema.get(schemaprop))
  File "/home/matze/venv/local/lib/python2.7/site-packages/validictory/validator.py", line 427, in validate_format
    format_validator(self, fieldname, value, format_option)
  File "/home/matze/venv/local/lib/python2.7/site-packages/validictory/validator.py", line 37, in validate_format_datetime
    "'%(format_option)s' format" % locals())
validictory.validator.ValidationError: Value '0000-10-03T15:35:05Z' of field 'mydate' is not in 'date-time' format
>>> 

fails with:
validictory.validator.ValidationError: Value '0000-10-03T15:35:05Z' of field 'mydate' is not in 'date-time' format

which I believe is a bug.

Environment:
python 2.7 from ubuntu-sources (12.04)
validictory 0.8.3 installed via pip

Option to disallow unknown fields

schema = {'type': 'object', 'properties': {'test':{'type': 'string'}}}
data = {"test":"this_is_fine", "name": "not_in_the_schema"}

This will validate just fine, even if the 'name' field is not included in the schema definition. Depending on the use case this might be fine or not. An option for allowing unknown fields would be nice

"oneOf"

Hi,

did anyone here ever need "oneOf" from the JSON Schema specification (http://json-schema.org/example2.html)?

validictory does not support that yet, am I right?

Thanks,
Daniel

Allow passing a callable argument to enum

Would you be open to a pull request that makes it such that I can pass a lambda as the argument to enum, which will accept x and return a context-relevant list of values?

Validating a list of 'allowed values'

In an 'object' (dict), I have one field which I would define with a list of 'allowed values'. Ideally I would use the type=array definition:

{'type': 'object', 'properties': {'contact_type: {'items': [{'pattern': 'agent'}, {'pattern': 'client'}], 'type': 'array'}}}

in my use case, something like these would validate:
{"contact_type": ["agent"]}
{"contact_type": ["client", "agent"]}

while these wouldn't:
{"contact_type": ["blah", "client"]}
{"contact_type": []}
{"contact_type": ["agent", "client", "blah"]}

Currently I cannot achieve this validation because the array validator expects 1) the same number of items 2) at the same index position. I was wondering if I'm doing it wrong and there's actually a way to achieve this already, or if a new kind of validation item would be necessary (something like a "variable array").

Superfluous quote in blank validation

There is an extra, superfluous, quote at the end of the blank validation output.

Example taken from docs:

import json
import validictory

data = json.loads(''' {"hello": "", "testing": ""}''')

schema = {
    "properties": {
        "hello": {
            "blank": True # passes
        },
        "testing": {
            "blank": False # fails
        }
    }
}

try:
    validictory.validate(data, schema)
except validictory.validator.FieldValidationError as e:
    print e

Output:

Value '' for field 'testing' cannot be blank'

Regards,

John

Minor typo in docs

Under the "enum" section of the docs page found here: http://validictory.readthedocs.org/en/latest/usage.html

--> "tommrow" should be "tomorrow"

apply_default_to_data should not rely on required_by_default

Hi,

it seems that right now the option apply_default_to_data only works when required_by_default=False is passed to validictory.validate. Otherwise a RequiredFieldValidationError is thrown, even though a default is supplied.
Wouldn't it make more sense to first apply the default values and then check whether all required fields are present?

The drawback of the current design is that you apparently can't use both apply_default_to_data=True and required_by_default=True at the same time, but this restriction seems unnecessary.

invokation order of validators causes problems

I've problems while validating one field in my schema with a custom format validator.

hier ist a snippet of my schema containing the field i like to validate:

    "periods": {"type": "array",
                "description": "Contract periods",
                "required": True,
                "items": contract_period,
                "format": "sl:contract-periods",
                "minItems": 1}

I've registered a custom formatter called 'sl:contract-periods' which does some checks with the contained items in the field.
Since the method __validate() loops thru all schema properties to determine the required validators the validator called validate_format will get invoked earlier than the method validate_minItems , validate_optional or validate_type_array. So in my custom validator i have to check if the value is not None, a list-type and not empty which should get checked by the particular validators.

One possible solution could be to hardcode a list of validators to force a sorting order and appending the not hardcoded validators to the list for the loop. But there may be nicer ways...

additionalProperties with patternProperties does not work

Slightly confusing behaviour:

Passes as expected

schema = {
    "patternProperties": {
        "[a-c]": {
            "properties": {
                "cellid": {
                    "type": "integer"}
            }, "additionalProperties": False,
        }
    }
}
string = """{"c": { "cellid": 1}}"""
data = json.loads(string)
validictory.validate(data, schema)

Should pass, but doesn't

schema = {
    "patternProperties": {
        "[a-c]": {
            "properties": {
                "cellid": {
                    "type": "integer"}
            }, "additionalProperties": False,
        }
    }, "additionalProperties": False,
}
string = """{"c": { "cellid": 1}}"""
data = json.loads(string)
validictory.validate(data, schema)

Should fail, but doesn't

schema = {
    "patternProperties": {
        "[a-c]": {
            "properties": {
                "cellid": {
                    "type": "integer"}
            }, "additionalProperties": False,
        }
    }, "additionalProperties": False,
}
string = """{"d": { "cellid": 1}}"""
data = json.loads(string)
validictory.validate(data, schema)

Does additionalProperties therefore not match patternProperties?

I've tested #42, and it looks like @juanmb has fixed the issues with his Pull request.

Integer max/min errors are reports as floats

Hi,

Integer maximum and minimum exceptions are reported as floats.

Here is an example based on one of the examples in the docs:

import json
import validictory


data = json.loads(''' {"result": 10, "resultTwo": 12}''')

schema = {
    "properties": {
        "result": { # passes
            "minimum": 9,
            "maximum": 10
        },
        "resultTwo": { # fails
            "type": "integer",
            "minimum": 13
        }
    }
}

try:
    validictory.validate(data, schema)
except validictory.validator.FieldValidationError as e:
    print e

Running this give:

Value 12 for field 'resultTwo' is less than minimum value: 13.000000

While this isn't strictly incorrect it looks a little odd.

Replacing the format string %f with %g in the error handler would probably be a sufficient fix rather than varying the format string based on the validation type.

Regards,

John

Need more descriptive field name

Failed to validate field '_data' list schema: Length of value 'foo' for field '_data' must be greater than or equal to 10

I'm adding validictory to a csv data analysis & validation tool. With wide csv files I need something more specific than field '_data' failed the data validation check.

Would it be possible to include the title, or at least the index number for the list item in the error message?

Custom validation properties?

How can I use a custom function (that returns True/False) to validate fields against? Something like

data = simplejson.loads('{myString : "shoes"}')

schema = {
'myString' : isMyHat($value)
}
validictory.validate(data,schema)

def isMyHat(value):
if value == 'hat':
return True
else:
return False

Support for ISO 8601 +/-00:00 time zone designator

I need to specify a time zone in a field but the current validation strategy does not allow this. Can we have support for it please? Thanks!

Sometimes raises TypeError instead of FieldValidationError

I'm migrating my project to python3 and I've hit a weird problem.
Sometimes, the order of the schema is altered and the format appears to be before the type, so validictory tries to check the date format against types that aren't string, raising a TypeError instead of a FieldValidationError.

>>> import validictory
>>> validictory.__version__
'0.9.3'
>>> validictory.validate(1, {'type':'string', 'format':'date-time'})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.3/dist-packages/validictory/__init__.py", line 36, in validate
    return v.validate(data, schema)
  File "/usr/local/lib/python3.3/dist-packages/validictory/validator.py", line 584, in validate
    self._validate(data, schema)
  File "/usr/local/lib/python3.3/dist-packages/validictory/validator.py", line 587, in _validate
    self.__validate("_data", {"_data": data}, schema)
  File "/usr/local/lib/python3.3/dist-packages/validictory/validator.py", line 618, in __validate
    newschema.get(schemaprop))
  File "/usr/local/lib/python3.3/dist-packages/validictory/validator.py", line 480, in validate_format
    format_validator(self, fieldname, value, format_option)
  File "/usr/local/lib/python3.3/dist-packages/validictory/validator.py", line 44, in validate_format_datetime
    datetime.strptime(value, dateformat_string)
TypeError: must be str, not int

This doesn't happen always. I've seen the wrong error being thrown like half of the times. If I close this python3 interpreter session and open a new one, I might get the right error:

>>> import validictory
>>> validictory.__version__
'0.9.3'
>>> validictory.validate(1, {'type':'string', 'format':'date-time'})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.3/dist-packages/validictory/__init__.py", line 36, in validate
    return v.validate(data, schema)
  File "/usr/local/lib/python3.3/dist-packages/validictory/validator.py", line 584, in validate
    self._validate(data, schema)
  File "/usr/local/lib/python3.3/dist-packages/validictory/validator.py", line 587, in _validate
    self.__validate("_data", {"_data": data}, schema)
  File "/usr/local/lib/python3.3/dist-packages/validictory/validator.py", line 618, in __validate
    newschema.get(schemaprop))
  File "/usr/local/lib/python3.3/dist-packages/validictory/validator.py", line 224, in validate_type
    value, fieldname, fieldtype=fieldtype)
  File "/usr/local/lib/python3.3/dist-packages/validictory/validator.py", line 160, in _error
    raise FieldValidationError(message, fieldname, value)
validictory.validator.FieldValidationError: Value 1 for field '_data' is not of type string

I suppose python3 makes funnier things with the dictionary order than python2, as I haven't seen this error before.
Can you force the type validation to be done before the format validation?

I'm using python 3.3.2-14ubuntu1 on ubuntu trusty. Please let me know if I can help collecting more information.

Additional Schema Options

Is it possible to add a schema option to verify for example if the data is unique inside the database?
Example :

schema = {
    'type' : 'object',
    'properties' : {
        {'name' : {'type' : 'string', 'dbunique' : 'true'}
    }
}

data = {'name' : 'john'}

try:
    validictory.validate(data, schema)
except ValueError, error:
    print error

Value 'john' is not unique inside the database

handling $ref pointers to other schemas?

Hi James and Sunlight Labs,

Is there support for $ref pointers in validictory? This is defined in draft-03 of the json-schema spec. It allows a schema to define types by referring to a schema at a URL.

Here is an example of such a schema:

http://trec-kba.org/schemas/v1.0/stream-item.json

which has a $ref to other schemas, like this one:

http://trec-kba.org/schemas/v1.0/content-item.json

jrf

Change the defaults for "Schema Options"

How do I change the defaults for certain schema options?

For Example the documentation about the 'blank' schema option states:

'blank'
If False, validate that string values are not blank (the empty string).
The default value of this parameter is set when initializing SchemaValidator. 
By default it is False.

How do I set the default for blank to "True"?

Some of the json data I need to process has lots of null and optional values. I, unfortunately, have no control over this.

JSON-Schema Test Suite

I quickly ran the "JSON-Schema-Test-Suite" that apparently many projects are using:
https://github.com/json-schema/JSON-Schema-Test-Suite:

Ran 245 tests in 0.147 seconds
FAILED (failures=37, errors=107, skipped=0)

I can share the results and commit my code if anybody is interested. Most cases fail for good reasons I think.

What are your opinions about a test suite like this?

I think it would be good to know which features from the specification are supported and which are not.

A way to see all errors, not only the first

I like this library!

But, it would really be nice if I could see all of the applicable errors in the input, rather than only the first.

Is there a way to validate datetime objects?

TypeError during null date

A type error is being raised when validating this property:

...
"shipDate": null, 
...

against this schema:

...
"shipDate": {
    "required": true, 
    "type": [
        "string", 
        null
    ], 
    "format": "date"
}, 
...

stack trace:

...
  File "/Library/Python/2.6/site-packages/validictory/__init__.py", line 26, in validate
    return v.validate(data,schema)
  File "/Library/Python/2.6/site-packages/validictory/validator.py", line 454, in validate
    self._validate(data, schema)
  File "/Library/Python/2.6/site-packages/validictory/validator.py", line 457, in _validate
    self.__validate("_data", {"_data": data}, schema)
  File "/Library/Python/2.6/site-packages/validictory/validator.py", line 486, in __validate
    newschema.get(schemaprop))
  File "/Library/Python/2.6/site-packages/validictory/validator.py", line 151, in validate_properties
    properties.get(eachProp))
  File "/Library/Python/2.6/site-packages/validictory/validator.py", line 486, in __validate
    newschema.get(schemaprop))
  File "/Library/Python/2.6/site-packages/validictory/validator.py", line 151, in validate_properties
    properties.get(eachProp))
  File "/Library/Python/2.6/site-packages/validictory/validator.py", line 486, in __validate
    newschema.get(schemaprop))
  File "/Library/Python/2.6/site-packages/validictory/validator.py", line 257, in validate_additionalProperties
    additionalProperties)
  File "/Library/Python/2.6/site-packages/validictory/validator.py", line 486, in __validate
    newschema.get(schemaprop))
  File "/Library/Python/2.6/site-packages/validictory/validator.py", line 151, in validate_properties
    properties.get(eachProp))
  File "/Library/Python/2.6/site-packages/validictory/validator.py", line 486, in __validate
    newschema.get(schemaprop))
  File "/Library/Python/2.6/site-packages/validictory/validator.py", line 354, in validate_format
    format_validator(self, fieldname, value, format_option)
  File "/Library/Python/2.6/site-packages/validictory/validator.py", line 20, in validate_format_datetime
    datetime.strptime(value, dateformat_string)
TypeError: strptime() argument 1 must be string, not None

Validation error displayed when data doesn't match any of several types is hard to decipher and not very useful

I've noticed that when a schema has a list of possible types and the data doesn't match any of them, the error message is not terribly helpful -- it just says that the value doesn't match the schema and leaves it to the user to figure out why.

An example:

Value {'a': 1} for field '_data' doesn't match any of 2 subtypes in [{'type': 'object', 'properties': {'a': {'type': 'integer'}, 'b': {'type': 'integer'}}}, {'type': 'object', 'properties': {'a': {'type': 'integer'}, 'c': {'type': 'integer'}}}]

This is hard to read, because long JSON schemas that are not indented are hard to read (for me at least). It also doesn't tell you that the problem is that b or c is missing -- the user has to inspect the schema themselves to determine this.

If we indent the schema, it's a little easier to see what's going on, although the user still has to inspect it to see that b or c is missing.

Value {'a': 1} for field '_data' is not of type 
[
    {'type': 'object', 'properties': {'a': {'type': 'integer'}, 'b': {'type': 'integer'}}},
    {'type': 'object', 'properties': {'a': {'type': 'integer'}, 'c': {'type': 'integer'}}}
]

This is from https://gist.github.com/msabramo/4963773; which is a small test program that illustrates that the error messages are very useful when matching against a single type, but not so useful when matching against a list of types.

I am not sure exactly of how to provide a message that is useful and succint -- this is kind of hard because it's difficult for validictory to know which type the data is closest to. One idea is to have the error message include a list of all of the validation errors for each of the "sub types" -- I will be sending a PR soon that illustrates this approach.

Validation technique

What's the proper way to report errors back to the client? If I'm using regexs - I don't think the user would like to see the regex in his error.

Where in the structure is the error?

FieldValidationErrors have a fieldname, which doesn't reflect where in the structure something went wrong.

I hope this illustrates my point: https://gist.github.com/edmund-huber/5835626 . If I had more than one field named 'ho' I wouldn't know which one went wrong.

JSONSchema

What is validictory's policy about compliance with JSONSchema specification(s)?

Docs mention that "Schema format is based on the JSON Schema proposal" but is it strictly compliant or has it's own variation?

patternProperties failing

Hello.

I'm using this code from validictory tests:

import validictory
schema = { "patternProperties": {"[abc]": {"type": "boolean"}}}
data = { "a":True, "d":"foo"}
validictory.validate(data, schema)

If I understand correctly, it should not validate this because d is not in [abc] and value type of d is string, not boolean. Homewer, it validates fine. I'm using validictory 1.0.0 from PyPi

Property becomes required if "format" option is present

Solved by adding a check for the value in validate_format_ *

Unit test code:

import unittest
import validictory

class FormatTestCase(unittest.TestCase):

    def setUp(self):
        self.schema = {
            u'type': u'object',
            u'properties': {
                u'startdate': {u'type': u'string', u'format': u'date-time', u'required': False}
            }
        }

    def testFormat(self):
        validictory.validate({}, self.schema, required_by_default=False)
        self.assertTrue(True)

if __name__ == u'__main__':
    unittest.main()

Correct licence is "Expat"

MIT Licence is ambiguous. The correct name of the licence you are using is "Expat".
https://directory.fsf.org/wiki/License:Expat

Please correct this in setup.py. Thanks

Please move tests out of package directory

I tied packaging validoctory for guix, but running the test failed. The reasons according to pytest good practice are:

• tests are places within the package directory
• there is an __init__.py in the tests directory.

How to test an array with N items that validate against M schemas

I'm trying to validate an array that can have any number of items in it. Each of these items can be one of 5 or 6 schemas. So, for example:

schemaPoint = {
    "type": "object",
    "properties": {
        "type": {
            "pattern": "Point"
        },
        "coordinates": {
            "type": "array"
            "items": {
                "type": "array",
                "minItems": 2,
                "maxItems": 3
            }
        }
    }
}

schemaLineString = {
    "type": "object",
    "properties": {
        "type": {
            "pattern": "LineString"
        },
        "coordinates": {
            "type": "array",
            "minItems": 2,
            "items": {
                "type": "array",
                "minItems": 2,
                "maxItems": 3
            }
        }
    }
}

There are a number of other schemas, but I'm omitting them for brevity.

So, what I need to do is create a GeometryCollection schema that has a geometries property that is an array. This array should contain items like schemaPoint, schemaLineString and quite a few others I've not shown here.

My first attempt is something like the following:

schemaGeometryCollection = {
    "type": "object",
    "properties": {
        "type": {
            "pattern": "GeometryCollection"
        },
        "geometries": {
            "type": "array",
            "items": [schemaPoint, schemaLineString]
        }
    }
}

Essentially what I want in the above is for the geometries array above to have any number of items in it, so long as each of those items matches either schemaPoint or schemaLineString.

Am I making any sense?

Need help interpreting validictory.validator.FieldValidationError

I'm getting an error:

validictory.validator.FieldValidationError: Length of list  <the json data>  for field '_data' is not equal to length of schema list

Inside of my json data there does not exist a field named '_data'. Could you help dispel my confusion?

The length of the array of json objects could be quite large.

Lots more detail including: Stacktrace, Python Code, and JSON file.
Here is my Error Message:

Traceback (most recent call last):
  File "~/get_events/json/group_json_schema.py", line 88, in <module>
    validictory.validate(sample_group_data, group_schema)
  File "C:\Python27\lib\site-packages\validictory\__init__.py", line 36, in validate
    return v.validate(data, schema)
  File "C:\Python27\lib\site-packages\validictory\validator.py", line 584, in validate
    self._validate(data, schema)
  File "C:\Python27\lib\site-packages\validictory\validator.py", line 587, in _validate
    self.__validate("_data", {"_data": data}, schema)
  File "C:\Python27\lib\site-packages\validictory\validator.py", line 618, in __validate
    newschema.get(schemaprop))
  File "C:\Python27\lib\site-packages\validictory\validator.py", line 260, in validate_items
    "of schema list", value, fieldname)
  File "C:\Python27\lib\site-packages\validictory\validator.py", line 160, in _error
    raise FieldValidationError(message, fieldname, value)
validictory.validator.FieldValidationError: Length of list [{u'category': {u'shortname': u'Spirituality', u'id': 22, u'name': u'New Age & Spirituality'}, u'city': u'Sedona', u'join_mode': u'open', u'lat': 34.790000915527344, u'description': u'<p>The purpose of CENTER SPACE Sedona is to foster and facilitate spiritual, personal and community empowerment and enlightenment. <a href="http://www.centerspace.com">CENTER SPACE</a> is a non-profit membership organization that has been publishing, producing and sharing publications, seminars and membership services since 1985.</p>\n<p>For examples of prior programs and events featuring world-class authors and trainers for up to 1,800 participants, see <a href="http://normanvincentpeale.wordpress.com" target="_blank"></a><a href="http://normanvincentpeale.wordpress.com" target="_blank"></a><a href="http://normanvincentpeale.wordpress.com" target="_blank"></a><a href="http://normanvincentpeale.wordpress.com" target="_blank"></a><a href="http://normanvincentpeale.wordpress.com" target="_blank"></a><a href="http://normanvincentpeale.wordpress.com" target="_blank"><a href="http://normanvincentpeale.wordpress.com" title="http://normanvincentpeale.wordpress.com" target="_blank">http://normanvincentpeale.wordpress.com</a></a>, <a href="http://www.candoresourcecenter.com">www.candoresourcecenter.com</a> and <a href="http://www.seedonow.com">www.seedonow.com</a>.</p>\n<p>Formal establishment of a Sedona Chapter was a result of the new S<a href="http://www.sedonacollaborativeenterprises.com">edona Collaborative Enterprises</a> (www.sedonacollaborativeenterprises.com). Participants were invited to share their priority interests in six main project areas that follow in the order of partcipants\' priorities including: #1 Fostering Spiritual and Community Enlightenment; #2 <a href="http://www.sedonacollaborativeenterprises.com/intentional_communities.htm">Cocreating Intentional Community</a>;  #3<a href="http://www.sedonacollaborativeenterprises.com/retreat_centers.htm"> Cocreating a Retreat Center</a>; #4 <a href="http://www.usmark.org">Establishing a Public Company for Empowerment and Enlightenment</a>; #5 Expanding the <a href="http://www.ultimatedestinyuniversity.org">Ultimate Destiny University for Successful Living</a>; and #6 <a href="http://www.sedonacollaborativeenterprises.com/series_llc.htm">Creating a new Series Limited Liability Company</a> to attract funding for various for-profit ventures.</p>\n<p>As the top shared interest was spiritual, personal and community enlightenment, we decided to establish a Meet Up group as part of the process of sharing our vision, sense of mission and invite any other individuals and organizations who resonate with them to participate.</p>\n<p>If you are interested in spiritually centered approaches to solving the personal, community, national and global "success puzzles" we face including social, economic and environmental challenges, please do join this Meet Up and share your wisdom, expertise and experiences with us.</p>\n<p>Thank you for taking time to visit and review what we have shared so far.</p>\n<p>PS. If you would like an overview of how we are helping introduce the spiritual principles and practices known as New Thought, please visit <a href="http://www.universalcenterforspiritualliving.org">Universal Center for Spiritual Living</a>. In addition,&nbsp; the more ulimate vision we are participating in to help Solve Our Personal, Community and Global Success Puzzles is shared at <a href="http://www.ultimatesuccesspuzzle.com">Ultimate Success Puzzle.com</a></p>', u'created': 1288380544000L, u'country': u'US', u'who': u'Members', u'lon': -111.76000213623047, u'visibility': u'public', u'photos': [{u'thumb_link': u'http://photos3.meetupstatic.com/photos/event/1/3/9/4/thumb_337385012.jpeg', u'highres_link': u'http://photos3.meetupstatic.com/photos/event/1/3/9/4/highres_337385012.jpeg', u'photo_link': u'http://photos3.meetupstatic.com/photos/event/1/3/9/4/600_337385012.jpeg', u'id': 337385012}], u'state': u'AZ', u'group_photo': {u'thumb_link': u'http://photos4.meetupstatic.com/photos/event/5/2/3/8/thumb_19161048.jpeg', u'highres_link': u'http://photos4.meetupstatic.com/photos/event/5/2/3/8/highres_19161048.jpeg', u'photo_link': u'http://photos4.meetupstatic.com/photos/event/5/2/3/8/600_19161048.jpeg', u'id': 19161048}, u'link': u'http://www.meetup.com/Center-for-Spiritual-Personal-And-Community-Enlightenment/', u'members': 24, u'urlname': u'Center-for-Spiritual-Personal-And-Community-Enlightenment', u'next_event': {u'time': 1398303000000L, u'id': u'177972502', u'name': u'Exploring Possible Collaborative StewardHeirShip of 134 Acre Soda Springs Ranch'}, u'timezone': u'US/Arizona', u'organizer': {u'bio': u'I have lived in intentional communities for 25 years, have an MS in Community Economic Development, founded non-profits & companies that provide resources for personal, community & spiritual enlightenment. Seeking like hearted individuals in Sedona.', u'id': 8043119, u'name': u'Charles'}, u'id': 1730140, u'name': u'Center for Spiritual, Personal And Community Enlightenment'}, {u'category': {u'shortname': u'Social', u'id': 31, u'name': u'Socializing'}, u'city': u'Sedona', u'join_mode': u'open', u'lat': 34.790000915527344, u'description': u"<p>Wild Boomer Women is an instant community of girlfriends, events and activities for women over 40.</p>\n<p>Join us on the journey of getting out and having fun. Don't stay home just because you can't find other girlfriends to go out with, or you have other girlfriends but they don't seem to want to have fun. Meet new like minded girlfriends.</p>\n<p>We have been having fun in the greater Phoenix area since 2008 and formed a private club called The Bucket List Club in January, 2012 - we now have over 320 members. \xa0You can also join that group and attend events if you ever go down the hill.</p>\n<p>Please visit our website at wildboomerwomen.com to find out more about the group and get on our mailing list.</p>\n<p>HERE ARE THE REQUIREMENTS TO JOIN THIS GROUP: You must be open to new adventures, be a postivie thinker, outgoing, willing to share about yourself and open to having fun to join this group. If you are a negative person, complain or gossip, this group is not for you. This is not just a movie group or a dinner group - although we may do these things too. This is for women seeking adventure and fun. It is also not a business networking group although that sometimes happens naturally. Our main mission is all about having fun. Edit description</p>", u'created': 1245122691000L, u'country': u'US', u'who': u'Wild Boomer Women', u'lon': -111.76000213623047, u'visibility': u'public', u'photos': [{u'thumb_link': u'http://photos2.meetupstatic.com/photos/event/b/5/9/0/thumb_10546480.jpeg', u'highres_link': u'http://photos2.meetupstatic.com/photos/event/b/5/9/0/highres_10546480.jpeg', u'photo_link': u'http://photos4.meetupstatic.com/photos/event/b/5/9/0/600_10546480.jpeg', u'id': 10546480}, {u'thumb_link': u'http://photos2.meetupstatic.com/photos/event/b/5/9/1/thumb_10546481.jpeg', u'highres_link': u'http://photos4.meetupstatic.com/photos/event/b/5/9/1/highres_10546481.jpeg', u'photo_link': u'http://photos4.meetupstatic.com/photos/event/b/5/9/1/600_10546481.jpeg', u'id': 10546481}, {u'thumb_link': u'http://photos3.meetupstatic.com/photos/event/a/8/b/e/thumb_17263198.jpeg', u'highres_link': u'http://photos3.meetupstatic.com/photos/event/a/8/b/e/highres_17263198.jpeg', u'photo_link': u'http://photos3.meetupstatic.com/photos/event/a/8/b/e/600_17263198.jpeg', u'id': 17263198}, {u'thumb_link': u'http://photos4.meetupstatic.com/photos/event/a/a/f/f/thumb_17263775.jpeg', u'highres_link': u'http://photos2.meetupstatic.com/photos/event/a/a/f/f/highres_17263775.jpeg', u'photo_link': u'http://photos4.meetupstatic.com/photos/event/a/a/f/f/600_17263775.jpeg', u'id': 17263775}], u'state': u'AZ', u'group_photo': {u'thumb_link': u'http://photos4.meetupstatic.com/photos/event/b/a/9/2/thumb_147467762.jpeg', u'highres_link': u'http://photos2.meetupstatic.com/photos/event/b/a/9/2/highres_147467762.jpeg', u'photo_link': u'http://photos4.meetupstatic.com/photos/event/b/a/9/2/600_147467762.jpeg', u'id': 147467762}, u'link': u'http://www.meetup.com/Wild-Boomer-Women-Sedona/', u'members': 174, u'urlname': u'Wild-Boomer-Women-Sedona', u'next_event': {u'time': 1398391200000L, u'id': u'171999522', u'name': u'10th ANNUAL FINDING THE GIFT IN SHIFT - SAVE $50!'}, u'timezone': u'US/Arizona', u'organizer': {u'bio': u"I'm a baby boomer, divorced, love life, and love to have fun.  I have a women's social club called Wild Boomer Women.  Check us out wildboomerwomen.com and find out how you can join the fun.", u'id': 3878002, u'name': u'Sue'}, u'id': 1473727, u'name': u'Wild Boomer Women (tm) Sedona'}, {u'category': {u'shortname': u'Community', u'id': 4, u'name': u'Community & Environment'}, u'city': u'Sedona', u'join_mode': u'open', u'lat': 34.790000915527344, u'description': u'<p>The Sedona Collaborative Enterprises Resource Center provides expanded access to resources that help individuals, organizations and communities realize more of their potential, partly by practicing the principles of cooperation, community and collaboration. A couple of previous initiatives for fostering collaboration preceded formation of This MeetUp Group.</p>\n<p>They include an earlier blog site at <a href="http://sedonacollaborative.wordpress.com/" target="_blank">http://sedonacollaborative.wordpress.com/</a> and a web site at&nbsp;&nbsp;&nbsp;&nbsp; <a href="http://www.sedonacollaborativeenterprises.com/" target="_blank">http://www.sedonacollaborativeenterprises.com/</a>. The most recent development that led to this meetUp is the invitation for a new cause-oriented company, Universal Stewardheirship, Inc. <a href="http://www.universalstewardheirship.com/invitation.htm" target="_blank">www.universalstewardheirship.com/invitation.htm</a><br>\n\n<br>\n\n You may also enjoy reviewing the Interest Survey at <a href="http://survey.constantcontact.com/survey/a07e32pl99agfew39sw/start" target="_blank">http://survey.constantcontact.com/survey/a07e32pl99agfew39sw/start</a></p>\n<p>This new Meet Up group is based on about 40 years of prior experience cocreating model Community Resource Centers in conjunction with United Way Agencies, Community Development Corporations, Public Housing Authorities, churches, communities and other non-profit organizations.</p>\n<p>The Meet Up group members will have access to hundreds of programs, products and services from our cofounding organizations and sponsors.They include <a href="http://www.ultimatedestinyuniversity.org">Ultimate Destiny University for Successful Living</a>, <a href="http://www.centerspace.com">CENTER SPACE</a> (the Center for Spiritual, Personal And Community Empowerment and Enlightenment) and <a href="http://www.candoresourcecenter.com">CAN DO!</a></p>\n<p>In addition to the main Meet Up based in Sedona, AZ, local groups anywhere will be able to provide their members with an ongoing calendar of seminars, workshops, playshops, webinars, membership services, coaching and mentoring services, and access to income producing components including affiliate marketing programs that will earn money for local Meet Up Groups. Just a few examples of income generating affiliate programs are described at <a href="http://www.ultimatepublishingandmarketingresources.com">Ultimate Publishing and marketing Resources.com</a></p>\n<p>&nbsp;</p>\n<p>&nbsp;</p>', u'created': 1290452969000L, u'country': u'US', u'who': u'Members', u'lon': -111.76000213623047, u'visibility': u'public', u'photos': [{u'thumb_link': u'http://photos1.meetupstatic.com/photos/event/4/6/0/2/thumb_336437922.jpeg', u'highres_link': u'http://photos1.meetupstatic.com/photos/event/4/6/0/2/highres_336437922.jpeg', u'photo_link': u'http://photos1.meetupstatic.com/photos/event/4/6/0/2/600_336437922.jpeg', u'id': 336437922}, {u'thumb_link': u'http://photos3.meetupstatic.com/photos/event/1/4/e/8/thumb_337385352.jpeg', u'highres_link': u'http://photos3.meetupstatic.com/photos/event/1/4/e/8/highres_337385352.jpeg', u'photo_link': u'http://photos3.meetupstatic.com/photos/event/1/4/e/8/600_337385352.jpeg', u'id': 337385352}], u'state': u'AZ', u'group_photo': {u'thumb_link': u'http://photos2.meetupstatic.com/photos/event/1/2/2/c/thumb_63004652.jpeg', u'highres_link': u'http://photos2.meetupstatic.com/photos/event/1/2/2/c/highres_63004652.jpeg', u'photo_link': u'http://photos2.meetupstatic.com/photos/event/1/2/2/c/600_63004652.jpeg', u'id': 63004652}, u'link': u'http://www.meetup.com/Sedona-Collaborative-Enterprises/', u'members': 11, u'urlname': u'Sedona-Collaborative-Enterprises', u'next_event': {u'time': 1398303000000L, u'id': u'177961192', u'name': u'Exploring Possible Collaborative StewardHeirShip of 134 Acre Soda Springs Ranch'}, u'timezone': u'US/Arizona', u'organizer': {u'bio': u'I have lived in intentional communities for 25 years, have an MS in Community Economic Development, founded non-profits & companies that provide resources for personal, community & spiritual enlightenment. Seeking like hearted individuals in Sedona.', u'id': 8043119, u'name': u'Charles'}, u'id': 1741533, u'name': u'Sedona Collaborative Enterprises Resource Center'}] for field '_data' is not equal to length of schema list

Process finished with exit code 1

Here is my code including my schema. Keep in mind that the array from the file only contains 3 json objects.

import json
import validictory


# Open sample group json data from meetup.
with open(u'correct_meetup_group_data.json', 'r') as json_data:
    sample_group_data = json.load(json_data)

group_schema = {
    "title": "Meetup Get/2/Groups Schema",
    "description": "Meetup Groups API version 2 http://www.meetup.com/meetup_api/docs/2/groups/",

    "type": "array",
    "items": [{
        "type": "object",
        "properties": {
            "category": {
                "type": "object",
                "properties": {
                     "shortname": {"type": "string"},
                     "id": {"type": "integer"},
                     "name": {"type": "string"},
                 }
            },
            "city": {"type": "string"},
            "who": {"type": "string"},
            "description": {"type": "string"},
            "created": {"type": "integer"},
            "country": {"type": "string"},
            "photos": {
                 "items":[{
                         "type": "object",
                          "properties": {
                              "photo_link": {"type": "string"},
                              "thumb_link": {"type": "string"},
                              "id": {"type": "integer"},
                              "highres_link": {"type": "string"},
                          },
                }]
             },
            "join_mode": {"type": "string"},
            "lon": {"type": "number"},
            "visibility": "string",
            "next_event": {
                "type": "object",
                "properties": {
                    "id": {"type": "integer"},
                    "name": {"type": "string"},
                    "time": {"type": "integer"},
                }
            },
            "state": {"type": "string"},
            "group_photo": {
                "type": "object",
                "properties":{
                        "photo_link": {"type": "string"},
                        "thumb_link": {"type": "string"},
                        "id": {"type": "integer"},
                        "highres_link": {"type": "string"},
                }
            },
            "link": {"type": "string"},
            "members": {"type": "integer"},
            "timezone": {"type": "string"},
            "lat": {"type": "number"},
            "urlname": {"type": "string"},
            "organizer": {
                "type": "object",
                "properties":{
                    "bio": {"type": "string"},
                    "id": {"type": "integer"},
                    "name": {"type": "string"},
                }
            },
            "id": {"type": "integer"},
            "name": {"type": "string"},
        }
    }]
}

validictory.validate(sample_group_data, group_schema)

Here are the contents for the file 'correct_meetup_group_data.json'

[{"category": {"shortname": "Spirituality", "id": 22, "name": "New Age & Spirituality"}, "city": "Sedona", "who": "Members", "description": "<p>The purpose of CENTER SPACE Sedona is to foster and facilitate spiritual, personal and community empowerment and enlightenment. <a href=\"http://www.centerspace.com\">CENTER SPACE</a> is a non-profit membership organization that has been publishing, producing and sharing publications, seminars and membership services since 1985.</p>\n<p>For examples of prior programs and events featuring world-class authors and trainers for up to 1,800 participants, see <a href=\"http://normanvincentpeale.wordpress.com\" target=\"_blank\"></a><a href=\"http://normanvincentpeale.wordpress.com\" target=\"_blank\"></a><a href=\"http://normanvincentpeale.wordpress.com\" target=\"_blank\"></a><a href=\"http://normanvincentpeale.wordpress.com\" target=\"_blank\"></a><a href=\"http://normanvincentpeale.wordpress.com\" target=\"_blank\"></a><a href=\"http://normanvincentpeale.wordpress.com\" target=\"_blank\"><a href=\"http://normanvincentpeale.wordpress.com\" title=\"http://normanvincentpeale.wordpress.com\" target=\"_blank\">http://normanvincentpeale.wordpress.com</a></a>, <a href=\"http://www.candoresourcecenter.com\">www.candoresourcecenter.com</a> and <a href=\"http://www.seedonow.com\">www.seedonow.com</a>.</p>\n<p>Formal establishment of a Sedona Chapter was a result of the new S<a href=\"http://www.sedonacollaborativeenterprises.com\">edona Collaborative Enterprises</a> (www.sedonacollaborativeenterprises.com). Participants were invited to share their priority interests in six main project areas that follow in the order of partcipants' priorities including: #1 Fostering Spiritual and Community Enlightenment; #2 <a href=\"http://www.sedonacollaborativeenterprises.com/intentional_communities.htm\">Cocreating Intentional Community</a>;  #3<a href=\"http://www.sedonacollaborativeenterprises.com/retreat_centers.htm\"> Cocreating a Retreat Center</a>; #4 <a href=\"http://www.usmark.org\">Establishing a Public Company for Empowerment and Enlightenment</a>; #5 Expanding the <a href=\"http://www.ultimatedestinyuniversity.org\">Ultimate Destiny University for Successful Living</a>; and #6 <a href=\"http://www.sedonacollaborativeenterprises.com/series_llc.htm\">Creating a new Series Limited Liability Company</a> to attract funding for various for-profit ventures.</p>\n<p>As the top shared interest was spiritual, personal and community enlightenment, we decided to establish a Meet Up group as part of the process of sharing our vision, sense of mission and invite any other individuals and organizations who resonate with them to participate.</p>\n<p>If you are interested in spiritually centered approaches to solving the personal, community, national and global \"success puzzles\" we face including social, economic and environmental challenges, please do join this Meet Up and share your wisdom, expertise and experiences with us.</p>\n<p>Thank you for taking time to visit and review what we have shared so far.</p>\n<p>PS. If you would like an overview of how we are helping introduce the spiritual principles and practices known as New Thought, please visit <a href=\"http://www.universalcenterforspiritualliving.org\">Universal Center for Spiritual Living</a>. In addition,&nbsp; the more ulimate vision we are participating in to help Solve Our Personal, Community and Global Success Puzzles is shared at <a href=\"http://www.ultimatesuccesspuzzle.com\">Ultimate Success Puzzle.com</a></p>", "created": 1288380544000, "country": "US", "photos": [{"photo_link": "http://photos3.meetupstatic.com/photos/event/1/3/9/4/600_337385012.jpeg", "thumb_link": "http://photos3.meetupstatic.com/photos/event/1/3/9/4/thumb_337385012.jpeg", "id": 337385012, "highres_link": "http://photos3.meetupstatic.com/photos/event/1/3/9/4/highres_337385012.jpeg"}], "join_mode": "open", "lon": -111.76000213623047, "visibility": "public", "next_event": {"id": "177972502", "name": "Exploring Possible Collaborative StewardHeirShip of 134 Acre Soda Springs Ranch", "time": 1398303000000}, "state": "AZ", "group_photo": {"photo_link": "http://photos4.meetupstatic.com/photos/event/5/2/3/8/600_19161048.jpeg", "thumb_link": "http://photos4.meetupstatic.com/photos/event/5/2/3/8/thumb_19161048.jpeg", "id": 19161048, "highres_link": "http://photos4.meetupstatic.com/photos/event/5/2/3/8/highres_19161048.jpeg"}, "link": "http://www.meetup.com/Center-for-Spiritual-Personal-And-Community-Enlightenment/", "members": 24, "timezone": "US/Arizona", "lat": 34.790000915527344, "urlname": "Center-for-Spiritual-Personal-And-Community-Enlightenment", "organizer": {"bio": "I have lived in intentional communities for 25 years, have an MS in Community Economic Development, founded non-profits & companies that provide resources for personal, community & spiritual enlightenment. Seeking like hearted individuals in Sedona.", "id": 8043119, "name": "Charles"}, "id": 1730140, "name": "Center for Spiritual, Personal And Community Enlightenment"}, {"category": {"shortname": "Social", "id": 31, "name": "Socializing"}, "city": "Sedona", "who": "Wild Boomer Women", "description": "<p>Wild Boomer Women is an instant community of girlfriends, events and activities for women over 40.</p>\n<p>Join us on the journey of getting out and having fun. Don't stay home just because you can't find other girlfriends to go out with, or you have other girlfriends but they don't seem to want to have fun. Meet new like minded girlfriends.</p>\n<p>We have been having fun in the greater Phoenix area since 2008 and formed a private club called The Bucket List Club in January, 2012 - we now have over 320 members. \u00a0You can also join that group and attend events if you ever go down the hill.</p>\n<p>Please visit our website at wildboomerwomen.com to find out more about the group and get on our mailing list.</p>\n<p>HERE ARE THE REQUIREMENTS TO JOIN THIS GROUP: You must be open to new adventures, be a postivie thinker, outgoing, willing to share about yourself and open to having fun to join this group. If you are a negative person, complain or gossip, this group is not for you. This is not just a movie group or a dinner group - although we may do these things too. This is for women seeking adventure and fun. It is also not a business networking group although that sometimes happens naturally. Our main mission is all about having fun. Edit description</p>", "created": 1245122691000, "country": "US", "photos": [{"photo_link": "http://photos4.meetupstatic.com/photos/event/b/5/9/0/600_10546480.jpeg", "thumb_link": "http://photos2.meetupstatic.com/photos/event/b/5/9/0/thumb_10546480.jpeg", "id": 10546480, "highres_link": "http://photos2.meetupstatic.com/photos/event/b/5/9/0/highres_10546480.jpeg"}, {"photo_link": "http://photos4.meetupstatic.com/photos/event/b/5/9/1/600_10546481.jpeg", "thumb_link": "http://photos2.meetupstatic.com/photos/event/b/5/9/1/thumb_10546481.jpeg", "id": 10546481, "highres_link": "http://photos4.meetupstatic.com/photos/event/b/5/9/1/highres_10546481.jpeg"}, {"photo_link": "http://photos3.meetupstatic.com/photos/event/a/8/b/e/600_17263198.jpeg", "thumb_link": "http://photos3.meetupstatic.com/photos/event/a/8/b/e/thumb_17263198.jpeg", "id": 17263198, "highres_link": "http://photos3.meetupstatic.com/photos/event/a/8/b/e/highres_17263198.jpeg"}, {"photo_link": "http://photos4.meetupstatic.com/photos/event/a/a/f/f/600_17263775.jpeg", "thumb_link": "http://photos4.meetupstatic.com/photos/event/a/a/f/f/thumb_17263775.jpeg", "id": 17263775, "highres_link": "http://photos2.meetupstatic.com/photos/event/a/a/f/f/highres_17263775.jpeg"}], "join_mode": "open", "lon": -111.76000213623047, "visibility": "public", "next_event": {"id": "171999522", "name": "10th ANNUAL FINDING THE GIFT IN SHIFT - SAVE $50!", "time": 1398391200000}, "state": "AZ", "group_photo": {"photo_link": "http://photos4.meetupstatic.com/photos/event/b/a/9/2/600_147467762.jpeg", "thumb_link": "http://photos4.meetupstatic.com/photos/event/b/a/9/2/thumb_147467762.jpeg", "id": 147467762, "highres_link": "http://photos2.meetupstatic.com/photos/event/b/a/9/2/highres_147467762.jpeg"}, "link": "http://www.meetup.com/Wild-Boomer-Women-Sedona/", "members": 174, "timezone": "US/Arizona", "lat": 34.790000915527344, "urlname": "Wild-Boomer-Women-Sedona", "organizer": {"bio": "I'm a baby boomer, divorced, love life, and love to have fun.  I have a women's social club called Wild Boomer Women.  Check us out wildboomerwomen.com and find out how you can join the fun.", "id": 3878002, "name": "Sue"}, "id": 1473727, "name": "Wild Boomer Women (tm) Sedona"}, {"category": {"shortname": "Community", "id": 4, "name": "Community & Environment"}, "city": "Sedona", "who": "Members", "description": "<p>The Sedona Collaborative Enterprises Resource Center provides expanded access to resources that help individuals, organizations and communities realize more of their potential, partly by practicing the principles of cooperation, community and collaboration. A couple of previous initiatives for fostering collaboration preceded formation of This MeetUp Group.</p>\n<p>They include an earlier blog site at <a href=\"http://sedonacollaborative.wordpress.com/\" target=\"_blank\">http://sedonacollaborative.wordpress.com/</a> and a web site at&nbsp;&nbsp;&nbsp;&nbsp; <a href=\"http://www.sedonacollaborativeenterprises.com/\" target=\"_blank\">http://www.sedonacollaborativeenterprises.com/</a>. The most recent development that led to this meetUp is the invitation for a new cause-oriented company, Universal Stewardheirship, Inc. <a href=\"http://www.universalstewardheirship.com/invitation.htm\" target=\"_blank\">www.universalstewardheirship.com/invitation.htm</a><br>\n\n<br>\n\n You may also enjoy reviewing the Interest Survey at <a href=\"http://survey.constantcontact.com/survey/a07e32pl99agfew39sw/start\" target=\"_blank\">http://survey.constantcontact.com/survey/a07e32pl99agfew39sw/start</a></p>\n<p>This new Meet Up group is based on about 40 years of prior experience cocreating model Community Resource Centers in conjunction with United Way Agencies, Community Development Corporations, Public Housing Authorities, churches, communities and other non-profit organizations.</p>\n<p>The Meet Up group members will have access to hundreds of programs, products and services from our cofounding organizations and sponsors.They include <a href=\"http://www.ultimatedestinyuniversity.org\">Ultimate Destiny University for Successful Living</a>, <a href=\"http://www.centerspace.com\">CENTER SPACE</a> (the Center for Spiritual, Personal And Community Empowerment and Enlightenment) and <a href=\"http://www.candoresourcecenter.com\">CAN DO!</a></p>\n<p>In addition to the main Meet Up based in Sedona, AZ, local groups anywhere will be able to provide their members with an ongoing calendar of seminars, workshops, playshops, webinars, membership services, coaching and mentoring services, and access to income producing components including affiliate marketing programs that will earn money for local Meet Up Groups. Just a few examples of income generating affiliate programs are described at <a href=\"http://www.ultimatepublishingandmarketingresources.com\">Ultimate Publishing and marketing Resources.com</a></p>\n<p>&nbsp;</p>\n<p>&nbsp;</p>", "created": 1290452969000, "country": "US", "photos": [{"photo_link": "http://photos1.meetupstatic.com/photos/event/4/6/0/2/600_336437922.jpeg", "thumb_link": "http://photos1.meetupstatic.com/photos/event/4/6/0/2/thumb_336437922.jpeg", "id": 336437922, "highres_link": "http://photos1.meetupstatic.com/photos/event/4/6/0/2/highres_336437922.jpeg"}, {"photo_link": "http://photos3.meetupstatic.com/photos/event/1/4/e/8/600_337385352.jpeg", "thumb_link": "http://photos3.meetupstatic.com/photos/event/1/4/e/8/thumb_337385352.jpeg", "id": 337385352, "highres_link": "http://photos3.meetupstatic.com/photos/event/1/4/e/8/highres_337385352.jpeg"}], "join_mode": "open", "lon": -111.76000213623047, "visibility": "public", "next_event": {"id": "177961192", "name": "Exploring Possible Collaborative StewardHeirShip of 134 Acre Soda Springs Ranch", "time": 1398303000000}, "state": "AZ", "group_photo": {"photo_link": "http://photos2.meetupstatic.com/photos/event/1/2/2/c/600_63004652.jpeg", "thumb_link": "http://photos2.meetupstatic.com/photos/event/1/2/2/c/thumb_63004652.jpeg", "id": 63004652, "highres_link": "http://photos2.meetupstatic.com/photos/event/1/2/2/c/highres_63004652.jpeg"}, "link": "http://www.meetup.com/Sedona-Collaborative-Enterprises/", "members": 11, "timezone": "US/Arizona", "lat": 34.790000915527344, "urlname": "Sedona-Collaborative-Enterprises", "organizer": {"bio": "I have lived in intentional communities for 25 years, have an MS in Community Economic Development, founded non-profits & companies that provide resources for personal, community & spiritual enlightenment. Seeking like hearted individuals in Sedona.", "id": 8043119, "name": "Charles"}, "id": 1741533, "name": "Sedona Collaborative Enterprises Resource Center"}]

Unsupported format types: regex, color, style, phone, email, ip-address, ipv6, host-name