opennti/nti.schema

Enhancements for zope.schema

nti.schema

nti.schema includes utilities for working with schema-driven development using zope.schema.

For complete details and the changelog, see the documentation.

Overview

Some of the most useful features include:

• nti.schema.interfaces.find_most_derived_interface for finding a bounded interface.
• nti.schema.eqhash.EqHash is a class-decorator for creating efficient, correct implementations of equality and hashing.
• nti.schema.field contains various schema fields, including a Variant type and more flexible collection types, all of which produce better validation errors.
• nti.schema.fieldproperty contains field properties that can adapt to interfaces or decode incoming text. The function createDirectFieldProperties can assign just the necessary properties automatically.

nti.schema's Issues

nti.schema.jsonschema should turn abstract collections into concrete values

That makes the most sense for UIs and JSON. See
OpenNTI/nti.assessment#3 (comment)

DeprecationWarning: InvalidValue: Use zope.schema.interfaces.InvalidValue.with_field_and_value.

nti/schema/field.py:77: DeprecationWarning: InvalidValue: Use zope.schema.interfaces.InvalidValue.with_field_and_value.

SchemaConfigured try to set defaults for readonly fields (nti.schema-1.1.3)

in SchemaConfigured mixin class

The following code is executed

 for f, fields in _iteritems(schema):
          if not hasattr(self, f):
              setattr(self, f, fields.default)

We are getting a "AttributeError: can't set attribute" because it tries to set up a default to a readonly field

class IGradeBookEntry(IContainer,IContained):

    DueDate = Date(title="The date on which the assignment is due", 
                   required=False,
                   readonly=True)

@interface.implementer(IGradeBookEntry, IAttributeAnnotatable)
class GradeBookEntry(SchemaConfigured,
					 CheckingLastModifiedBTreeContainer,
					 Contained):
	createDirectFieldProperties(IGradeBookEntry)

ge  = GradeBookEntry()

Carlos

SchemaConfigured may be slower than necessary, especially when creating many of the same type object

In one internal API, creating 10K transient objects had the SchemaConfigured portion occupying roughly half the total time; many of those objects were of the same class. This needs a benchmark to prove and measure.

It looks like there are some low-hanging fruits that can be plucked, plus potentially some caching opportunities.

Nested field schema should use JsonSchemafier

When we attempt to get the schema of a field with an Interface value, we create a schemafier of the same type as self (link below). In our case, this is an AccountProfileSchemafier object that expects account profile fields in the result and attempts to edit/update them.

I think we should use the regular JsonSchemafier impl when getting field schemas.

https://github.com/NextThought/nti.schema/blob/master/src/nti/schema/jsonschema.py#L237

Return title/description in JsonSchemafier

It would be useful to return the fields' title/description in JsonSchemafier

Update tox.ini

We need a coverage environment (or more) that ensure at least one platform reaches 100% coverage by itself.

make nti.schema.interfaces.InvalidValue a class again

Changing to a function was a backwards-incompatible change; there are instances of subclassing or of catching/isinstance checks.

Incorrect plurality in some cases for fields with min length

When a field with a min length raises TooShort during validation, if the min length is 1, the plurality is incorrect, e.g. "Body is too short. Please use at least 1 characters."

Add support for ``IFromBytes``

in zope.schema 4.8.

Support Python 3

This will mean two things:

• Dropping dependency on plone.i18n
• Dropping dependency on dm.zope.schema.

Neither have been ported to Python 3 yet. We're pinned to an old version of plone.18n as it is.

Poor handling of Dict objects missing a key/value type inside a Variant -> AttributeError

Leading to nearly inscrutable tracebacks:

  File "/nti.schema-1.15.1-py2.7.egg/nti/schema/field.py", line 509, in fromObject
    return converter(obj)
  File "//nti.schema-1.15.1-py2.7.egg/nti/schema/field.py", line 362, in __call__
    return self.fromObject(value)
  File "//nti.schema-1.15.1-py2.7.egg/nti/schema/field.py", line 209, in _map_fromObject
    return _MapFromObject(field).fromObject(value)
  File "//nti.schema-1.15.1-py2.7.egg/nti/schema/field.py", line 803, in fromObject
    result = self._do_fromObject(context)
  File "//nti.schema-1.15.1-py2.7.egg/nti/schema/field.py", line 832, in _do_fromObject
    return {key_converter(k): value_converter(v) for k, v in _iteritems(context)}
  File "//nti.schema-1.15.1-py2.7.egg/nti/schema/field.py", line 832, in <dictcomp>
    return {key_converter(k): value_converter(v) for k, v in _iteritems(context)}
  File "//nti.schema-1.15.1-py2.7.egg/nti/schema/field.py", line 368, in __call__
    self.field.validate(value)
AttributeError: 'NoneType' object has no attribute 'validate'

We can detect this when creating the variant and it adds the map_fromObject method to Dict instances, and then either

1. fail the creation; or
2. fill in a do-nothing converter for the missing value; or
3. ignore both the key and value type and refuse to do any conversion; that's closest to what nti.schema < 1.7 did

I think the original intention may have been to ignore them both (choice 3), but now I prefer choice 1. Because this is happening inside a Variant, where we go with the first thing that fits, automatically filling in a do-nothing converter could be quite surprising. The user can fill in such a field themselves (Object(Interface)) if they want.

Better test coverage

Stop omitting tests from coverage measurement

They're important too. There have been numerous times I have found tests that weren't even executing only because of missing coverage.

BeforeSchemaFieldAssignedEvent uses FieldPropertyStoredThroughField mangled name

It is useful for subscribers on IBeforeSchemaFieldAssignedEvent to have access to the field name being set on the context. The event object has a name field but in the case of a FieldPropertyStoredThroughField this is a mangled name of the form __st_TermsAgreement_st not the attribute name for the field TermsAgreement.

I'm not sure what the best solution here is, we could:

a) unmangle the name when we create the event. This seems cleanest but there may be subscribers assuming something about the existing mangled names?
b) expose the guts of FieldValidationMixin.__fixup_name__ that subscribers of the event could use to cleanup the name before using it.
c) stash the unmangled name on the event as a new attribute.

@jamadden Thoughts? I lean towards option a or b.

Updates for zope.schema 4.6.0

I just pushed zope.schema 4.6.0 to PyPI. It has many, many changes, updates and fixes. I tested that they didn't actually break anything here, but much of what we do is now obsolete or can be optimized.

RuntimeError: maximum recursion depth exceeded while calling a Python object

Jason,

with the latest release of nti.schema, we are getting a maximum recursion depth error.. in one of our legacy class.

from nti.schema.interfaces import InvalidValue

class InvalidData(InvalidValue):
    """
    Invalid Value
    """

    i18n_message = None

    def __str__(self):
        if self.i18n_message:
            return translate(self.i18n_message)
        return super(InvalidData, self).__str__()

    def doc(self):
        if self.i18n_message:
            return self.i18n_message
        return self.__class__.__doc__

class EmailAddressInvalid(InvalidData):
    """
    Invalid email address.
    """

    i18n_message = _(u"The email address you have entered is not valid.")

    def __init__(self, address):
        super(EmailAddressInvalid, self).__init__(address, value=address)

we raise a EmailAddressInvalid(address) if an invalid address is given. but this generates the recursion error

I take we need to overwrite the with_field_and_value method?

Carlos

Update travis.yml

• Add PyPy3
• Update PyPy2 to float

Now that both are current. The old PyPy-5.4.1 would no longer install the needed SSL extensions, I believe.

UnboundLocalError Exception ; Variant class _validate field

Jason,

I am getting a UnboundLocalError: local variable 'e' referenced before assignment

line 287, in _validate (field.py)
self._reraise_validation_error(e, value)
UnboundLocalError: local variable 'e' referenced before assignment

It seems a 'tab' issue?

Thanks

Carlos

Use IField subclasses to expose more/better schema information

As mentioned in #44 (comment), there is quite a bit of schema information we're not exposing (such as min/max for numbers and date ranges) and some information that we're exposing that make no sense (min_length/max_length for Object fields). I think we could use the implemented interfaces to better guide what data we expose, producing more accurate schemas.

The question is, would any of that be useful to schema consumers?

Extensible UI Type tags

It would be useful to be able to arbitrarily define new UI tags on fields and have them returned on the schema.

Drop explicit support for Python 3.4

• PyPy3 will cover Python 3.5

Return nested schemas in JsonSchemafier

Return nested item schemas in JsonSchemafier. Currently when building a schema, we'll return the item schema for a sequence type (e.g. ListOrTuple), but without the underlying schema information for the contained type.

assert in FieldValidationMixin._reraise_validation_error

Jason,

We are raising a ValidationError within a field constraint .. e.g.

description = ValidText(title=u'Biography',
                            description=u"A short overview of who you are and what you "
                            u"do. Will be displayed on your author page, linked "
                                        u"from the items you create.",
                            max_length=140,  # twitter
                                        required=False,
                            constraint=checkCannotBeBlank)

since we are reusing the constraint function for other interfaces we don't specified the 'field' in the actual validation error exception.

line 173 in field.py

assert e.field is not None, "A _validate method failed to use ex.with_field_and_value()"

does not allow to reraise the error properly.

Could we relax this assertion?

Carlos

Better docs

Need a least a decent README.