p1c2u/jsonschema-path

Object-oriented JSONSchema

jsonschema-path's Issues

Allow to set verify to requests handler

Follow up python-openapi/openapi-spec-validator#148

Forward slash as default path parts separator

Follow the jsonschema standard.

Project rename

Because of possible confusion with other projects i.e. jsonschema-specifications and more straight forward naming the project should be renamed to jsonschema-path.

Object of type datetime is not JSON serializable

Original issue python-openapi/openapi-spec-validator#185

TypeError on Python 3.12b4

tests/integration/test_schema_path.py:7: in <module>
    import responses
        BytesIO    = <class '_io.BytesIO'>
        Path       = <class 'pathlib.Path'>
        __builtins__ = <builtins>
        __cached__ = '/home/runner/work/jsonschema-spec/jsonschema-spec/tests/integration/__pycache__/test_schema_path.cpython-312.pyc'
        __doc__    = None
        __file__   = '/home/runner/work/jsonschema-spec/jsonschema-spec/tests/integration/test_schema_path.py'
        __loader__ = <_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7fb1cfa9ae70>
        __name__   = 'test_schema_path'
        __package__ = ''
        __spec__   = ModuleSpec(name='test_schema_path', loader=<_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7fb1cfa9ae70>, origin='/home/runner/work/jsonschema-spec/jsonschema-spec/tests/integration/test_schema_path.py')
        dumps      = <function dumps at 0x7fb1cfb9a7a0>
        mock       = <module 'unittest.mock' from '/opt/hostedtoolcache/Python/3.12.0-beta.3/x64/lib/python3.12/unittest/mock.py'>
        pytest     = <module 'pytest' from '/home/runner/work/jsonschema-spec/jsonschema-spec/.venv/lib/python3.12/site-packages/pytest/__init__.py'>
.venv/lib/python3.12/site-packages/responses/__init__.py:40: in <module>
    from typing_extensions import Literal
        Any        = typing.Any
        Callable   = typing.Callable
        ConnectionError = <class 'requests.exceptions.ConnectionError'>
        Dict       = typing.Dict
        FirstMatchRegistry = <class 'responses.registries.FirstMatchRegistry'>
        HTTPAdapter = <class 'requests.adapters.HTTPAdapter'>
        Iterable   = typing.Iterable
        Iterator   = typing.Iterator
        List       = typing.List
        Mapping    = typing.Mapping
        MaxRetryError = <class 'urllib3.exceptions.MaxRetryError'>
        Optional   = typing.Optional
        Pattern    = <class 're.Pattern'>
        RetryError = <class 'requests.exceptions.RetryError'>
        Sequence   = typing.Sequence
        Sized      = typing.Sized
        TYPE_CHECKING = False
        Tuple      = typing.Tuple
        Type       = typing.Type
        Union      = typing.Union
        _ThreadingLock = <built-in function allocate_lock>
        __builtins__ = <builtins>
        __cached__ = '/home/runner/work/jsonschema-spec/jsonschema-spec/.venv/lib/python3.12/site-packages/responses/__pycache__/__init__.cpython-312.pyc'
        __doc__    = None
        __file__   = '/home/runner/work/jsonschema-spec/jsonschema-spec/.venv/lib/python3.12/site-packages/responses/__init__.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7fb1ce8da570>
        __name__   = 'responses'
        __package__ = 'responses'
        __path__   = ['/home/runner/work/jsonschema-spec/jsonschema-spec/.venv/lib/python3.12/site-packages/responses']
        __spec__   = ModuleSpec(name='responses', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7fb1ce8da570>, origin='/h...le_search_locations=['/home/runner/work/jsonschema-spec/jsonschema-spec/.venv/lib/python3.12/site-packages/responses'])
        _json_params_matcher = <function json_params_matcher at 0x7fb1ce76a980>
        _query_string_matcher = <function query_string_matcher at 0x7fb1ce76ab60>
        _urlencoded_params_matcher = <function urlencoded_params_matcher at 0x7fb1ce76a8e0>
        client     = <module 'http.client' from '/opt/hostedtoolcache/Python/3.12.0-beta.3/x64/lib/python3.12/http/client.py'>
        groupby    = <class 'itertools.groupby'>
        inspect    = <module 'inspect' from '/opt/hostedtoolcache/Python/3.12.0-beta.3/x64/lib/python3.12/inspect.py'>
        json_module = <module 'json' from '/opt/hostedtoolcache/Python/3.12.0-beta.3/x64/lib/python3.12/json/__init__.py'>
        logging    = <module 'logging' from '/opt/hostedtoolcache/Python/3.12.0-beta.3/x64/lib/python3.12/logging/__init__.py'>
        matchers   = <module 'responses.matchers' from '/home/runner/work/jsonschema-spec/jsonschema-spec/.venv/lib/python3.12/site-packages/responses/matchers.py'>
        namedtuple = <function namedtuple at 0x7fb1d09b4ea0>
        overload   = <function overload at 0x7fb1d084f240>
        partialmethod = <class 'functools.partialmethod'>
        registries = <module 'responses.registries' from '/home/runner/work/jsonschema-spec/jsonschema-spec/.venv/lib/python3.12/site-packages/responses/registries.py'>
        warn       = <built-in function warn>
        wraps      = <function wraps at 0x7fb1d09b6200>
        yaml       = <module 'yaml' from '/home/runner/work/jsonschema-spec/jsonschema-spec/.venv/lib/python3.12/site-packages/yaml/__init__.py'>
.venv/lib/python3.12/site-packages/typing_extensions.py:1174: in <module>
    class TypeVar(typing.TypeVar, _DefaultMixin, _root=True):
E   TypeError: type 'typing.TypeVar' is not an acceptable base type

Chain reaction with responses and typing-extensions.

Add support for pathlib in Spec.from_file_path

Original python-openapi/openapi-core#500

The jsonschema-path limit with referencing conflicts with latest jsonschema-specifications

Problem

The jsonschema-specifications 2023.11.1 has referencing>=0.31 dependency where jsonschema-path has referencing<0.31.0 which makes the two incompatible.

This causes weird issues, as currently when we try to get pip to find best combination of dependencies for Apache Airflow it installs the latest jsonschema-specification and downgrades versions of openapi-schema-validator and opanapi-spec-validator to some old versions that cause import errors - because those old versions do not require "jsonschema-path".

Both referencing 0.31.0 and jsonschema-specification 2023.11.1 have been released 3 hours ago and the problem was detected by Airflow's canary build - for example here:

https://github.com/apache/airflow/actions/runs/6869294174/job/18682462347#step:6:13103

Here is the result of it:

And it causes this error:

During handling of the above exception, another exception occurred:
/usr/local/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/providers/amazon/aws/sensors/test_cloud_formation.py:38: in <module>
    class TestCloudFormationCreateStackSensor:
tests/providers/amazon/aws/sensors/test_cloud_formation.py:40: in TestCloudFormationCreateStackSensor
    def setup_method(self, method):
/usr/local/lib/python3.8/site-packages/moto/__init__.py:18: in f
    module = importlib.import_module(module_name, "moto")
/usr/local/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
/usr/local/lib/python3.8/site-packages/moto/cloudformation/__init__.py:1: in <module>
    from .models import cloudformation_backends
/usr/local/lib/python3.8/site-packages/moto/cloudformation/models.py:21: in <module>
    from .parsing import ResourceMap, Output, OutputMap, Export
/usr/local/lib/python3.8/site-packages/moto/cloudformation/parsing.py:31: in <module>
    from moto.apigateway import models as apigw_models  # noqa  # pylint: disable=all
/usr/local/lib/python3.8/site-packages/moto/apigateway/__init__.py:1: in <module>
    from .models import apigateway_backends
/usr/local/lib/python3.8/site-packages/moto/apigateway/models.py:18: in <module>
    from openapi_spec_validator import validate_spec as validate  # type: ignore
/usr/local/lib/python3.8/site-packages/openapi_spec_validator/__init__.py:2: in <module>
    from openapi_spec_validator.shortcuts import validate_spec
/usr/local/lib/python3.8/site-packages/openapi_spec_validator/shortcuts.py:8: in <module>
    from openapi_spec_validator.validation import openapi_spec_validator_proxy
/usr/local/lib/python3.8/site-packages/openapi_spec_validator/validation/__init__.py:7: in <module>
    from openapi_schema_validator import oas30_format_checker
/usr/local/lib/python3.8/site-packages/openapi_schema_validator/__init__.py:3: in <module>
    from openapi_schema_validator.shortcuts import validate
/usr/local/lib/python3.8/site-packages/openapi_schema_validator/shortcuts.py:9: in <module>
    from openapi_schema_validator.validators import OAS31Validator
/usr/local/lib/python3.8/site-packages/openapi_schema_validator/validators.py:5: in <module>
    from jsonschema import _legacy_validators
E   ImportError: cannot import name '_legacy_validators' from 'jsonschema' (/usr/local/lib/python3.8/site-packages/jsonschema/__init__.py)
----- generated xml file: /files/test_result-providers_amazon-postgres.xml -----

Expected behaviour

It would be great if you relase jsonschama-path with bumped limit for referencing

We will workaround it temporarily but relaxing the limit for referencing on jsonschema-path should fix the problem.

Side coment: Also I think it would make sense to remove the limit altogether - usually libraries do not have strong "upper-binding" limits on other dependencies, unless there are some known breaking changes coming (which I think is unlikely in this case).

Test failures (regressions) with Python 3.12

Not the “Poetry way,” but a clean way to reproduce:

$ python3.11 -VV
Python 3.11.4 (main, Jun  7 2023, 00:00:00) [GCC 13.1.1 20230511 (Red Hat 13.1.1-2)]

gh repo clone p1c2u/jsonschema-spec
cd jsonschema-spec
python3.11 -m venv _e
. _e/bin/activate
pip install -e .
pip install pytest pytest-cov responses
python -m pytest

This confirms everything is working as expected with Python 3.11:

================================================= 27 passed in 0.29s =================================================

Now, repeat with Python 3.12:

$ python3.12 -VV
Python 3.12.0b3 (main, Jun 21 2023, 00:00:00) [GCC 13.1.1 20230614 (Red Hat 13.1.1-4)]

python3.12 -m venv _f
. _f/bin/activate
pip install -e .
pip install pytest pytest-cov responses
python -m pytest

============================================== short test summary info ===============================================
FAILED tests/integration/test_schema_path.py::TestSchemaPathFromFilePath::test_file_path - AttributeError: '_io.BufferedReader' object has no attribute 'decode'
FAILED tests/integration/test_schema_path.py::TestSchemaPathFromFilePath::test_file_path_relative - AttributeError: '_io.BufferedReader' object has no attribute 'decode'
FAILED tests/integration/test_schema_path.py::TestSchemaPathOpen::test_file - referencing.exceptions.Unresolvable: /tmp/tmpvferprav#/Info
FAILED tests/integration/test_schema_path.py::TestSchemaPathOpen::test_file_ref - AttributeError: '_io.BufferedReader' object has no attribute 'decode'
FAILED tests/integration/test_schema_path.py::TestSchemaPathOpen::test_double_ref - AttributeError: '_io.BufferedReader' object has no attribute 'decode'
FAILED tests/integration/test_schema_path.py::TestSchemaPathOpen::test_ref_recursive - referencing.exceptions.Unresolvable: recursive.yaml#/parameters/RecursiveReference
FAILED tests/unit/test_paths.py::TestSchemaPathFromPath::test_no_kwargs - AttributeError: '_io.BufferedReader' object has no attribute 'decode'
FAILED tests/unit/test_paths.py::TestSchemaPathFromFilePath::test_no_kwargs - AttributeError: '_io.BufferedReader' object has no attribute 'decode'
============================================ 8 failed, 19 passed in 1.29s ============================================

Full output: python312.txt

Overly-tight pins on referencing

Congratulations on the 0.2.0 release! 🎉

One concern noted downstream, the ^ specifiers act unpredictably, generating the following metadata in the wheel:

Requires-Dist: referencing (>=0.24.4,<0.25.0)

This may not have been the intended tightness of the spec, as the upstream package is already on 0.29.0, and likely includes bug fixes, features, compatibility with its rust-based upstream, and performance benefits... and it seems unlikely that features would have been taken away already for such a relatively new package.

This is not such a big issue with e.g. jsonschema-specifications, as the ^ works differently for non-0.* versions, and is probably closer to what is intended.

Requires-Dist: jsonschema-specifications (>=2023.3.4,<2024.0.0)

Store resolved references in accessor

Resolved references are not updated in accessor. This cause resolving process to triggered many times.

For more information see python-openapi/openapi-spec-validator#260

Support referencing 0.32.0

By not allowing referencing v0.32.0, it's causing a dependency resolution failure with pip-tools when also using moto.

BaseFilePathHandler has different working logic in Python 3.7 and Python 3.8, causing error reading files.

Seems like the difference between typing and typing_extensions is causing a problem later on in Python 3.7 vs. Python 3.8. https://github.com/p1c2u/jsonschema-spec/blob/da5e04c7b3481e62555d9c3820496030b188d1de/jsonschema_spec/handlers/protocols.py#L5-L10

Here's a quick code snippet to test, for some reason the fh object in Python 3.8 is an instance of SupportsRead, and Python 3.7 it is not.

from urllib.request import urlopen
from jsonschema_spec.handlers.protocols import SupportsRead

with urlopen('file:///testing.yml') as fh:         # ref#1
	if isinstance(fh, SupportsRead):               # ref#2
		print("supports read!")
		
# simplifies to the following for testing, above is just more readable
with urlopen('file:///testing.yml') as fh: isinstance(fh, SupportsRead)

This causes code in file.py to continue to parsed_url = urlparse(uri). Which will give the following error: AttributeError: '_io.BufferedReader' object has no attribute 'decode'. This in turn causes openapi-spec to break when running Spec.from_file_path in Python 3.7 when it works in Python 3.8.

I need to go eat, but you might understand what's happening more. And I imagine you'll want it fixed as it breaks < Python 3.7 versions. I'll figure out why the diff occurs later.

references

ref#1: https://github.com/p1c2u/jsonschema-spec/blob/da5e04c7b3481e62555d9c3820496030b188d1de/jsonschema_spec/handlers/urllib.py#L17-L18
ref#2: https://github.com/p1c2u/jsonschema-spec/blob/da5e04c7b3481e62555d9c3820496030b188d1de/jsonschema_spec/handlers/file.py#L38-L39