Comments (11)
Then we have to carry that metaclass around until V3. And in V3 we'll need a compatibility metaclass for the compatibility metaclass, right?
from bump-pydantic.
Can you show me some expected input / expected transformations?
from bump-pydantic.
@Kludex well hard to tell... I'm just comparing docs for v1 and v2 - that does not look like a trivial case:
class PostCodeAnnotation:
@classmethod
def __get_pydantic_core_schema__(
cls, _source_type: Any, _handler: GetCoreSchemaHandler
) -> core_schema.CoreSchema:
return core_schema.no_info_after_validator_function(
cls.validate,
core_schema.str_schema(),
)
@classmethod
def __get_pydantic_json_schema__(
cls, schema: core_schema.CoreSchema, handler: GetJsonSchemaHandler
) -> JsonSchemaValue:
json_schema = handler(schema)
json_schema.update(
# simplified regex here for brevity, see the wikipedia link above
pattern='^[A-Z]{1,2}[0-9][A-Z0-9]? ?[0-9][A-Z]{2}$',
# some example postcodes
examples=['SP11 9DG', 'W1J 7BU'],
)
return json_schema
@classmethod
def validate(cls, v: str):
m = post_code_regex.fullmatch(v.upper())
if m:
return f'{m.group(1)} {m.group(2)}'
else:
raise PydanticCustomError('postcode', 'invalid postcode format')
PostCode = Annotated[str, PostCodeAnnotation]
from bump-pydantic.
What would be the input for this expected code?
from bump-pydantic.
it's from official docs:
from bump-pydantic.
For the __modify_schema__
, we could:
- Find the
FunctionDef
named__modify_schema__
, and save the second argument name (e.g.field_schema
). - If
field_schema.update
is found, then addfield_schema = handler(schema)
in the first line after theFunctionDef
, andreturn field_schema
in the last line. - If
field_schema.update
is not found, add a TODO note telling to update manually.
For the __get_validator__
, I think is a bit more complicated... Would it be safe to assume core_schema.no_info_after_validator_function
on every code source? π€
from bump-pydantic.
I think it is not auto solvable problem as in __get_validators__
you can return all sort of things and have multiple validators which does not look like a case for __modify_schema__
...
@samuelcolvin @dmontagu @hramezani maybe pydantic should have some backwards(deprecated) compatible approach to custom fields ?
overall to me custom fields feels very low-level implementation - I am not able to remember it without looking at examples :)
Maybe there should exist some simpler approach for 80+% of use cases, like:
class PostCode(str):
@classmethod
def __pydantic_validate__(cls, v):
if v not in UK_DATABASE_CALL:
raise ValidationError
return v
which should atomatically add needed __modify_schema__
guts
from bump-pydantic.
There is a simpler approach for 80% of the use cases:
from annotated_types import Predicate # or `pydantic.PlainValidator`, etc.
PostalCode = Annotated[str, Predicate(lambda v: True if v in UK_DATABASE_CALL else False)]
Hence why this section comes before the __get_pydantic_core_schema__
section.
For what it's worth to get multiple validators in __get_pydantic_core_schema__
you can either use a chain
validator:
from typing import Any, Callable, Dict, Iterable, List
from pydantic_core import CoreSchema, core_schema
from pydantic import (
TypeAdapter,
GetCoreSchemaHandler,
GetJsonSchemaHandler,
)
from pydantic.json_schema import JsonSchemaValue
class PostalCode(str):
@classmethod
def __get_pydantic_core_schema__(
cls, _source_type: Any, _handler: GetCoreSchemaHandler
) -> core_schema.CoreSchema:
val_func_schemas: List[CoreSchema] = []
for validation_function in getattr(cls, '__get_validators__', lambda: ())():
val_func_schemas.append(core_schema.no_info_plain_validator_function(validation_function))
return core_schema.chain_schema(val_func_schemas)
@classmethod
def __get_pydantic_json_schema__(
cls, schema: core_schema.CoreSchema, handler: GetJsonSchemaHandler
) -> JsonSchemaValue:
json_schema = {'type': 'string'}
modify_schema = getattr(cls, '__modify_schema__', None)
if modify_schema is not None:
json_schema = modify_schema(json_schema) or json_schema
return json_schema
@classmethod
def __get_validators__(cls) -> Iterable[Callable[..., Any]]:
yield cls.validate1
yield cls.validate2
@classmethod
def validate1(cls, v: str) -> str:
return v * 2
@classmethod
def validate2(cls, v: str) -> str:
return v[:5]
@classmethod
def __modify_schema__(cls, field_schema: Dict[str, Any]) -> Dict[str, Any]:
# __modify_schema__ should mutate the dict it receives in place,
# the returned value will be ignored
field_schema.update(
# simplified regex here for brevity, see the wikipedia link above
pattern='^[A-Z]{1,2}[0-9][A-Z0-9]? ?[0-9][A-Z]{2}$',
# some example postcodes
examples=['SP11 9DG', 'w1j7bu'],
)
return field_schema
ta = TypeAdapter(PostalCode)
assert ta.validate_python('abc') == 'abcab'
assert ta.json_schema() == {'examples': ['SP11 9DG', 'w1j7bu'], 'pattern': '^[A-Z]{1,2}[0-9][A-Z0-9]? ?[0-9][A-Z]{2}$', 'type': 'string'}
I guess bump-pydantic
could insert some version of this automatically. The one thing I don't think is trivial to figure out automatically is the json_schema = {'type': 'string'}
. I think Pydantic V1 just brute forced this by iterating through the bases and trying to generate a schema for each base and just returning the first one, which is obviously supper buggy but I guess works in very simple cases. The thing is those very simple cases are now better served by the Annotated[str, ...]
pattern so we don't want to encourage it.
from bump-pydantic.
There is a simpler approach for 80% of the use cases:
from annotated_types import Predicate # or `pydantic.PlainValidator`, etc. PostalCode = Annotated[str, Predicate(lambda v: True if v in UK_DATABASE_CALL else False)]
but this is not something you can automate with a bump-pydantic tool (too complex)
I think the best solution would be to introduce some meta class into pydantic codebase that will help transition from v1 to v2
class V1CustomField:
... magic with metaclasses ...
# then finally all you will have to do to migrate v1 to v2 is to add extra parent class:
class PostCode(str, V1CustomField): # <--- !!!
@classmethod
def __get_validators__(cls):
yield cls.validate
@classmethod
def validate(cls, v):
...
from bump-pydantic.
Then we have to carry that metaclass around until V3. And in V3 we'll need a compatibility metaclass for the compatibility metaclass, right?
I feel some sarcasm here :) but no - you make it with deprecation warning and remove in v3
there are already bunch of deprecated code - https://github.com/pydantic/pydantic/tree/main/pydantic/deprecated
from bump-pydantic.
Itβs definitely something to consider. We can always add it in the next minor release.
from bump-pydantic.
Related Issues (20)
- RecursionError when processing file with a large number of enums HOT 1
- Field(example=...) needs a codemod. HOT 1
- BP008: Not Handling conint(ge= , lt=) Correctly HOT 1
- Literal values "and" and "or" cause ParserSyntaxError HOT 10
- env -> validation_alias? HOT 1
- License info missing in classifiers and/or meta data HOT 3
- On Windows machines with >= 32 cores (more than 64 logical processors) execution fails HOT 1
- @root_validator will not updated to model_validator HOT 1
- [BUG]: bumb-pydantic doesnt change files HOT 2
- Set libcst dependency version HOT 1
- Rule BP002: Replace Config class by model_config attribute does not keep the arbitrary_types_allowed argument value
- uncovered case in validator->field_validator
- Issue with Literal["$"] being converted HOT 2
- Ignore .tox files HOT 2
- Add support for handling unannotated fields
- Examples in codemod files do not all work HOT 1
- Log.txt file automatic creation in PWD is kind of an anti-feature HOT 3
- Not Transformed automatically for other parent class except BaseModel HOT 2
- I have a big pile of improvements to upstream HOT 2
- BP008 may not be working as intended. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bump-pydantic.