Giter Site home page Giter Site logo

bump-pydantic's Introduction

Bump Pydantic ♻️

PyPI - Version PyPI - Python Version

Bump Pydantic is a tool to help you migrate your code from Pydantic V1 to V2.

Note

If you find bugs, please report them on the issue tracker.

Table of contents


Installation

The installation is as simple as:

pip install bump-pydantic

Usage

bump-pydantic is a CLI tool, hence you can use it from your terminal.

It's easy to use. If your project structure is:

repository/
└── my_package/
    └── <python source files>

Then you'll want to do:

cd /path/to/repository
bump-pydantic my_package

Check diff before applying changes

To check the diff before applying the changes, you can run:

bump-pydantic --diff <path>

Apply changes

To apply the changes, you can run:

bump-pydantic <path>

Rules

You can find below the list of rules that are applied by bump-pydantic.

It's also possible to disable rules by using the --disable option.

BP001: Add default None to Optional[T], Union[T, None] and Any fields

  • ✅ Add default None to Optional[T] fields.

The following code will be transformed:

class User(BaseModel):
    name: Optional[str]

Into:

class User(BaseModel):
    name: Optional[str] = None

BP002: Replace Config class by model_config attribute

  • ✅ Replace Config class by model_config = ConfigDict().
  • ✅ Rename old Config attributes to new model_config attributes.
  • ✅ Add a TODO comment in case the transformation can't be done automatically.
  • ✅ Replace Extra enum by string values.

The following code will be transformed:

from pydantic import BaseModel, Extra


class User(BaseModel):
    name: str

    class Config:
        extra = Extra.forbid

Into:

from pydantic import ConfigDict, BaseModel


class User(BaseModel):
    name: str

    model_config = ConfigDict(extra="forbid")

BP003: Replace Field old parameters to new ones

  • ✅ Replace Field old parameters to new ones.
  • ✅ Replace field: Enum = Field(Enum.VALUE, const=True) by field: Literal[Enum.VALUE] = Enum.VALUE.

The following code will be transformed:

from typing import List

from pydantic import BaseModel, Field


class User(BaseModel):
    name: List[str] = Field(..., min_items=1)

Into:

from typing import List

from pydantic import BaseModel, Field


class User(BaseModel):
    name: List[str] = Field(..., min_length=1)

BP004: Replace imports

  • ✅ Replace BaseSettings from pydantic to pydantic_settings.
  • ✅ Replace Color and PaymentCardNumber from pydantic to pydantic_extra_types.

BP005: Replace GenericModel by BaseModel

  • ✅ Replace GenericModel by BaseModel.

The following code will be transformed:

from typing import Generic, TypeVar
from pydantic.generics import GenericModel

T = TypeVar('T')

class User(GenericModel, Generic[T]):
    name: str

Into:

from typing import Generic, TypeVar
from pydantic import BaseModel

T = TypeVar('T')

class User(BaseModel, Generic[T]):
    name: str

BP006: Replace __root__ by RootModel

  • ✅ Replace __root__ by RootModel.

The following code will be transformed:

from typing import List

from pydantic import BaseModel

class User(BaseModel):
    age: int
    name: str

class Users(BaseModel):
    __root__ = List[User]

Into:

from typing import List

from pydantic import RootModel, BaseModel

class User(BaseModel):
    age: int
    name: str

class Users(RootModel[List[User]]):
    pass

BP007: Replace decorators

  • ✅ Replace @validator by @field_validator.
  • ✅ Replace @root_validator by @model_validator.

The following code will be transformed:

from pydantic import BaseModel, validator, root_validator


class User(BaseModel):
    name: str

    @validator('name', pre=True)
    def validate_name(cls, v):
        return v

    @root_validator(pre=True)
    def validate_root(cls, values):
        return values

Into:

from pydantic import BaseModel, field_validator, model_validator


class User(BaseModel):
    name: str

    @field_validator('name', mode='before')
    def validate_name(cls, v):
        return v

    @model_validator(mode='before')
    def validate_root(cls, values):
        return values

BP008: Replace con* functions by Annotated versions

  • ✅ Replace constr(*args) by Annotated[str, StringConstraints(*args)].
  • ✅ Replace conint(*args) by Annotated[int, Field(*args)].
  • ✅ Replace confloat(*args) by Annotated[float, Field(*args)].
  • ✅ Replace conbytes(*args) by Annotated[bytes, Field(*args)].
  • ✅ Replace condecimal(*args) by Annotated[Decimal, Field(*args)].
  • ✅ Replace conset(T, *args) by Annotated[Set[T], Field(*args)].
  • ✅ Replace confrozenset(T, *args) by Annotated[Set[T], Field(*args)].
  • ✅ Replace conlist(T, *args) by Annotated[List[T], Field(*args)].

The following code will be transformed:

from pydantic import BaseModel, constr


class User(BaseModel):
    name: constr(min_length=1)

Into:

from pydantic import BaseModel, StringConstraints
from typing_extensions import Annotated


class User(BaseModel):
    name: Annotated[str, StringConstraints(min_length=1)]

BP009: Mark Pydantic "protocol" functions in custom types with proper TODOs

  • ✅ Mark __get_validators__ as to be replaced by __get_pydantic_core_schema__.
  • ✅ Mark __modify_schema__ as to be replaced by __get_pydantic_json_schema__.

The following code will be transformed:

class SomeThing:
    @classmethod
    def __get_validators__(cls):
        yield from []

    @classmethod
    def __modify_schema__(cls, field_schema, field):
        if field:
            field_schema['example'] = "Weird example"

Into:

class SomeThing:
    @classmethod
    # TODO[pydantic]: We couldn't refactor `__get_validators__`, please create the `__get_pydantic_core_schema__` manually.
    # Check https://docs.pydantic.dev/latest/migration/#defining-custom-types for more information.
    def __get_validators__(cls):
        yield from []

    @classmethod
    # TODO[pydantic]: We couldn't refactor `__modify_schema__`, please create the `__get_pydantic_json_schema__` manually.
    # Check https://docs.pydantic.dev/latest/migration/#defining-custom-types for more information.
    def __modify_schema__(cls, field_schema, field):
        if field:
            field_schema['example'] = "Weird example"

BP010: Add type annotations or TODO comments to fields without them

  • ✅ Add type annotations based on the default value for a few types that can be inferred, like bool, str, int, float.
  • ✅ Add # TODO[pydantic]: add type annotation comments to fields that can't be inferred.

The following code will be transformed:

from pydantic import BaseModel, Field

class Potato(BaseModel):
    name: str
    is_sale = True
    tags = ["tag1", "tag2"]
    price = 10.5
    description = "Some item"
    active = Field(default=True)
    ready = Field(True)
    age = Field(10, title="Age")

Into:

from pydantic import BaseModel, Field

class Potato(BaseModel):
    name: str
    is_sale: bool = True
    # TODO[pydantic]: add type annotation
    tags = ["tag1", "tag2"]
    price: float = 10.5
    description: str = "Some item"
    active: bool = Field(default=True)
    ready: bool = Field(True)
    age: int = Field(10, title="Age")

License

This project is licensed under the terms of the MIT license.

bump-pydantic's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

bump-pydantic's Issues

Literal values "and" and "or" cause ParserSyntaxError

Using the string value "and" or "or" for Literal causes a ParserSyntaxError

from typing import Literal
from pydantic import BaseModel

class Foo(BaseModel):
    bar: Literal["and"]
An error happened on test.py.
Traceback (most recent call last):
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/bump_pydantic/main.py", line 185, in run_codemods
    output_tree = transformer.transform_module(input_tree)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/codemod/_codemod.py", line 108, in transform_module
    return self.transform_module_impl(tree_with_metadata)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/codemod/_visitor.py", line 32, in transform_module_impl
    return tree.visit(self)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/module.py", line 90, in visit
    result = super(Module, self).visit(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/base.py", line 218, in visit
    should_visit_children = visitor.on_visit(self)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/matchers/_visitors.py", line 511, in on_visit
    return CSTTransformer.on_visit(self, node)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_visitors.py", line 44, in on_visit
    retval = visit_func(node)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/codemod/visitors/_remove_imports.py", line 300, in visit_Module
    node.visit(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/module.py", line 90, in visit
    result = super(Module, self).visit(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/base.py", line 218, in visit
    should_visit_children = visitor.on_visit(self)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/matchers/_visitors.py", line 718, in on_visit
    return CSTVisitor.on_visit(self, node)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_visitors.py", line 123, in on_visit
    retval = visit_func(node)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/codemod/visitors/_gather_unused_imports.py", line 71, in visit_Module
    node.visit(annotation_visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/module.py", line 90, in visit
    result = super(Module, self).visit(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/module.py", line 74, in _visit_and_replace_children
    body=visit_body_sequence(self, "body", self.body, visitor),
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 227, in visit_body_sequence
    return tuple(visit_body_iterable(parent, fieldname, children, visitor))
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 193, in visit_body_iterable
    new_child = child.visit(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/statement.py", line 1931, in _visit_and_replace_children
    body=visit_required(self, "body", self.body, visitor),
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 81, in visit_required
    result = node.visit(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/statement.py", line 697, in _visit_and_replace_children
    body=visit_body_sequence(self, "body", self.body, visitor),
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 227, in visit_body_sequence
    return tuple(visit_body_iterable(parent, fieldname, children, visitor))
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 193, in visit_body_iterable
    new_child = child.visit(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/statement.py", line 442, in _visit_and_replace_children
    body=visit_sequence(self, "body", self.body, visitor),
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 177, in visit_sequence
    return tuple(visit_iterable(parent, fieldname, children, visitor))
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 159, in visit_iterable
    new_child = child.visit(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/statement.py", line 1542, in _visit_and_replace_children
    annotation=visit_required(self, "annotation", self.annotation, visitor),
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 81, in visit_required
    result = node.visit(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/expression.py", line 1673, in _visit_and_replace_children
    annotation=visit_required(self, "annotation", self.annotation, visitor),
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 81, in visit_required
    result = node.visit(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/expression.py", line 1604, in _visit_and_replace_children
    slice=visit_sequence(self, "slice", self.slice, visitor),
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 177, in visit_sequence
    return tuple(visit_iterable(parent, fieldname, children, visitor))
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 159, in visit_iterable
    new_child = child.visit(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/expression.py", line 1549, in _visit_and_replace_children
    slice=visit_required(self, "slice", self.slice, visitor),
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 81, in visit_required
    result = node.visit(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/expression.py", line 1463, in _visit_and_replace_children
    value=visit_required(self, "value", self.value, visitor),
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 81, in visit_required
    result = node.visit(visitor)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_nodes/base.py", line 218, in visit
    should_visit_children = visitor.on_visit(self)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/matchers/_visitors.py", line 718, in on_visit
    return CSTVisitor.on_visit(self, node)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_visitors.py", line 123, in on_visit
    retval = visit_func(node)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/codemod/visitors/_gather_string_annotation_names.py", line 65, in visit_SimpleString
    self.handle_any_string(node)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/codemod/visitors/_gather_string_annotation_names.py", line 74, in handle_any_string
    mod = cst.parse_module(value)
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_parser/entrypoints.py", line 109, in parse_module
    result = _parse(
  File "/Users/sebastian/.local/pipx/venvs/bump-pydantic/lib/python3.10/site-packages/libcst/_parser/entrypoints.py", line 55, in _parse
    return parse(source_str)
libcst._exceptions.ParserSyntaxError: Syntax Error @ 1:1.
parser error: error at 1:3: expected one of (, *, +, -, ..., AWAIT, EOF, False, NUMBER, None, True, [, break, continue, lambda, match, not, pass, ~

and
^

PydanticSchemaGenerationError using constr

I'm working to update my code base to work on v2, and I'm having trouble because constr doesn't seem to be working properly.

Example:

from pydantic.v1 import constr
from pydantic import TypeAdapter

TypeAdapter(constr(min_length=1)).validate_python("x")

That last line is throwing an exception:

Traceback (most recent call last):
  File "[...]_venv/lib/python3.10/site-packages/pydantic/type_adapter.py", line 163, in __init__
    core_schema = _getattr_no_parents(type, '__pydantic_core_schema__')
  File "[...]_venv/lib/python3.10/site-packages/pydantic/type_adapter.py", line 97, in _getattr_no_parents
    raise AttributeError(attribute)
AttributeError: __pydantic_core_schema__

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "[...]_venv/lib/python3.10/site-packages/pydantic/type_adapter.py", line 165, in __init__
    core_schema = _get_schema(type, config_wrapper, parent_depth=_parent_depth + 1)
  File "[...]_venv/lib/python3.10/site-packages/pydantic/type_adapter.py", line 80, in _get_schema
    schema = gen.generate_schema(type_)
  File "[...]_venv/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 280, in generate_schema
    return self._generate_schema_for_type(
  File "[...]_venv/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 301, in _generate_schema_for_type
    schema = self._generate_schema(obj)
  File "[...]_venv/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 586, in _generate_schema
    return self._arbitrary_type_schema(obj, obj)
  File "[...]_venv/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 638, in _arbitrary_type_schema
    raise PydanticSchemaGenerationError(
pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for <class 'pydantic.v1.types.ConstrainedStrValue'>. Set `arbitrary_types_allowed=True` in the model_config to ignore this error or implement `__get_pydantic_core_schema__` on your type to fully support it.

If you got this error by calling handler(<some type>) within `__get_pydantic_core_schema__` then you likely need to call `handler.generate_schema(<some type>)` since we do not call `__get_pydantic_core_schema__` on `<some type>` otherwise to avoid infinite recursion.

For further information visit https://errors.pydantic.dev/2.0.3/u/schema-for-unknown-type

Using cpython 3.10.11, pydantic 2.0.3, and pydantic_core 2.3.0

Migration of methods

Hi all,

when is method-migration planned? I saw it on the roadmap. Has it been put down?

M

env -> validation_alias?

Hi. This is potentially just a question:

I've noticed bump-pydantic converted my instances of Field(..., env="SOME_ENV") to Field(validation_alias="SOME_ENV").

The documentation for the parameter isn't really illuminating:

'Whitelist' validation step. The field will be the single one allowed by the alias or set of aliases defined.

From its name I can infer that it just defines an alternative name for the field. However, what env used to do was take the value for the field from the environment variable SOME_ENV. Is this still the case with validation_alias? If not, I would frankly avoid "autofixing" this in bump-pydantic, since it's basically guaranteed to generate hard to debug errors. At most what I would do would be to yield an error the user can see in the logs, then they'd be aware of the problem, but if the previous functionality has been removed and must be implemented manually, then changing things (semi-)silently does more harm than good.
If the functionality has stayed in place then I have qualms with Pydantic's own documentation and name for this parameter, but that's a story for another repo.

Missing support for utf-8

It appears that this project does not support utf-8

create file in empty folder with contents

from pydantic import BaseModel

#just some boilerplateClass
class Myclass:
    test:int # this will break parsing >šěěčřžýáíéů

causes folowing traceback

RemoteTraceback: 
"""
Traceback (most recent call last):
  File "C:\Program Files\Python311\Lib\multiprocessing\pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\bump_pydantic\main.py", line 106, in visit_class_def
    code = Path(filename).read_text()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\pathlib.py", line 1059, in read_text
    return f.read()
           ^^^^^^^^
  File "C:\Program Files\Python311\Lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 126: character maps to <undefined>
"""

The above exception was the direct cause of the following exception:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ C:\Program Files\Python311\Lib\site-packages\bump_pydantic\main.py:70 in main                    │
│                                                                                                  │
│    67 │   │   task = progress.add_task(description="Looking for Pydantic Models...", total=len   │
│    68 │   │   with multiprocessing.Pool() as pool:                                               │
│    69 │   │   │   partial_visit_class_def = functools.partial(visit_class_def, metadata_manage   │
│ ❱  70 │   │   │   for local_scratch in pool.imap_unordered(partial_visit_class_def, files):      │
│    71 │   │   │   │   progress.advance(task)                                                     │
│    72 │   │   │   │   for key, value in local_scratch.items():                                   │
│    73 │   │   │   │   │   scratch.setdefault(key, value).update(value)                           │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │                 console = <console width=154 ColorSystem.TRUECOLOR>                          │ │
│ │                    diff = False                                                              │ │
│ │                 disable = []                                                                 │ │
│ │                   files = ['test.py']                                                        │ │
│ │               files_str = [WindowsPath('test.py')]                                           │ │
│ │                log_file = None                                                               │ │
│ │        metadata_manager = <libcst.metadata.full_repo_manager.FullRepoManager object at       │ │
│ │                           0x000001E4E7AC4490>                                                │ │
│ │                 package = WindowsPath('.')                                                   │ │
│ │ partial_visit_class_def = functools.partial(<function visit_class_def at                     │ │
│ │                           0x000001E4E5FF2C00>,                                               │ │
│ │                           <libcst.metadata.full_repo_manager.FullRepoManager object at       │ │
│ │                           0x000001E4E7AC4490>, WindowsPath('.'))                             │ │
│ │                    pool = <multiprocessing.pool.Pool state=TERMINATE pool_size=16>           │ │
│ │                progress = <rich.progress.Progress object at 0x000001E4E7AC5210>              │ │
│ │               providers = {                                                                  │ │
│ │                           │   <class                                                         │ │
│ │                           'libcst.metadata.name_provider.FullyQualifiedNameProvider'>,       │ │
│ │                           │   <class 'libcst.metadata.scope_provider.ScopeProvider'>         │ │
│ │                           }                                                                  │ │
│ │                 scratch = {}                                                                 │ │
│ │                    task = 0                                                                  │ │
│ │                 version = None                                                               │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ C:\Program Files\Python311\Lib\multiprocessing\pool.py:873 in next                               │
│                                                                                                  │
│   870 │   │   success, value = item                                                              │
│   871 │   │   if success:                                                                        │
│   872 │   │   │   return value                                                                   │
│ ❱ 873 │   │   raise value                                                                        │
│   874 │                                                                                          │
│   875 │   __next__ = next                    # XXX                                               │
│   876                                                                                            │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │    item = (                                                                                  │ │
│ │           │   False,                                                                         │ │
│ │           │   UnicodeDecodeError('charmap', b'from pydantic import BaseModel\r\n\r\n#just    │ │
│ │           some boilerplateClass\r\nclass Myclass:\r\n    test:int # this will break parsing  │ │
│ │           >\xc5\xa1\xc4\x9b\xc4\x9b\xc4\x8d\xc5\x99\xc5\xbe\xc3\xbd\xc3\xa1\xc3\xad\xc3\xa9… │ │
│ │           126, 127, 'character maps to <undefined>')                                         │ │
│ │           )                                                                                  │ │
│ │    self = <multiprocessing.pool.IMapUnorderedIterator object at 0x000001E4E7AC6DD0>          │ │
│ │ success = False                                                                              │ │
│ │ timeout = None                                                                               │ │
│ │   value = UnicodeDecodeError('charmap', b'from pydantic import BaseModel\r\n\r\n#just some   │ │
│ │           boilerplateClass\r\nclass Myclass:\r\n    test:int # this will break parsing       │ │
│ │           >\xc5\xa1\xc4\x9b\xc4\x9b\xc4\x8d\xc5\x99\xc5\xbe\xc3\xbd\xc3\xa1\xc3\xad\xc3\xa9… │ │
│ │           126, 127, 'character maps to <undefined>')                                         │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 126: character maps to <undefined>

this is on windows 10 with python 3.11

ParserSyntaxError: Syntax Error "in" bug report

An error happened on myschema.py.
Traceback (most recent call last):
  File "env/lib/python3.10/site-packages/bump_pydantic/main.py", line 185, in run_codemods
    output_tree = transformer.transform_module(input_tree)
  File "env/lib/python3.10/site-packages/libcst/codemod/_command.py", line 87, in transform_module
    tree = self._instantiate_and_run(transform, tree)
  File "env/lib/python3.10/site-packages/libcst/codemod/_command.py", line 57, in _instantiate_and_run
    return inst.transform_module(tree)
  File "env/lib/python3.10/site-packages/libcst/codemod/_codemod.py", line 108, in transform_module
    return self.transform_module_impl(tree_with_metadata)
  File "env/lib/python3.10/site-packages/libcst/codemod/_visitor.py", line 32, in transform_module_impl
    return tree.visit(self)
  File "env/lib/python3.10/site-packages/libcst/_nodes/module.py", line 90, in visit
    result = super(Module, self).visit(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/base.py", line 218, in visit
    should_visit_children = visitor.on_visit(self)
  File "env/lib/python3.10/site-packages/libcst/matchers/_visitors.py", line 511, in on_visit
    return CSTTransformer.on_visit(self, node)
  File "env/lib/python3.10/site-packages/libcst/_visitors.py", line 44, in on_visit
    retval = visit_func(node)
  File "env/lib/python3.10/site-packages/libcst/codemod/visitors/_remove_imports.py", line 300, in visit_Module
    node.visit(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/module.py", line 90, in visit
    result = super(Module, self).visit(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/base.py", line 218, in visit
    should_visit_children = visitor.on_visit(self)
  File "env/lib/python3.10/site-packages/libcst/matchers/_visitors.py", line 718, in on_visit
    return CSTVisitor.on_visit(self, node)
  File "env/lib/python3.10/site-packages/libcst/_visitors.py", line 123, in on_visit
    retval = visit_func(node)
  File "env/lib/python3.10/site-packages/libcst/codemod/visitors/_gather_unused_imports.py", line 71, in visit_Module
    node.visit(annotation_visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/module.py", line 90, in visit
    result = super(Module, self).visit(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/module.py", line 74, in _visit_and_replace_children
    body=visit_body_sequence(self, "body", self.body, visitor),
  File "env/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 227, in visit_body_sequence
    return tuple(visit_body_iterable(parent, fieldname, children, visitor))
  File "env/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 193, in visit_body_iterable
    new_child = child.visit(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/statement.py", line 1931, in _visit_and_replace_children
    body=visit_required(self, "body", self.body, visitor),
  File "env/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 81, in visit_required
    result = node.visit(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/statement.py", line 697, in _visit_and_replace_children
    body=visit_body_sequence(self, "body", self.body, visitor),
  File "env/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 227, in visit_body_sequence
    return tuple(visit_body_iterable(parent, fieldname, children, visitor))
  File "env/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 193, in visit_body_iterable
    new_child = child.visit(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/statement.py", line 442, in _visit_and_replace_children
    body=visit_sequence(self, "body", self.body, visitor),
  File "env/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 177, in visit_sequence
    return tuple(visit_iterable(parent, fieldname, children, visitor))
  File "env/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 159, in visit_iterable
    new_child = child.visit(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/statement.py", line 1542, in _visit_and_replace_children
    annotation=visit_required(self, "annotation", self.annotation, visitor),
  File "env/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 81, in visit_required
    result = node.visit(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/expression.py", line 1673, in _visit_and_replace_children
    annotation=visit_required(self, "annotation", self.annotation, visitor),
  File "env/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 81, in visit_required
    result = node.visit(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/expression.py", line 1604, in _visit_and_replace_children
    slice=visit_sequence(self, "slice", self.slice, visitor),
  File "env/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 177, in visit_sequence
    return tuple(visit_iterable(parent, fieldname, children, visitor))
  File "env/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 159, in visit_iterable
    new_child = child.visit(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/expression.py", line 1549, in _visit_and_replace_children
    slice=visit_required(self, "slice", self.slice, visitor),
  File "env/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 81, in visit_required
    result = node.visit(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/expression.py", line 1463, in _visit_and_replace_children
    value=visit_required(self, "value", self.value, visitor),
  File "env/lib/python3.10/site-packages/libcst/_nodes/internal.py", line 81, in visit_required
    result = node.visit(visitor)
  File "env/lib/python3.10/site-packages/libcst/_nodes/base.py", line 218, in visit
    should_visit_children = visitor.on_visit(self)
  File "env/lib/python3.10/site-packages/libcst/matchers/_visitors.py", line 718, in on_visit
    return CSTVisitor.on_visit(self, node)
  File "env/lib/python3.10/site-packages/libcst/_visitors.py", line 123, in on_visit
    retval = visit_func(node)
  File "env/lib/python3.10/site-packages/libcst/codemod/visitors/_gather_string_annotation_names.py", line 65, in visit_SimpleString
    self.handle_any_string(node)
  File "env/lib/python3.10/site-packages/libcst/codemod/visitors/_gather_string_annotation_names.py", line 74, in handle_any_string
    mod = cst.parse_module(value)
  File "env/lib/python3.10/site-packages/libcst/_parser/entrypoints.py", line 109, in parse_module
    result = _parse(
  File "env/lib/python3.10/site-packages/libcst/_parser/entrypoints.py", line 55, in _parse
    return parse(source_str)
libcst._exceptions.ParserSyntaxError: Syntax Error @ 1:1.
parser error: error at 1:2: expected one of (, *, +, -, ..., AWAIT, EOF, False, NUMBER, None, True, [, break, continue, lambda, match, not, pass, ~

in
^

code

class Schema(BaseModel):
    option: Literal["in", "not_in", "exact"] = "in"

Unicode in code files causes text decoding to fail on Windows (cp1252 encoding)

Issue

Code files containing Unicode characters such as emojis cause bump-pydantic to exit with a UnicodeDecodeError due to the default text encoding used by open() on Windows being cp1252.

Steps to reproduce

  1. Simply run the tool against its own source code with bump-pydantic bump_pydantic. The emoji ♻️ will cause it to fail.

Details

Full traceback

RemoteTraceback: 
"""
Traceback (most recent call last):
  File "C:\Program Files\Python\3.10\lib\multiprocessing\pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "C:\Users\Lemonyte\Desktop\bump-pydantic\bump_pydantic\main.py", line 106, in visit_class_def
    code = Path(filename).read_text()
  File "C:\Program Files\Python\3.10\lib\pathlib.py", line 1135, in read_text
    return f.read()
  File "C:\Program Files\Python\3.10\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in position 923: character maps to <undefined>
"""

The above exception was the direct cause of the following exception:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ C:\Users\Lemonyte\Desktop\bump-pydantic\bump_pydantic\main.py:70 in main                         │
│                                                                                                  │
│    67 │   │   task = progress.add_task(description="Looking for Pydantic Models...", total=len   │
│    68 │   │   with multiprocessing.Pool() as pool:                                               │
│    69 │   │   │   partial_visit_class_def = functools.partial(visit_class_def, metadata_manage   │
│ ❱  70 │   │   │   for local_scratch in pool.imap_unordered(partial_visit_class_def, files):      │
│    71 │   │   │   │   progress.advance(task)                                                     │
│    72 │   │   │   │   for key, value in local_scratch.items():                                   │
│    73 │   │   │   │   │   scratch.setdefault(key, value).update(value)                           │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │                 console = <console width=155 ColorSystem.TRUECOLOR>                          │ │
│ │                    diff = False                                                              │ │
│ │                 disable = []                                                                 │ │
│ │                   files = [                                                                  │ │
│ │                           │   'bump_pydantic\\main.py',                                      │ │
│ │                           │   'bump_pydantic\\__init__.py',                                  │ │
│ │                           │   'bump_pydantic\\__main__.py',                                  │ │
│ │                           │   'bump_pydantic\\codemods\\add_default_none.py',                │ │
│ │                           │   'bump_pydantic\\codemods\\class_def_visitor.py',               │ │
│ │                           │   'bump_pydantic\\codemods\\field.py',                           │ │
│ │                           │   'bump_pydantic\\codemods\\replace_config.py',                  │ │
│ │                           │   'bump_pydantic\\codemods\\replace_generic_model.py',           │ │
│ │                           │   'bump_pydantic\\codemods\\replace_imports.py',                 │ │
│ │                           │   'bump_pydantic\\codemods\\root_model.py',                      │ │
│ │                           │   ... +4                                                         │ │
│ │                           ]                                                                  │ │
│ │               files_str = [                                                                  │ │
│ │                           │   WindowsPath('bump_pydantic/main.py'),                          │ │
│ │                           │   WindowsPath('bump_pydantic/__init__.py'),                      │ │
│ │                           │   WindowsPath('bump_pydantic/__main__.py'),                      │ │
│ │                           │   WindowsPath('bump_pydantic/codemods/add_default_none.py'),     │ │
│ │                           │   WindowsPath('bump_pydantic/codemods/class_def_visitor.py'),    │ │
│ │                           │   WindowsPath('bump_pydantic/codemods/field.py'),                │ │
│ │                           │   WindowsPath('bump_pydantic/codemods/replace_config.py'),       │ │
│ │                           │                                                                  │ │
│ │                           WindowsPath('bump_pydantic/codemods/replace_generic_model.py'),    │ │
│ │                           │   WindowsPath('bump_pydantic/codemods/replace_imports.py'),      │ │
│ │                           │   WindowsPath('bump_pydantic/codemods/root_model.py'),           │ │
│ │                           │   ... +4                                                         │ │
│ │                           ]                                                                  │ │
│ │                     key = 'class_def_visitor'                                                │ │
│ │           local_scratch = {                                                                  │ │
│ │                           │   'class_def_visitor': defaultdict(<class 'set'>, {              │ │
│ │                           │   │                                                              │ │
│ │                           'bump_pydantic.codemods.replace_generic_model.ReplaceGenericModel… │ │
│ │                           {                                                                  │ │
│ │                           │   │   │   'libcst.codemod.VisitorBasedCodemodCommand'            │ │
│ │                           │   │   }                                                          │ │
│ │                           │   })                                                             │ │
│ │                           }                                                                  │ │
│ │                log_file = None                                                               │ │
│ │        metadata_manager = <libcst.metadata.full_repo_manager.FullRepoManager object at       │ │
│ │                           0x000001D1F3F6FEE0>                                                │ │
│ │                 package = WindowsPath('bump_pydantic')                                       │ │
│ │ partial_visit_class_def = functools.partial(<function visit_class_def at                     │ │
│ │                           0x000001D1F3F5B1C0>,                                               │ │
│ │                           <libcst.metadata.full_repo_manager.FullRepoManager object at       │ │
│ │                           0x000001D1F3F6FEE0>, WindowsPath('bump_pydantic'))                 │ │
│ │                    pool = <multiprocessing.pool.Pool state=TERMINATE pool_size=12>           │ │
│ │                progress = <rich.progress.Progress object at 0x000001D1F3FB0850>              │ │
│ │               providers = {                                                                  │ │
│ │                           │   <class                                                         │ │
│ │                           'libcst.metadata.name_provider.FullyQualifiedNameProvider'>,       │ │
│ │                           │   <class 'libcst.metadata.scope_provider.ScopeProvider'>         │ │
│ │                           }                                                                  │ │
│ │                 scratch = {                                                                  │ │
│ │                           │   'class_def_visitor': defaultdict(<class 'set'>, {              │ │
│ │                           │   │                                                              │ │
│ │                           'bump_pydantic.codemods.replace_generic_model.ReplaceGenericModel… │ │
│ │                           {                                                                  │ │
│ │                           │   │   │   'libcst.codemod.VisitorBasedCodemodCommand'            │ │
│ │                           │   │   }                                                          │ │
│ │                           │   })                                                             │ │
│ │                           }                                                                  │ │
│ │                    task = 0                                                                  │ │
│ │                   value = defaultdict(<class 'set'>, {                                       │ │
│ │                           │                                                                  │ │
│ │                           'bump_pydantic.codemods.replace_generic_model.ReplaceGenericModel… │ │
│ │                           {                                                                  │ │
│ │                           │   │   'libcst.codemod.VisitorBasedCodemodCommand'                │ │
│ │                           │   }                                                              │ │
│ │                           })                                                                 │ │
│ │                 version = None                                                               │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ C:\Program Files\Python\3.10\lib\multiprocessing\pool.py:873 in next                             │
│                                                                                                  │
│   870 │   │   success, value = item                                                              │
│   871 │   │   if success:                                                                        │
│   872 │   │   │   return value                                                                   │
│ ❱ 873 │   │   raise value                                                                        │
│   874 │                                                                                          │
│   875__next__ = next                    # XXX                                               │876                                                                                            │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │    item = (                                                                                  │ │
│ │           │   False,                                                                         │ │
│ │           │   UnicodeDecodeError('charmap', b'import difflib\r\nimport functools\r\nimport   │ │
│ │           multiprocessing\r\nimport os\r\nimport time\r\nfrom contextlib import              │ │
│ │           nullcontext\r\nfrom pathlib import Path\r\nfrom typing import Any, Callable, Dict, │ │
│ │           Iterable, List, Type, TypeVar, Union\r\n\r\nimport libcst as cst\r\nfrom           │ │
│ │           libcst.codemod import CodemodContext, ContextAwareTransformer\r\nfrom              │ │
│ │           libcst.helpers import calculate_module_and_package\r\nfrom libcst.metadata import  │ │
│ │           FullRepoManager, FullyQualifiedNameProvider, ScopeProvider\r\nfrom rich.console    │ │
│ │           import Console\r\nfrom rich.progress import Progress\r\nfrom typer import          │ │
│ │           Argument, Exit, Option, Typer, echo\r\nfrom typing_extensions import               │ │
│ │           ParamSpec\r\n\r\nfrom bump_pydantic import __version__\r\nfrom                     │ │
│ │           bump_pydantic.codemods import Rule, gather_codemods\r\nfrom                        │ │
│ │           bump_pydantic.codemods.class_def_visitor import ClassDefVisitor\r\nfrom            │ │
│ │           bump_pydantic.markers.find_base_model import find_base_model\r\n\r\napp =          │ │
│ │           Typer(\r\n    help="Convert Pydantic from V1 to V2 \xe2\x99\xbb\xef\xb8\x8f",\r\n  │ │
│ │           invoke_without_command=True,\r\n    add_completion=False,\r\n)\r\n\r\nP =          │ │
│ │           ParamSpec("P")\r\nT = TypeVar("T")\r\n\r\n\r\ndef version_callback(value:          │ │
│ │           bool):\r\n    if value:\r\n        echo(f"bump-pydantic version:                   │ │
│ │           {__version__}")\r\n        raise Exit()\r\n\r\n\r\n@app.callback()\r\ndef          │ │
│ │           main(\r\n    package: Path = Argument(..., exists=True, dir_okay=True,             │ │
│ │           allow_dash=False),\r\n    diff: bool = Option(False, help="Show diff instead of    │ │
│ │           applying changes."),\r\n    disable: List[Rule] = Option(default=[], help="Disable │ │
│ │           a rule."),\r\n    log_file: Union[Path, None] = Option(None, help="Log file to     │ │
│ │           write to."),\r\n    version: bool = Option(\r\n        None,\r\n                   │ │
│ │           "--version",\r\n        callback=version_callback,\r\n        is_eager=True,\r\n   │ │
│ │           help="Show the version and exit.",\r\n    ),\r\n):\r\n    # NOTE:                  │ │
│ │           LIBCST_PARSER_TYPE=native is required according to                                 │ │
│ │           https://github.com/Instagram/LibCST/issues/487.\r\n                                │ │
│ │           os.environ["LIBCST_PARSER_TYPE"] = "native"\r\n\r\n    console = Console()\r\n     │ │
│ │           files_str = list(package.glob("**/*.py"))\r\n    files =                           │ │
│ │           [str(file.relative_to(".")) for file in files_str]\r\n\r\n    providers =          │ │
│ │           {FullyQualifiedNameProvider, ScopeProvider}\r\n    metadata_manager =              │ │
│ │           FullRepoManager(".", files, providers=providers)  # type: ignore[arg-type]\r\n     │ │
│ │           metadata_manager.resolve_cache()\r\n\r\n    scratch: dict[str, Any] = {}\r\n       │ │
│ │           with Progress(*Progress.get_default_columns(), transient=True) as progress:\r\n    │ │
│ │           task = progress.add_task(description="Looking for Pydantic Models...",             │ │
│ │           total=len(files))\r\n        with multiprocessing.Pool() as pool:\r\n              │ │
│ │           partial_visit_class_def = functools.partial(visit_class_def, metadata_manager,     │ │
│ │           package)\r\n            for local_scratch in                                       │ │
│ │           pool.imap_unordered(partial_visit_class_def, files):\r\n                           │ │
│ │           progress.advance(task)\r\n                for key, value in                        │ │
│ │           local_scratch.items():\r\n                    scratch.setdefault(key,              │ │
│ │           value).update(value)\r\n\r\n    find_base_model(scratch)\r\n\r\n    start_time =   │ │
│ │           time.time()\r\n\r\n    codemods = gather_codemods(disabled=disable)\r\n\r\n        │ │
│ │           log_ctx_mgr = log_file.open("a+") if log_file else nullcontext()\r\n               │ │
│ │           partial_run_codemods = functools.partial(run_codemods, codemods, metadata_manager, │ │
│ │           scratch, package, diff)\r\n\r\n    with Progress(*Progress.get_default_columns(),  │ │
│ │           transient=True) as progress:\r\n        task =                                     │ │
│ │           progress.add_task(description="Executing codemods...", total=len(files))\r\n       │ │
│ │           with multiprocessing.Pool() as pool, log_ctx_mgr as log_fp:  # type:               │ │
│ │           ignore[attr-defined]\r\n            for error_msg in                               │ │
│ │           pool.imap_unordered(partial_run_codemods, files):\r\n                              │ │
│ │           progress.advance(task)\r\n                if error_msg is None:\r\n                │ │
│ │           continue\r\n\r\n                if log_fp is None:\r\n                             │ │
│ │           color_diff(console, error_msg)\r\n                else:\r\n                        │ │
│ │           log_fp.writelines(error_msg)\r\n\r\n            if log_fp:\r\n                     │ │
│ │           log_fp.write("Run successfully!\\n")\r\n\r\n    modified = [Path(f) for f in files │ │
│ │           if os.stat(f).st_mtime > start_time]\r\n    if modified:\r\n                       │ │
│ │           print(f"Refactored {len(modified)} files.")\r\n\r\n\r\ndef                         │ │
│ │           visit_class_def(metadata_manager: FullRepoManager, package: Path, filename: str)   │ │
│ │           -> Dict[str, Any]:\r\n    code = Path(filename).read_text()\r\n    module =        │ │
│ │           cst.parse_module(code)\r\n    module_and_package =                                 │ │
│ │           calculate_module_and_package(str(package), filename)\r\n\r\n    context =          │ │
│ │           CodemodContext(\r\n        metadata_manager=metadata_manager,\r\n                  │ │
│ │           filename=filename,\r\n        full_module_name=module_and_package.name,\r\n        │ │
│ │           full_package_name=module_and_package.package,\r\n    )\r\n    visitor =            │ │
│ │           ClassDefVisitor(context=context)\r\n    visitor.transform_module(module)\r\n       │ │
│ │           return context.scratch\r\n\r\n\r\ndef capture_exception(func: Callable[P, T]) ->   │ │
│ │           Callable[P, Union[T, Iterable[str]]]:\r\n    @functools.wraps(func)\r\n    def     │ │
│ │           wrapper(*args: P.args, **kwargs: P.kwargs) -> Union[T, Iterable[str]]:\r\n         │ │
│ │           try:\r\n            return func(*args, **kwargs)\r\n        except Exception as    │ │
│ │           exc:\r\n            func_args = [repr(arg) for arg in args]\r\n                    │ │
│ │           func_kwargs = [f"{key}={repr(value)}" for key, value in kwargs.items()]\r\n        │ │
│ │           return [f"{func.__name__}({\', \'.join(func_args +                                 │ │
│ │           func_kwargs)})\\n{exc}"]\r\n\r\n    return                                         │ │
│ │           wrapper\r\n\r\n\r\n@capture_exception\r\ndef run_codemods(\r\n    codemods:        │ │
│ │           List[Type[ContextAwareTransformer]],\r\n    metadata_manager: FullRepoManager,\r\n │ │
│ │           scratch: Dict[str, Any],\r\n    package: Path,\r\n    diff: bool,\r\n    filename: │ │
│ │           str,\r\n) -> Union[List[str], None]:\r\n    module_and_package =                   │ │
│ │           calculate_module_and_package(str(package), filename)\r\n    context =              │ │
│ │           CodemodContext(\r\n        metadata_manager=metadata_manager,\r\n                  │ │
│ │           filename=filename,\r\n        full_module_name=module_and_package.name,\r\n        │ │
│ │           full_package_name=module_and_package.package,\r\n    )\r\n                         │ │
│ │           context.scratch.update(scratch)\r\n\r\n    file_path = Path(filename)\r\n    with  │ │
│ │           file_path.open("r+") as fp:\r\n        code = fp.read()\r\n                        │ │
│ │           fp.seek(0)\r\n\r\n        input_tree = cst.parse_module(code)\r\n\r\n        for   │ │
│ │           codemod in codemods:\r\n            transformer = codemod(context=context)\r\n\r\n │ │
│ │           output_tree = transformer.transform_module(input_tree)\r\n            input_tree = │ │
│ │           output_tree\r\n\r\n        output_code = input_tree.code\r\n        if code !=     │ │
│ │           output_code:\r\n            if diff:\r\n                lines =                    │ │
│ │           difflib.unified_diff(\r\n                    code.splitlines(keepends=True),\r\n   │ │
│ │           output_code.splitlines(keepends=True),\r\n                                         │ │
│ │           fromfile=filename,\r\n                    tofile=filename,\r\n                     │ │
│ │           )\r\n                return list(lines)\r\n            else:\r\n                   │ │
│ │           fp.write(output_code)\r\n                fp.truncate()\r\n\r\n    return           │ │
│ │           None\r\n\r\n\r\ndef color_diff(console: Console, lines: Iterable[str]) ->          │ │
│ │           None:\r\n    for line in lines:\r\n        line = line.rstrip("\\n")\r\n        if │ │
│ │           line.startswith("+"):\r\n            console.print(line, style="green")\r\n        │ │
│ │           elif line.startswith("-"):\r\n            console.print(line, style="red")\r\n     │ │
│ │           elif line.startswith("^"):\r\n            console.print(line, style="blue")\r\n    │ │
│ │           else:\r\n            console.print(line, style="white")\r\n', 923, 924, 'character │ │
│ │           maps to <undefined>')                                                              │ │
│ │           )                                                                                  │ │
│ │    self = <multiprocessing.pool.IMapUnorderedIterator object at 0x000001D1F4030F70>          │ │
│ │ success = False                                                                              │ │
│ │ timeout = None                                                                               │ │
│ │   value = UnicodeDecodeError('charmap', b'import difflib\r\nimport functools\r\nimport       │ │
│ │           multiprocessing\r\nimport os\r\nimport time\r\nfrom contextlib import              │ │
│ │           nullcontext\r\nfrom pathlib import Path\r\nfrom typing import Any, Callable, Dict, │ │
│ │           Iterable, List, Type, TypeVar, Union\r\n\r\nimport libcst as cst\r\nfrom           │ │
│ │           libcst.codemod import CodemodContext, ContextAwareTransformer\r\nfrom              │ │
│ │           libcst.helpers import calculate_module_and_package\r\nfrom libcst.metadata import  │ │
│ │           FullRepoManager, FullyQualifiedNameProvider, ScopeProvider\r\nfrom rich.console    │ │
│ │           import Console\r\nfrom rich.progress import Progress\r\nfrom typer import          │ │
│ │           Argument, Exit, Option, Typer, echo\r\nfrom typing_extensions import               │ │
│ │           ParamSpec\r\n\r\nfrom bump_pydantic import __version__\r\nfrom                     │ │
│ │           bump_pydantic.codemods import Rule, gather_codemods\r\nfrom                        │ │
│ │           bump_pydantic.codemods.class_def_visitor import ClassDefVisitor\r\nfrom            │ │
│ │           bump_pydantic.markers.find_base_model import find_base_model\r\n\r\napp =          │ │
│ │           Typer(\r\n    help="Convert Pydantic from V1 to V2 \xe2\x99\xbb\xef\xb8\x8f",\r\n  │ │
│ │           invoke_without_command=True,\r\n    add_completion=False,\r\n)\r\n\r\nP =          │ │
│ │           ParamSpec("P")\r\nT = TypeVar("T")\r\n\r\n\r\ndef version_callback(value:          │ │
│ │           bool):\r\n    if value:\r\n        echo(f"bump-pydantic version:                   │ │
│ │           {__version__}")\r\n        raise Exit()\r\n\r\n\r\n@app.callback()\r\ndef          │ │
│ │           main(\r\n    package: Path = Argument(..., exists=True, dir_okay=True,             │ │
│ │           allow_dash=False),\r\n    diff: bool = Option(False, help="Show diff instead of    │ │
│ │           applying changes."),\r\n    disable: List[Rule] = Option(default=[], help="Disable │ │
│ │           a rule."),\r\n    log_file: Union[Path, None] = Option(None, help="Log file to     │ │
│ │           write to."),\r\n    version: bool = Option(\r\n        None,\r\n                   │ │
│ │           "--version",\r\n        callback=version_callback,\r\n        is_eager=True,\r\n   │ │
│ │           help="Show the version and exit.",\r\n    ),\r\n):\r\n    # NOTE:                  │ │
│ │           LIBCST_PARSER_TYPE=native is required according to                                 │ │
│ │           https://github.com/Instagram/LibCST/issues/487.\r\n                                │ │
│ │           os.environ["LIBCST_PARSER_TYPE"] = "native"\r\n\r\n    console = Console()\r\n     │ │
│ │           files_str = list(package.glob("**/*.py"))\r\n    files =                           │ │
│ │           [str(file.relative_to(".")) for file in files_str]\r\n\r\n    providers =          │ │
│ │           {FullyQualifiedNameProvider, ScopeProvider}\r\n    metadata_manager =              │ │
│ │           FullRepoManager(".", files, providers=providers)  # type: ignore[arg-type]\r\n     │ │
│ │           metadata_manager.resolve_cache()\r\n\r\n    scratch: dict[str, Any] = {}\r\n       │ │
│ │           with Progress(*Progress.get_default_columns(), transient=True) as progress:\r\n    │ │
│ │           task = progress.add_task(description="Looking for Pydantic Models...",             │ │
│ │           total=len(files))\r\n        with multiprocessing.Pool() as pool:\r\n              │ │
│ │           partial_visit_class_def = functools.partial(visit_class_def, metadata_manager,     │ │
│ │           package)\r\n            for local_scratch in                                       │ │
│ │           pool.imap_unordered(partial_visit_class_def, files):\r\n                           │ │
│ │           progress.advance(task)\r\n                for key, value in                        │ │
│ │           local_scratch.items():\r\n                    scratch.setdefault(key,              │ │
│ │           value).update(value)\r\n\r\n    find_base_model(scratch)\r\n\r\n    start_time =   │ │
│ │           time.time()\r\n\r\n    codemods = gather_codemods(disabled=disable)\r\n\r\n        │ │
│ │           log_ctx_mgr = log_file.open("a+") if log_file else nullcontext()\r\n               │ │
│ │           partial_run_codemods = functools.partial(run_codemods, codemods, metadata_manager, │ │
│ │           scratch, package, diff)\r\n\r\n    with Progress(*Progress.get_default_columns(),  │ │
│ │           transient=True) as progress:\r\n        task =                                     │ │
│ │           progress.add_task(description="Executing codemods...", total=len(files))\r\n       │ │
│ │           with multiprocessing.Pool() as pool, log_ctx_mgr as log_fp:  # type:               │ │
│ │           ignore[attr-defined]\r\n            for error_msg in                               │ │
│ │           pool.imap_unordered(partial_run_codemods, files):\r\n                              │ │
│ │           progress.advance(task)\r\n                if error_msg is None:\r\n                │ │
│ │           continue\r\n\r\n                if log_fp is None:\r\n                             │ │
│ │           color_diff(console, error_msg)\r\n                else:\r\n                        │ │
│ │           log_fp.writelines(error_msg)\r\n\r\n            if log_fp:\r\n                     │ │
│ │           log_fp.write("Run successfully!\\n")\r\n\r\n    modified = [Path(f) for f in files │ │
│ │           if os.stat(f).st_mtime > start_time]\r\n    if modified:\r\n                       │ │
│ │           print(f"Refactored {len(modified)} files.")\r\n\r\n\r\ndef                         │ │
│ │           visit_class_def(metadata_manager: FullRepoManager, package: Path, filename: str)   │ │
│ │           -> Dict[str, Any]:\r\n    code = Path(filename).read_text()\r\n    module =        │ │
│ │           cst.parse_module(code)\r\n    module_and_package =                                 │ │
│ │           calculate_module_and_package(str(package), filename)\r\n\r\n    context =          │ │
│ │           CodemodContext(\r\n        metadata_manager=metadata_manager,\r\n                  │ │
│ │           filename=filename,\r\n        full_module_name=module_and_package.name,\r\n        │ │
│ │           full_package_name=module_and_package.package,\r\n    )\r\n    visitor =            │ │
│ │           ClassDefVisitor(context=context)\r\n    visitor.transform_module(module)\r\n       │ │
│ │           return context.scratch\r\n\r\n\r\ndef capture_exception(func: Callable[P, T]) ->   │ │
│ │           Callable[P, Union[T, Iterable[str]]]:\r\n    @functools.wraps(func)\r\n    def     │ │
│ │           wrapper(*args: P.args, **kwargs: P.kwargs) -> Union[T, Iterable[str]]:\r\n         │ │
│ │           try:\r\n            return func(*args, **kwargs)\r\n        except Exception as    │ │
│ │           exc:\r\n            func_args = [repr(arg) for arg in args]\r\n                    │ │
│ │           func_kwargs = [f"{key}={repr(value)}" for key, value in kwargs.items()]\r\n        │ │
│ │           return [f"{func.__name__}({\', \'.join(func_args +                                 │ │
│ │           func_kwargs)})\\n{exc}"]\r\n\r\n    return                                         │ │
│ │           wrapper\r\n\r\n\r\n@capture_exception\r\ndef run_codemods(\r\n    codemods:        │ │
│ │           List[Type[ContextAwareTransformer]],\r\n    metadata_manager: FullRepoManager,\r\n │ │
│ │           scratch: Dict[str, Any],\r\n    package: Path,\r\n    diff: bool,\r\n    filename: │ │
│ │           str,\r\n) -> Union[List[str], None]:\r\n    module_and_package =                   │ │
│ │           calculate_module_and_package(str(package), filename)\r\n    context =              │ │
│ │           CodemodContext(\r\n        metadata_manager=metadata_manager,\r\n                  │ │
│ │           filename=filename,\r\n        full_module_name=module_and_package.name,\r\n        │ │
│ │           full_package_name=module_and_package.package,\r\n    )\r\n                         │ │
│ │           context.scratch.update(scratch)\r\n\r\n    file_path = Path(filename)\r\n    with  │ │
│ │           file_path.open("r+") as fp:\r\n        code = fp.read()\r\n                        │ │
│ │           fp.seek(0)\r\n\r\n        input_tree = cst.parse_module(code)\r\n\r\n        for   │ │
│ │           codemod in codemods:\r\n            transformer = codemod(context=context)\r\n\r\n │ │
│ │           output_tree = transformer.transform_module(input_tree)\r\n            input_tree = │ │
│ │           output_tree\r\n\r\n        output_code = input_tree.code\r\n        if code !=     │ │
│ │           output_code:\r\n            if diff:\r\n                lines =                    │ │
│ │           difflib.unified_diff(\r\n                    code.splitlines(keepends=True),\r\n   │ │
│ │           output_code.splitlines(keepends=True),\r\n                                         │ │
│ │           fromfile=filename,\r\n                    tofile=filename,\r\n                     │ │
│ │           )\r\n                return list(lines)\r\n            else:\r\n                   │ │
│ │           fp.write(output_code)\r\n                fp.truncate()\r\n\r\n    return           │ │
│ │           None\r\n\r\n\r\ndef color_diff(console: Console, lines: Iterable[str]) ->          │ │
│ │           None:\r\n    for line in lines:\r\n        line = line.rstrip("\\n")\r\n        if │ │
│ │           line.startswith("+"):\r\n            console.print(line, style="green")\r\n        │ │
│ │           elif line.startswith("-"):\r\n            console.print(line, style="red")\r\n     │ │
│ │           elif line.startswith("^"):\r\n            console.print(line, style="blue")\r\n    │ │
│ │           else:\r\n            console.print(line, style="white")\r\n', 923, 924, 'character │ │
│ │           maps to <undefined>')                                                              │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in position 923: character maps to <undefined>

help="Convert Pydantic from V1 to V2 ♻️",

Solution

This issue is solved by specifying encoding="utf-8" in every file I/O operation. If accepted, I would like to work on this fix myself.

Not working on windows

Getting error 'bump-pydantic' is not recognized as an internal or external command,
operable program or batch file.

Potential Inconsistency with Inner Config Migration

First of all, I want to express my gratitude for developing this incredible tool. It has greatly assisted me in adopting Pydantic v2, and I have achieved exceptional results in my projects. Thank you for your valuable contribution! 🚀

I have encountered an issue where the bump-pydantic tool is exhibiting unexpected behavior by replacing the inner Config in non-Pydantic objects.

alice-biometrics/petisco-fastapi-example#11

To reproduce this behaviour you can use the following code (pip install petisco):

from meiga import BoolResult, Failure, Success
from petisco import Controller, DomainError


class IsNotPositive(DomainError):
    ...


class MyController(Controller):
    class Config:
        success_handler = lambda _: print("The result is positive")
        failure_handler = lambda _: print("The result is negative")

    def execute(self, value: int) -> BoolResult:
        if value < 0:
            return Failure(IsNotPositive())
        return Success(True)


result_success = MyController().execute(1)
result_failure = MyController().execute(-1)

print(result_success.transform())
print(result_failure.transform())

The Controller inherits from a metaclass class Controller(metaclass=MetaController) and this metaclass gets information from config

class MetaController(type, ABC):
    middlewares: list[Middleware] = []

    def __new__(
        mcs, name: str, bases: tuple[Any], namespace: dict[str, Any]
    ) -> MetaController:
        config = namespace.get("Config")

        mapper = get_mapper(bases, config)

        if "execute" not in namespace:
            raise NotImplementedError(
                "Petisco Controller must implement an execute method"
            )

        new_namespace = {}
        for attributeName, attribute in namespace.items():
            if isinstance(attribute, FunctionType) and attribute.__name__ == "execute":
                if iscoroutinefunction(attribute):
                    attribute = async_wrapper(attribute, name, config, mapper)
                else:
                    attribute = wrapper(attribute, name, config, mapper)
            new_namespace[attributeName] = attribute

        return super().__new__(mcs, name, bases, new_namespace)

    @abstractmethod
    def execute(self, *args: Any, **kwargs: Any) -> AnyResult:
        return NotImplementedMethodError

I guess bump-pydantic only have to update BaseModels objects. To ensure that it would be beneficial to introduce an additional check in the replace_config.py file.

Syntax error with built in library (multiprocessing)

Trying to run bump-pydantic with rocketry library which uses multiprocessing, and then this error caused by something inside multiprocessing.
Can't understand much what's going on based on error's output.


manimozaffar@Manis-MacBook-Pro-2 rocketry % bump-pydantic rocketry   
RemoteTraceback: 
"""
Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py", line
125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File 
"/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/bump_pydantic/ma
in.py", line 107, in visit_class_def
    module = cst.parse_module(code)
             ^^^^^^^^^^^^^^^^^^^^^^
  File 
"/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/libcst/_parser/e
ntrypoints.py", line 109, in parse_module
    result = _parse(
             ^^^^^^^
  File 
"/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/libcst/_parser/e
ntrypoints.py", line 55, in _parse
    return parse(source_str)
           ^^^^^^^^^^^^^^^^^
libcst._exceptions.ParserSyntaxError: Syntax Error @ 3:1.
parser error: error at 2:12: expected one of !=, %, &, (, *, **, +, ,, -, ., /, //, :, ;, <, <<, 
<=, =, ==, >, >=, >>, @, NEWLINE, [, ^, and, if, in, is, not, or, |

syntax error
           ^
"""

The above exception was the direct cause of the following exception:

╭────────────────────────────── Traceback (most recent call last) ──────────────────────────────╮
│ /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/bump_pydantic │
│ /main.py:70 in main                                                                           │
│                                                                                               │
│    67 │   │   task = progress.add_task(description="Looking for Pydantic Models...", total=le │
│    68 │   │   with multiprocessing.Pool() as pool:                                            │
│    69 │   │   │   partial_visit_class_def = functools.partial(visit_class_def, metadata_manag │
│ ❱  70 │   │   │   for local_scratch in pool.imap_unordered(partial_visit_class_def, files):   │
│    71 │   │   │   │   progress.advance(task)                                                  │
│    72 │   │   │   │   for key, value in local_scratch.items():                                │
│    73 │   │   │   │   │   scratch.setdefault(key, value).update(value)                        │
│                                                                                               │
│ ╭───────────────────────────────────────── locals ──────────────────────────────────────────╮ │
│ │                 console = <console width=97 ColorSystem.TRUECOLOR>                        │ │
│ │                    diff = False                                                           │ │
│ │                 disable = []                                                              │ │
│ │                   files = [                                                               │ │
│ │                           │   'rocketry/_base.py',                                        │ │
│ │                           │   'rocketry/session.py',                                      │ │
│ │                           │   'rocketry/__init__.py',                                     │ │
│ │                           │   'rocketry/application.py',                                  │ │
│ │                           │   'rocketry/exc.py',                                          │ │
│ │                           │   'rocketry/_setup.py',                                       │ │
│ │                           │   'rocketry/tasks/command.py',                                │ │
│ │                           │   'rocketry/tasks/code.py',                                   │ │
│ │                           │   'rocketry/tasks/__init__.py',                               │ │
│ │                           │   'rocketry/tasks/run_id.py',                                 │ │
│ │                           │   ... +214                                                    │ │
│ │                           ]                                                               │ │
│ │               files_str = [                                                               │ │
│ │                           │   PosixPath('rocketry/_base.py'),                             │ │
│ │                           │   PosixPath('rocketry/session.py'),                           │ │
│ │                           │   PosixPath('rocketry/__init__.py'),                          │ │
│ │                           │   PosixPath('rocketry/application.py'),                       │ │
│ │                           │   PosixPath('rocketry/exc.py'),                               │ │
│ │                           │   PosixPath('rocketry/_setup.py'),                            │ │
│ │                           │   PosixPath('rocketry/tasks/command.py'),                     │ │
│ │                           │   PosixPath('rocketry/tasks/code.py'),                        │ │
│ │                           │   PosixPath('rocketry/tasks/__init__.py'),                    │ │
│ │                           │   PosixPath('rocketry/tasks/run_id.py'),                      │ │
│ │                           │   ... +214                                                    │ │
│ │                           ]                                                               │ │
│ │                     key = 'class_def_visitor'                                             │ │
│ │           local_scratch = {'class_def_visitor': defaultdict(<class 'set'>, {})}           │ │
│ │                log_file = None                                                            │ │
│ │        metadata_manager = <libcst.metadata.full_repo_manager.FullRepoManager object at    │ │
│ │                           0x103bffa90>                                                    │ │
│ │                 package = PosixPath('rocketry')                                           │ │
│ │ partial_visit_class_def = functools.partial(<function visit_class_def at 0x103b4d6c0>,    │ │
│ │                           <libcst.metadata.full_repo_manager.FullRepoManager object at    │ │
│ │                           0x103bffa90>, PosixPath('rocketry'))                            │ │
│ │                    pool = <multiprocessing.pool.Pool state=TERMINATE pool_size=10>        │ │
│ │                progress = <rich.progress.Progress object at 0x103e788d0>                  │ │
│ │               providers = {                                                               │ │
│ │                           │   <class                                                      │ │
│ │                           'libcst.metadata.name_provider.FullyQualifiedNameProvider'>,    │ │
│ │                           │   <class 'libcst.metadata.scope_provider.ScopeProvider'>      │ │
│ │                           }                                                               │ │
│ │                 scratch = {                                                               │ │
│ │                           │   'class_def_visitor': defaultdict(<class 'set'>, {           │ │
│ │                           │   │   'rocketry.exc.SchedulerRestart': {                      │ │
│ │                           │   │   │   'builtins.Exception'                                │ │
│ │                           │   │   },                                                      │ │
│ │                           │   │   'rocketry.exc.SchedulerExit': {'builtins.Exception'},   │ │
│ │                           │   │   'rocketry.exc.TaskInactionException': {                 │ │
│ │                           │   │   │   'builtins.Exception'                                │ │
│ │                           │   │   },                                                      │ │
│ │                           │   │   'rocketry.exc.TaskTerminationException': {              │ │
│ │                           │   │   │   'builtins.Exception'                                │ │
│ │                           │   │   },                                                      │ │
│ │                           │   │   'rocketry.exc.TaskSetupError': {'builtins.Exception'},  │ │
│ │                           │   │   'rocketry.exc.TaskLoggingError': {                      │ │
│ │                           │   │   │   'builtins.Exception'                                │ │
│ │                           │   │   },                                                      │ │
│ │                           │   │   'rocketry.tasks.code.CodeTask': {                       │ │
│ │                           │   │   │   'rocketry.core.Task'                                │ │
│ │                           │   │   },                                                      │ │
│ │                           │   │   'rocketry.tasks.maintain.os.Restart': {                 │ │
│ │                           │   │   │   'rocketry.core.task.Task'                           │ │
│ │                           │   │   },                                                      │ │
│ │                           │   │   'rocketry.tasks.maintain.os.ShutDown': {                │ │
│ │                           │   │   │   'rocketry.core.task.Task'                           │ │
│ │                           │   │   },                                                      │ │
│ │                           │   │   'rocketry.tasks.command.CommandTask': {                 │ │
│ │                           │   │   │   'rocketry.core.task.Task'                           │ │
│ │                           │   │   },                                                      │ │
│ │                           │   │   ... +30                                                 │ │
│ │                           │   })                                                          │ │
│ │                           }                                                               │ │
│ │                    task = 0                                                               │ │
│ │                   value = defaultdict(<class 'set'>, {})                                  │ │
│ │                 version = None                                                            │ │
│ ╰───────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                               │
│ /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py:873 │
│ in next                                                                                       │
│                                                                                               │
│   870 │   │   success, value = item                                                           │
│   871 │   │   if success:                                                                     │
│   872 │   │   │   return value                                                                │
│ ❱ 873 │   │   raise value                                                                     │
│   874 │                                                                                       │
│   875 │   __next__ = next                    # XXX                                            │
│   876                                                                                         │
│                                                                                               │
│ ╭───────────────────────────────────────── locals ──────────────────────────────────────────╮ │
│ │    item = (                                                                               │ │
│ │           │   False,                                                                      │ │
│ │           │   ParserSyntaxError('parser error: error at 2:12: expected one of !=, %, &,   │ │
│ │           (, *, **, +, ,, -, ., /, //, :, ;, <, <<, <=, =, ==, >, >=, >>, @, NEWLINE, [,  │ │
│ │           ^, and, if, in, is, not, or, |', lines=[...], raw_line=3, raw_column=12)        │ │
│ │           )                                                                               │ │
│ │    self = <multiprocessing.pool.IMapUnorderedIterator object at 0x103e79250>              │ │
│ │ success = False                                                                           │ │
│ │ timeout = None                                                                            │ │
│ │   value = ParserSyntaxError('parser error: error at 2:12: expected one of !=, %, &, (, *, │ │
│ │           **, +, ,, -, ., /, //, :, ;, <, <<, <=, =, ==, >, >=, >>, @, NEWLINE, [, ^,     │ │
│ │           and, if, in, is, not, or, |', lines=[...], raw_line=3, raw_column=12)           │ │
│ ╰───────────────────────────────────────────────────────────────────────────────────────────╯ │
╰───────────────────────────────────────────────────────────────────────────────────────────────╯
ParserSyntaxError: Syntax Error @ 3:1.
parser error: error at 2:12: expected one of !=, %, &, (, *, **, +, ,, -, ., /, //, :, ;, <, <<, 
<=, =, ==, >, >=, >>, @, NEWLINE, [, ^, and, if, in, is, not, or, |

Bug: bump_pydantic convert constr v1 regex to v2 pattern

Initial Checks

  • I confirm that I'm using Pydantic V2

Description

bump_pydantic script should convert constr "regex" to "pattern"

i ran the bump_pydantic on my code base, and it did not convert anything, it seems it should at least convert constr(regex='') to constr(pattern='')

i am not sure if there is other stuff that bump_pydantic is missing,

Example Code

import pydantic

KUBE_SAVE_STR_30 = pydantic.constr(pattern="^(?![0-9]+$)(?!-)[a-z0-9-]{,30}(?<!-)$")

Python, Pydantic & OS Version

python 3.9

(venv) [sschultchen@sschultchen2pc common_Argocd]$ pip freeze
annotated-types==0.5.0
bump-pydantic==0.3.0
certifi==2023.5.7
charset-normalizer==3.1.0
click==8.1.4
idna==3.4
libcst==1.0.1
markdown-it-py==3.0.0
mdurl==0.1.2
mypy==1.4.1
mypy-extensions==1.0.0
pydantic==2.0.2
pydantic_core==2.1.2
Pygments==2.15.1
PyYAML==6.0
requests==2.31.0
rich==13.4.2
typer==0.9.0
typing-inspect==0.9.0
typing_extensions==4.6.3
urllib3==2.0.3

Selected Assignee: @adriangb

GenericModel (aliased import) not replaced

Hello,

I ran the migration script, and promptly ran our tests, and they failed due to the following error:

ImportError while importing test module '/.....py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/......py:6: in <module>
    from api.common.model import AppException
api/.......py:12: in <module>
    from pydantic.generics import GenericModel as PyGenericModel  # noqa
E   ImportError: cannot import name 'GenericModel' from 'pydantic.generics' (/...../lib/python3.10/site-packages/pydantic/generics.py)

My guess is that perhaps the migration script doesn't handle as imports?

If any more details or repro case is useful, I'll be happy to oblige.

Mark methods on arbitrary classes

pydantic specific methods like __get_validators__ should be marked with a TODO on arbitrary classes to make migration of custom types easier

Imports ConfigDict from pydantic instead from pydantic_settings

As a follow-up from this non-issue in pydantic_settings, I want to file this issue here. pydantic-bump seem to import ConfigDict from wrong package. And – if I got it right – it should import SettingsConfigDict in my example instead of ConfigDict.

I had this config in Pydantic v1:

from typing import Literal
from pydantic import BaseSettings


class Config(BaseSettings):
    log_level: Literal["NOTSET", "DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] = "WARNING"

    class Config:
        env_file = ".env"

Then, I executed dump-pydantic and got the following config file:

from typing import Literal
from pydantic import ConfigDict
from pydantic_settings import BaseSettings


class Config(BaseSettings):
    log_level: Literal["NOTSET", "DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] = "WARNING"
    model_config = ConfigDict(env_file=".env")

As one can see bump-pydantic imported ConfigDict from pydantic instead from pydantic_settings. And as the example uses SettingsConfigDict, I think bump-pydantic should use that as well.

Missing imports

from pydantic.typing import ForwardRef
from pydantic.typing import evaluate_forwardref

these two not handled

@root_validator will not updated to model_validator

Description

I have a @root_validator without having any parameter in my project like this:

from pydantic import BaseModel, root_validator

class Model(BaseModel):
    name: str

    @root_validator
    def validate_root(cls, values):
        return values

In this way, bump-pydantic can't detect it and will not update it to a new version and without any TODO or anything.
Also, if I define my root_validator like @root_validator(), it will be detected and converted to @model_validator() without defining mode parameter.
The only successful convert will happen if I have @root_validator(pre=True). I've not checked other scenarios.

If it is an accepted bug, I will be happy to work on it and make a PR

My Environment

Python 3.11.4
bump-pydantic==0.7.0

On Windows machines with >= 32 cores (more than 64 logical processors) execution fails

Shown as

(venv) my_dir>bump-pydantic --version      
bump-pydantic version: 0.7.0

(venv) my_dir>bump-pydantic .         
[13:44:38] Start bump-pydantic.                                                                                                                                                                                                                                                                                                                                        main.py:61
           Found 10 files to process                                                                                                                                                                                                                                                                                                                                   main.py:76
Exception in thread Thread-3 (_handle_workers):
Traceback (most recent call last):
  File "C:\_\py311\Lib\threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "C:\_\py311\Lib\threading.py", line 975, in run
    self._target(*self._args, **self._kwargs)
  File "C:\_\py311\Lib\multiprocessing\pool.py", line 522, in _handle_workers
    cls._wait_for_updates(current_sentinels, change_notifier)
  File "C:\_\py311\Lib\multiprocessing\pool.py", line 502, in _wait_for_updates
    wait(sentinels, timeout=timeout)
  File "C:\_\py311\Lib\multiprocessing\connection.py", line 878, in wait
    ready_handles = _exhaustive_wait(waithandle_to_obj.keys(), timeout)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\_\py311\Lib\multiprocessing\connection.py", line 810, in _exhaustive_wait
    res = _winapi.WaitForMultipleObjects(L, False, timeout)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: need at most 63 handles, got a sequence of length 66
Executing codemods... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━  80% 0:00:01Process SpawnPoolWorker-64:

Afterwards it freezes without any processing. Even Ctrl+C doesn't work.

Googling for it suggests that this is a common problem with multiprocessing on Windows.

Error for line with type comment

For example:

$bump-pydantic modules
CompileError: modules/utils.py:152: error: invalid syntax [syntax]

modules/utils.py:152

    if password := conf.password: # type: SecretStr 
        pwd_kwds.update(password=password.get_secret_value())

If I drop comment - error done

Detect missing pydantic dependency (e.g. pydantic-settings or extra types)

Hey team,

I wasn't sure if you would be comfortable with this, but what's your opinion on telling the user when they are missing a pydantic dependency, such as pydantic-settings is using BaseSettings in Pydantic V1?

This could spit out a warning (whether that's stdout, stderr, log.txt, or somewhere else) or a # TODO comment if the pydantic-settings package isn't installed.

This will help users more quickly identify when their application requires packages that are not currently installed.

To detect if a package is installed without importing it (for safety reasons), importlib.util can be used:

from importlib.util import find_spec

print(find_spec("pydantic-settings"))
# ''

print(find_spec("pydantic"))
# 'ModuleSpec(name='pydantic', loader=<_frozen_importlib_external.SourceFileLoader object at 0x104e18240>, origin='/Users/kkirsche/.asdf/installs/python/3.11.4/lib/python3.11/site-packages/pydantic/__init__.py', submodule_search_locations=['/Users/kkirsche/.asdf/installs/python/3.11.4/lib/python3.11/site-packages/pydantic'])'

Documentation on find_spec:
https://docs.python.org/3/library/importlib.html#importlib.util.find_spec

What changes should the codemod do?

Checklist

  • Config class to model_config attribute on BaseModel.
  • Replace imports that changed location .e.g. pydantic.json to pydantic.deprecated.json. I'm not doing this, since it's changing one that is deprecated, for the other that is deprecated as well.
  • Replace method names.
  • Add None default value to Union[T, None], Optional[None] and Any.
  • Field keywords renaming.
  • Remove Extra import, and replace by string e.g. Extra.allow -> "allow".
  • Warn on Color and PaymentMethod when not using pydantic-extra-types. Replace Color and PaymentCardNumber from pydantic to pydantic-extra-types.
  • Replace from pydantic import BaseSettings by from pydantic_settings import BaseSettings.
  • Replace Field(env=<str>) by Field(validation_alias=<str>).
  • Replace GenericModel, Generic[T] by BaseModel, Generic[T].
  • Replace __root__ to RootModel.

The migration guide may help here.

Copy on model validation

I had a copy_on_model_validation flag set to "none" on my pydantic models and could not find any documentation on the migration page on how to deal with this setting in v2. Seems from v1 discussions on issues raised on github that this is now irrelevant since Pydantic will never copy anymore when validating models, also considered by @PrettyWood to be incorrect behaviour.

Could save other people quite some time if this would make it to the pydantic migration docs and/or flagged by bump-pydantic as a superfluous setting now

class DefaultBaseModel(BaseModel):
-    class Config:
-        arbitrary_types_allowed = True
-        copy_on_model_validation = "none"
-        validate_assignment = True
+    # TODO: The following keys were removed: `copy_on_model_validation`.
+    # Check https://docs.pydantic.dev/dev-v2/migration/#changes-to-config for more information.
+    model_config = ConfigDict(arbitrary_types_allowed=True, copy_on_model_validation="none", validate_assignment=True)

In the ConfigDict this setting should not even be included anymore, it won't work either I presume.

[BUG]: bumb-pydantic doesnt change files

OS: Wndows 11
IDE: Cursor

I am in a fresh fork of patito in local dev, using a conda env.

THHs depends on pydantic v1 and I am looking to change to support v2, thus using bump-pydantic

(venv_patito)PS C:\Users\<user>\<wd>\patito> bump-pydantic src/patito
[10:38:16] Start bump-pydantic.                                                                                         main.py:61
           Found 10 files to process                                                                                    main.py:76
[10:38:32] Run successfully!                                                                                           main.py:149
(venv_patito)PS C:\Users\<user>\<wd>\patito> 

However, git doesn't track any changes and only generates an empty log.txt file. I've also been in src directory with the same outcome.

Shouldn't I get an error somewhere?

Add default `None` when using Field

Hey,

assume following code:

from typing import Optional

from pydantic import BaseModel, Field


class User(BaseModel):
    name: Optional[str]
    last_name: Optional[str] = Field(description="Some desc")

Running bump-pydantic on it, will add a None as default to name, but unfortunately it will not add a parameter to Field.

from typing import Optional

from pydantic import BaseModel, Field


class User(BaseModel):
    name: Optional[str]
    last_name: Optional[str] = Field(description="Some desc")

Expected output would be:

from typing import Optional

from pydantic import BaseModel, Field


class User(BaseModel):
    name: Optional[str] = None
    last_name: Optional[str] = Field(default=None, description="Some desc")

Thanks a lot for pydantic and bump-pydantic!

fin swimmer

License info missing in classifiers and/or meta data

Sorry to bother you.
Larger enterprises scan packages before allowing them into their internal pypi index, and one of the things they scan for is the license information. This is typically done via scanning the meta data (whatever pip show reports) and classifiers in pyproject.toml. Please can you add the necessary license information in both places? Thanks much!
You can mirror what the main pydantic package is doing because that one is working and passed our internal scanner.

BP008: Not Handling conint(ge= , lt=) Correctly

Python version: 3.10.12
bump-pydantic version: 0.6.1
Pydantic version: 2.3.0

I installed and ran bump-pydantic per the project's README.md file instructions against my FastAPI project*. The resulting diff doesn't handle conint(ge=, lt=) correctly.

Before running bump-pydantic, one of the models looks like:

 class GuestAppearance(BaseModel):
     """Appearance Information"""
 
    show_id: conint(ge=0, lt=2**31) = Field(title="Show ID")

The diff generated by bump-pydantic returns:

 class GuestAppearance(BaseModel):
     """Appearance Information"""
 
    show_id: Annotated = Field(title="Show ID")

It is losing both the int type and the ge and lt constraints. Replacing the expression 2**31 with 2147483648 doesn't change how bump-pydantic handles the conint per the following diff:

 class GuestAppearance(BaseModel):
     """Appearance Information"""
 
-    show_id: conint(ge=0, lt=2147483648) = Field(title="Show ID")
+    show_id: Annotated = Field(title="Show ID")

condecimal to Field issue

Hi, I'm experiencing some issues after making the changes with bump-pydantic

Before (this is working)

from pydantic import BaseModel, condecimal, Field

class ExpenseBase(BaseModel):
    name: str = Field(max_length=25)
    description: str | None = Field(max_length=512)
    amount: condecimal(max_digits=10, decimal_places=2)

class CreateExpense(ExpenseBase):
    pass

class CreateExpensePercentage(CreateExpense):
    users: dict[str, condecimal(max_digits=10, decimal_places=2)]

After (error)

from pydantic import BaseModel, Field
from decimal import Decimal
from typing_extensions import Annotated

class ExpenseBase(BaseModel):
    name: str = Field(max_length=25)
    description: str | None = Field(max_length=512)
    amount: Annotated[Decimal, Field(max_digits=10, decimal_places=2)]

class CreateExpense(ExpenseBase):
    pass

class CreateExpensePercentage(CreateExpense):
    users: dict[str, Field(max_digits=10, decimal_places=2)]

I receive:

pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for FieldInfo(annotation=NoneType, required=True, metadata=[PydanticGeneralMetadata(max_digits=10, decimal_places=2)]). Set `arbitrary_types_allowed=True` in the model_config to ignore this error or implement `__get_pydantic_core_schema__` on your type to fully support it.

If you got this error by calling handler(<some type>) within `__get_pydantic_core_schema__` then you likely need to call `handler.generate_schema(<some type>)` since we do not call `__get_pydantic_core_schema__` on `<some type>` otherwise to avoid infinite recursion.

For further information visit https://errors.pydantic.dev/2.0.3/u/schema-for-unknown-type

After adding "arbitrary_types_allowed=True"

from pydantic import BaseModel, Field, ConfigDict
from decimal import Decimal
from typing_extensions import Annotated

class ExpenseBase(BaseModel):
    name: str = Field(max_length=25)
    description: str | None = Field(max_length=512)
    amount: Annotated[Decimal, Field(max_digits=10, decimal_places=2)]

class CreateExpense(ExpenseBase):
    pass

class CreateExpensePercentage(CreateExpense):
    users: dict[str, Field(max_digits=10, decimal_places=2)]

    model_config = ConfigDict(arbitrary_types_allowed=True)

I receive:

/home/javier/.local/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py:628: UserWarning: FieldInfo(annotation=NoneType, required=True, metadata=[PydanticGeneralMetadata(max_digits=10, decimal_places=2)]) is not a Python type (it may be an instance of an object), Pydantic will allow any object with no validation since we cannot even enforce that the input is an instance of the given type. To get rid of this error wrap the type with `pydantic.SkipValidation`.
  warn(

Non-deterministic (and sometimes incorrect) finding of BaseModel classes

When trying to run BP001 (add default None) on our project, I've noticed something interesting.
The part when the tool is "Looking for Pydantic Models" with ClassDefVisitor yields different results when running more than once. I think I've managed to boil it down to a minimal example.

Setup

Lets consider 4 simple files: a.py, common.py, utils.py and z.py like so:
==> common.py <==

from pydantic import BaseModel

class NewBaseModel(BaseModel):
    foo: str | None

==> utils.py <==

from common import NewBaseModel

class CustomBaseModel(NewBaseModel):
    bar: str | None

==> a.py <==

from utils import CustomBaseModel

class AModel(CustomBaseModel):
    a: str | None

==> z.py <==

from utils import CustomBaseModel

class ZModel(CustomBaseModel):
    z: str | None

(apologies for terrible and potentially unintuitive naming 😅)

Requirements

There are 2 important things to note here:

  • There are at least 2 levels of inheritance from pydantic's BaseModel. That is: z.ZModel and a.AModel -> utils.CustomBaseModel -> common.NewBaseModel -> pydantic.BaseModel. Apparently this is important as the problem doesn't seem to occur when we get rid of one level (common.NewBaseModel for example).
  • Two "normal" modules, a and z, are listed respectively before and after the utility modules (common and utils) when processing.

Expected outcome

Running bump-pydantic . on such "project" should add None as a default value for fields in all 4 modules.

Observed outcome

But it turns out it does that in (more or less) half of the runs. I've run the tool 100 times and only 47 of them ended up in modifying all four files, 15 times it modified 3 of them (common.py, utils.py and z.py) and 38 times only 2 of them (common.py and utils.py).

Problem research

I see two issues here:

  1. non-deterministic behaviour and
  2. incorrect finding of BaseModel classes.

Looks like the problem occurs because of the way the queue is being constructed and the way files are being added to it later in

missing_files = set(files) - visited
if not queue and missing_files:
queue.append(next(iter(missing_files)))

since set by nature is unsorted, files will be added to the queue in a "random" order.
Although a.py will be processed first every time.
Fixing that non-determinism should be relatively simple. For example, we could make the queue initially the same as files, appendleft files coming from visitor.next_file and get rid of the missing_files bit.

Now, once we fix that, AModel will never get marked as a pydantic model, at least with the way I've fixed the first issue. Which I haven't figured out why, yet. I mean, it does get marked correctly ~50% of the times currently, so... 🤷. I haven't fully grokked the ClassDefVisitor code yet. Seems like there is some consideration for those cases there already.

For solutions I was considering either going through all of the files twice or allowing callers to specify fqns of custom base models.

I can open a PR to better demonstrate what I mean here.
This is what I mean about that non-determinism fix: 13bfe61 (from #118).

Feature request: process file individually

Currently, bump-pydantic only allows the user to pass a directory where to search for source files. Allowing the user to pass a file path directly would be helpful for incremental changes or viewing a diff for a single file.

If this feature is approved, I would like to work on it.

Error when module got re-referenced

app\services\inbox\Models\InboxEnum.py: error: Source file found twice under different module names: "InboxEnum" and "app.services.inbox.Models.InboxEnum"

InboxEnum is a set of StrEnum models
InboxEnum is in app\services\inbox\Models
InboxEnum is imported in multiple other files in app\services\inbox\Models

Set libcst dependency version

Hi, I installed this project into a venv that already had libcst==0.3 installed, but it didn't work because the calculate_module_and_package function was only introduced in in libcst 0.4.2

Literals trigger hard to understand errors

Hi,

I found 2 literals that trigger an error: in and On Hold. I saw a closed issue for "in", but I'm not sure it was fixed or just dismissed.

Anyway this is the error message:

    return parse(source_str)
           ^^^^^^^^^^^^^^^^^
libcst._exceptions.ParserSyntaxError: Syntax Error @ 1:1.
parser error: error at 1:2: expected one of (, *, +, -, ..., AWAIT, EOF, False, NUMBER, None, True, [, break, continue, lambda, match, not, pass, ~

in
^

Could bump-pydantic produce more helpful error message, e.g. with the whole line and line number? I took me a while to understand which "in" is the problem, because running grep in produced a lot of output :)

These are the reproducers:
1.

from typing import Literal

from pydantic import BaseModel


class MySuperPhoneObject(BaseModel):

    state: Literal["Closed", "On Hold", "Speaking", "Ringing"]

from typing import Literal

from pydantic import BaseModel


class MySuperPBXQueueObject(BaseModel):

    state: Literal["in", "out"]

Wrong conversion of allow_mutation

  • colour: t.Optional[t.Union[Colour, Finish]] = Field(allow_mutation=False)
  • colour: t.Optional[t.Union[Colour, Finish]] = Field(frozen=False)

Frozen should get value True here

Feature request: ignore patterns for files or directories

Currently, bump-pydantic considers all files ending in .py when scanning and applying modifications. In a codebase where source files are located at the project root, and a virtual environment is present, bump-pydantic takes a very long time to go through everything in the virtual environment's site-packages in addition to the actual project source files.

Allowing the user to specify patterns to ignore when scanning, using a command-line argument, will solve this problem.

Further, some default patterns may be put in place, for example:

  • Ignoring .venv as it is a common name for virtual environment directories, or
  • Ignoring everything in .gitignore if it is present, if not falling back to the previous point

If the user wishes to suppress default ignore patterns, they might do so using --ignore="" or something similar.
These implementation details are up to the maintainers to decide.

If this feature is approved, I would like to work on it.

Conversion of conint is losing the type int

I think I have a small issue for BP008 part.

This model:

class MyModel(BaseModel):
    # ...
    some_id: Optional[conint(ge=1, le=4294967295)] = None

will be turned into:

class MyModel(BaseModel):
    # ...
    some_id: Optional[Field(ge=1, le=4294967295)] = None

At runtime, we will get this exception:
TypeError: typing.Optional requires a single type. Got FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1), Le(le=4294967295)])

I was expecting a model along the line of:

class MyModel(BaseModel):
    # ...
    some_id: Optional[Annotated[int, Field(ge=1, le=4294967295)]] = None

Raising cryptic error message: `parser error: error at 1:5: expected NAME class ^`

Description

I would like to run bump-pydantic on the repository I am working on. After running it on the repository (.) I receive following error (truncated because of duplicates):

An error happened on v1/annotation/classes.py.
Traceback (most recent call last):
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/bump_pydantic/main.py", line 126, in run_codemods
    output_tree = transformer.transform_module(input_tree)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/codemod/_codemod.py", line 108, in transform_module
    return self.transform_module_impl(tree_with_metadata)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/codemod/_visitor.py", line 32, in transform_module_impl
    return tree.visit(self)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/module.py", line 90, in visit
    result = super(Module, self).visit(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/base.py", line 218, in visit
    should_visit_children = visitor.on_visit(self)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/matchers/_visitors.py", line 511, in on_visit
    return CSTTransformer.on_visit(self, node)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_visitors.py", line 44, in on_visit
    retval = visit_func(node)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/codemod/visitors/_remove_imports.py", line 300, in visit_Module
    node.visit(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/module.py", line 90, in visit
    result = super(Module, self).visit(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/base.py", line 218, in visit
    should_visit_children = visitor.on_visit(self)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/matchers/_visitors.py", line 718, in on_visit
    return CSTVisitor.on_visit(self, node)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_visitors.py", line 123, in on_visit
    retval = visit_func(node)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/codemod/visitors/_gather_unused_imports.py", line 71, in visit_Module
    node.visit(annotation_visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/module.py", line 90, in visit
    result = super(Module, self).visit(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/module.py", line 74, in _visit_and_replace_children
    body=visit_body_sequence(self, "body", self.body, visitor),
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/internal.py", line 227, in visit_body_sequence
    return tuple(visit_body_iterable(parent, fieldname, children, visitor))
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/internal.py", line 193, in visit_body_iterable
    new_child = child.visit(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/statement.py", line 1931, in _visit_and_replace_children
    body=visit_required(self, "body", self.body, visitor),
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/internal.py", line 81, in visit_required
    result = node.visit(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/statement.py", line 697, in _visit_and_replace_children
    body=visit_body_sequence(self, "body", self.body, visitor),
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/internal.py", line 227, in visit_body_sequence
    return tuple(visit_body_iterable(parent, fieldname, children, visitor))
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/internal.py", line 193, in visit_body_iterable
    new_child = child.visit(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/statement.py", line 442, in _visit_and_replace_children
    body=visit_sequence(self, "body", self.body, visitor),
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/internal.py", line 177, in visit_sequence
    return tuple(visit_iterable(parent, fieldname, children, visitor))
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/internal.py", line 159, in visit_iterable
    new_child = child.visit(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/statement.py", line 1542, in _visit_and_replace_children
    annotation=visit_required(self, "annotation", self.annotation, visitor),
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/internal.py", line 81, in visit_required
    result = node.visit(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/expression.py", line 1673, in _visit_and_replace_children
    annotation=visit_required(self, "annotation", self.annotation, visitor),
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/internal.py", line 81, in visit_required
    result = node.visit(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/expression.py", line 1604, in _visit_and_replace_children
    slice=visit_sequence(self, "slice", self.slice, visitor),
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/internal.py", line 177, in visit_sequence
    return tuple(visit_iterable(parent, fieldname, children, visitor))
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/internal.py", line 159, in visit_iterable
    new_child = child.visit(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/expression.py", line 1549, in _visit_and_replace_children
    slice=visit_required(self, "slice", self.slice, visitor),
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/internal.py", line 81, in visit_required
    result = node.visit(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/base.py", line 227, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/expression.py", line 1463, in _visit_and_replace_children
    value=visit_required(self, "value", self.value, visitor),
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/internal.py", line 81, in visit_required
    result = node.visit(visitor)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_nodes/base.py", line 218, in visit
    should_visit_children = visitor.on_visit(self)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/matchers/_visitors.py", line 718, in on_visit
    return CSTVisitor.on_visit(self, node)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_visitors.py", line 123, in on_visit
    retval = visit_func(node)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/codemod/visitors/_gather_string_annotation_names.py", line 65, in visit_SimpleString
    self.handle_any_string(node)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/codemod/visitors/_gather_string_annotation_names.py", line 74, in handle_any_string
    mod = cst.parse_module(value)
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_parser/entrypoints.py", line 109, in parse_module
    result = _parse(
  File "/home/XXX/.cache/pypoetry/virtualenvs/models-fO5v81y7-py3.8/lib/python3.8/site-packages/libcst/_parser/entrypoints.py", line 55, in _parse
    return parse(source_str)
libcst._exceptions.ParserSyntaxError: Syntax Error @ 1:1.
parser error: error at 1:5: expected NAME

class
^
...

I can get anything from the error message: what does it refer to?

I was looking at the libcst library issues too, but couldn't find anything related to that issue neither.

Steps to reproduce

  • git clone [email protected]:empaia/services/models.git
  • poetry shell
  • pip install bump-pydantic
  • poetry install
  • bump-pydantic .

Versions:

  • python 3.8
  • poetry 1.5.0
  • pip 23.1.2
  • bump-pydantic 0.3.0

Acceptance Criteria

  • Error message is clearer
  • Ideally, runs through without any errors.

RecursionError when processing file with a large number of enums

I'm getting the following error:
RecursionError: maximum recursion depth exceeded while calling a Python object
When running bump-pydantic on a file containing a very large number of enum.
Attached is a sample file that causes the problem. If you uncomment the last line the error will occur.
I'm using bump-pydantic 0.6.1 with python 3.9.12
enums.py.zip

Field(example=...) needs a codemod.

In V1, there is an example argument in the Field constructor. For V2, this field was renamed to examples (with an s), and changed to be a list of examples.

Enable pipeline

It looks like there's a need for enabling the pipeline on the repository settings 🤔

Required Optional field that uses `Field(...)` is made Non-Required

Required Optional Fields (eg. fields that are allowed to be None but should explicitly be specified) are made non-Required when Field + ... ellipse was used.

For example, using bump-pydantic on the official pydantic v1.10 example: https://docs.pydantic.dev/1.10/usage/models/#required-optional-fields

Turns the following code:

from typing import Optional
from pydantic import BaseModel, Field, ValidationError

class Model(BaseModel):
    a: Optional[int]
    b: Optional[int] = ...
    c: Optional[int] = Field(...)

Into:

from typing import Optional
from pydantic import BaseModel, Field, ValidationError

class Model(BaseModel):
    a: Optional[int] = None
    b: Optional[int] = ...
    c: Optional[int] = Field(None)

While one would have expected:

from typing import Optional
from pydantic import BaseModel, Field, ValidationError

class Model(BaseModel):
    a: Optional[int] = None
    b: Optional[int] = ...
    c: Optional[int] = Field(...)

bump-pydantic version = v0.6.1 (as of writing the lastest version)

Custom fields

Hi

do you have any plans on upgrading custom fields ?

class PostCode(str):

    @classmethod
    def __get_validators__(cls):
           yield cls.validate
    
    @classmethod
    def validate(cls, v):
       ...

What will happen to things that got removed ?

Hi

Looking forward for this tool

a question - what will happen to code in case some feature or attribute is removed ?

like let's take this example:

class Foo(BaseModel):
   class Config:
         getter_dict = SomeClass  # getter_dict is no longer present in pydantic v2

I guess in such cases it should ask user for some options like remove it or just stop(exit 1) ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.