jameshutchison/megamock

The developer experience (DevX) upgrade for Python mocking

megamock's Issues

Stop all MegaPatches is incompatible with autouse session patches

The pytest plugin currently has no segmentation between the different patches. stop_all_megapatches will disable session patches as well. It may make sense to refactor it so that patches live under a scope.

Improve typing

Currently, MegaMock objects are naive and don't have the attributes of the spec or wrapped object. Ideally, they do.

Track comparisons of `MegaMock` objects

This feature would record the stack traces for comparisons against attributes and record the corresponding "other" object and the result

Add performance helpers

When optimizing performance in tests, one thing you can do is take things that are deterministic, but expensive (such as encryption related functions) and cache them.

For example:

@pytest.fixture(scope="session", autouse=True)
def cache_derive_key() -> Iterable[None]:
    orig_derive_key = derive_key

    @functools.lru_cache()
    def derive_key_with_memoize(password: str, salt: str) -> bytes:
        return orig_derive_key(password, salt)

    MegaPatch.it(derive_key, new=derive_key_with_memoize)

To leverage this, you probably also have to make things that are random deterministic. Example:

uuids_stable = [uuid.uuid4() for _ in range(500)]


@pytest.fixture(autouse=True)
def deterministic_uuids() -> None:
    uuids = copy.copy(uuids_stable)
    MegaPatch.it(uuid.uuid4, new=lambda: uuids.pop())

This issue is to make generic functions to create these kinds of patches. I would imagine unique values can get created on demand rather than pre-making them. Then every test, the uuid4 will return the same values in the same order (similar to a side_effect)

Another example. This one does a cached list for each unique arguments.

pk_cache: defaultdict[
    tuple[tuple, tuple[tuple[str, Any], ...]], list[RSAPrivateKey]
] = defaultdict(list)


@pytest.fixture(autouse=True)
def deterministic_private_key():
    orig_generate_private_key = rsa.generate_private_key
    # create shallow copy
    cache = {k: copy.copy(v) for k, v in pk_cache.items()}

    def generate_private_key_with_per_test_cache(*args, **kwargs) -> rsa.RSAPrivateKey:
        key = (args, tuple(sorted(kwargs.items())))
        if result := cache.get(key):
            return result.pop(0)
        pk = orig_generate_private_key(*args, **kwargs)
        pk_cache[key].append(pk)
        return pk

    MegaPatch.it(rsa.generate_private_key, new=generate_private_key_with_per_test_cache)

Add async support

Async support is currently not tested / supported

Investigate porting `import_references.py` to Rust

The import references are a static object, consisting of sets, dictionaries, and strings. It seems like there could be a substantial performance improvement by porting this to Rust.

https://github.com/JamesHutchison/megamock/blob/main/megamock/import_references.py

I'm curious how integrating Rust works anyways. We can potentially use the existing Python code as a fallback.

Align class and instance mocks

The expected behavior for this:

MegaPatch(Foo)
Mega(Foo.some_method).use_real_logic()

Would be that Foo("s").some_method() uses the real logic.

Instead it returns a MegaMock object. The reason is that some_method is the class mock here instead of the instance_mock. Foo.return_value.some_method would make things work correctly, however that's confusing because Foo is going to be properly typed.

MegaPatch(Foo)
  - > Mocked class  <--- this is changed by Mega(Foo.some_method).use_real_logic()
     -> Mocked instance   <--- this is what we probably want, either way, it seems like bad programming to diverge the two intentionally

It seems like if I mock an attribute of a class, it should also mock the instance.

Cannot pass side_effect directly into `MegaPatch.it`

MegaPatch.it(Foo.some_method, side_effect=Exception("Unmocked call!"))

Getting this error:

        elif kwargs:
            # can't set keyword args when we aren't creating the mock
            # XXXX If new is a Mock we could call new.configure_mock(**kwargs)
>           raise TypeError("Can't pass kwargs to a mock we aren't creating")
E           TypeError: Can't pass kwargs to a mock we aren't creating

/usr/local/lib/python3.11/unittest/mock.py:1538: TypeError

`MegaPatch.it(requests.post)` not functioning as expected

Doesn't seem to be getting patched out. The underlying issue seems to be that requests.post isn't getting the reference recorded, so only the copy in requests.api.post is modified.

In other words, mock.patch("requests.post") will work because the name is correct. However, MegaPatch is not supplying that name

Static type hinting not working in PyCharm or IntelliJ

Hi,
can you explain, why the type hinting is only working in VC Code and not in PyCharm or IntelliJ?

When I type the following code, "mock_instance" won't get any type hinting in PyCharm or IntelliJ. It works in VS Code.
Is this a configuration problem?

def test_mega_mocking():
patch = MegaPatch.it(ClassICareAbout)
mock_instance = patch.megainstance
mock_instance.hello.return_value = "some value"

Best regards,
Christian

Clarify that freezegun is still preferred mocking library for time

MegaMock does not replace freezegun so instead of MegaPatch.it(datetime.today) or such, use freezegun

Give individual mocks a memberable name

Currently, if you're trying to differentiate different mocks, you have to look at the numeric IDs, which can be easy to forget. This is to add a random name, such as leaping giraffe, to each instance of a MegaMock object, making it easier to view a mock object once and realize a different mock isn't the same one.

nested assignment not taking

Example:

mock = MegaMock()
mock.option.collectonly = False

assert mock.option.collectonly is False  # fails

Cannot assign return_value through `patched.return_value`

This is currently not allowed. The documentation suggests this possible. The issue seems to simply be a missing setter, which could be populated to do self._new_value.return_value = val

Add support for `new_callable`

May already work, but not tested

Add support for MegaPatch as a function decorator

May already work, untested

De-name mangle `MegaMock.__wrapped`

IMO this creates some harder to follow code with no major benefits. We're forced to access the attribute as _MegaMockMixin__wrapped which requires knowing the implicit name mangling behavior.

Add support for detecting callable return type and returning appropriate mock

Currently, the result to the __call__ method on a mock is always a generic MegaMock. We could inspect the return type and return NonCallableMegaMock if the return type is not callable.

Mypy complains about missing type information

Skipping analyzing "megamock": module is installed, but missing library stubs or py.typed marker

Add ability to enable the real logic for a method

When a class is mocked, the methods are mocked too. In some cases, this is undesirable as there may be methods that don't need to be mocked, and having the actual logic would be beneficial. This issue is to add functionality to restore the original logic for a mocked object.

Have AI explain why "called once with" and similar calls failed

when dealing with complex data structures, it can be time consuming at times to figure out why the equality failed. This would shove the expected and actual to AI and have it tell you.

Subscripted generics error assigning to callable

class SummaryGenerator:
    _user_query: str
    _topics: list[dict]
    _notes_getter: Callable[[str], str]
    _chat: Callable[[str], str]

...

            mock = MegaMock.it(SummaryGenerator)
            mock._notes_getter = lambda notes_file: f"notes from {notes_file}"

gets the error

.venv/lib/python3.11/site-packages/megamock/megamocks.py:416: in _set_attr_annotations_check
    if not isinstance(value, allowed_values):
/usr/local/lib/python3.11/typing.py:1298: in __instancecheck__
    return self.__subclasscheck__(type(obj))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = typing.Callable[[str], str], cls = <class 'function'>

    def __subclasscheck__(self, cls):
>       raise TypeError("Subscripted generics cannot be used with"
                        " class and instance checks")
E       TypeError: Subscripted generics cannot be used with class and instance checks

Return value, etc must be assigned through mock and not mocked value

If you wish to use real logic, for example, you must do this:

        patch = MegaPatch.it(Foo)
        Mega(patch.return_value.some_method).use_real_logic()

and not this:

        MegaPatch.it(Foo)
        Mega(Foo.some_method).use_real_logic()

Just as classes and instances were aligned, this should also work. I tried taking a stab at this and it was a bit more challenging than expected. The underlying problem seems to be the one-way nature of linking classes and likewise the lack of linking to child mocks. Adding that linking in creates complexity. Another issue is that we have this piece of code:

            if self._linked_mock is not None:
                # why is the mock not called as a bound method
                return self._linked_mock(self, *args, **kwargs)

well, it turns out if you try the latter, it does in fact become bound, and you get an issue where too many arguments are passed in. I found this challenging to resolve, as there wasn't an obvious means to determine whether it was bound or not. Advice from the Internet wasn't working.

Track assignments of `MegaMock` attributes

This issue is to add a feature where all assignments to a mock have their stack traces recorded. This could save a developer debug time by making it easy to see the code path where things were assigned.

Investigate moving megamock properties under a single object

Instead of prefixing properties with mega or megamock, they could exist under a child class Mega. This issue is to investigate whether this is better and make the change if it is. It would be better to make this kind of interface change prior to announcing the existence of this project to a wider audience.

Example usage:

mm = MegaMock(...)
mm.Mega.spec is ..
mm.Mega.attr_assignments[...]

Add support for context manager

May already exist, not tested

Backwards type hint

MegaMock.it(SomeClass) type hints the class type and not the class instance. It appears that this use case was anticipated for but was simply backwards and missed.

Expected results:
foo = MegaMock.it(Foo) # foo is an instance of Foo
foo = MegaMock.it(Foo, instance=True) # foo is an instance of Foo
foo = MegaMock.it(Foo, instance=False) # foo is the class Foo

Unpeg the varname version

Seems to be pegged to a lower version. Don't think its needed?

Use get_type_hints for handling future annotations

The most recent change does a string check to handle "from future import annotations". get_type_hints would be a better, more idiomatic solution.

Deep rename not correctly patching

These two expect-fails should work:

    @pytest.mark.xfail
    def test_patch_that_is_renamed_in_non_test_module_1(self) -> None:
        patch = MegaPatch.it(Foo)
        patch.megainstance.some_method.return_value = "it worked"

        assert func_uses_foo() == "it worked"

    @pytest.mark.xfail
    def test_patch_that_is_renamed_in_non_test_module_2(self) -> None:
        from tests.unit.simple_app.does_rename import MyFoo

        patch = MegaPatch.it(MyFoo)
        patch.megainstance.some_method.return_value = "it worked"

        assert func_uses_foo() == "it worked"

Here's the import that's not being patched correctly

from .foo import Foo as MyFoo

foo_instance = MyFoo("something")


def func_that_uses_foo() -> str:
    foo_instance = MyFoo("something else")
    return foo_instance.some_method()

Improve mock names

Currently, mock objects get an automatically created name, such as mock and mock() then mock()(). It would be better if mock was replaced by the actual name of the object being spec'ed, which we have. Instances should be MyClass() while classes would be MyClass

Refactor test structure

MegaMock tests use nested classes
MegaPatch tests do not
The example code isn't named well. What is in Foo vs Bar?

Pure eval warning

/workspaces/python/.venv/lib/python3.11/site-packages/varname/utils.py:445: UsingExecWarning: 'pure_eval' is not installed. Using 'eval' to get the function that calls 'argname'. Try calling it using a variable reference to the function, or passing the function to 'argname' explicitly.
  warnings.warn(

Add spy functionality

A spied object is observed but the real logic is used.

Add support for decorator

May already exist. Not tested

MegaPatching renamed imports results in an error

Currently, if you import:
from foo import Foo as Bar

Then you will get an error if you attempt to MegaPatch Bar, because it is attempting to find Bar in the foo module.

Clean up duplicated simple_app

Add support for MegaPatch as a context manager

May already work, untested

Add pytest plugin

Need to inject the loader early on and also reset patches automatically after every test

GPT may avoid question

When asked a question about why you would use this library instead of built-in functionality, the GPT will spit out the spiel from the reference doc instead of answering the question.

Workaround is to call that out and try again on the follow-up message.

Example:
https://chat.openai.com/share/298092d7-8b01-421f-a97b-4c4184abf509

The likely cause is that the LLM is instructed to follow the directions exactly and is also given a list of valid things, without the right language to indicate that it shouldn't just recite things.

Add support for patching local object instances

Currently, you can patch objects but only if they live in the module scope. You cannot, for example, instantiate an object in a function and then patch that.

Documentation for `megamock_attr_assignments`

This is missing

Bug in reverse references logic is allowing things to function

Not sure how I missed this but the reverse references logic isn't actually working as expected.

https://github.com/JamesHutchison/megamock/blob/main/megamock/import_references.py#L87

components = original_name.split(".", 0)

This is the same as components = [original_name].

If you fix this, the MegaPatch logic breaks in various places. What ends up happening is that you have multiple patches for the same thing, and only one is necessary. I.E. you have foo.Foo.my_method, other_module.Foo.my_method, etc being patched and for whatever reason, adding multiple patches ends up undoing the original patch.

It could be what we want is to do exactly what we are doing - don't create multiple patches for class attributes or anything that doesn't live at the root level of a module.

Add guidance documentation

The guidance documentation will help developers understand why they would use it and what they should do in some situations. It should also help clarify "why use mocking at all when I can just write my code in XXX style"

Add conditional return values

You can currently due this via a side_effect function that takes in arguments and returns a value based on what they are. I wonder if there's an opportunity to leverage the match functionality to make it easy to streamline this sort of thing in an intuitive way.

If its not an obvious improvement over using side_effect, I would opt to omit the functionality.