pydanny/cached-property

A decorator for caching properties in classes.

Hi there 👋

• 🔭 I’m currently working for @octoenergy on decarbonizing the planet
• 😄 Pronouns: he/him
• 🐍 I code mostly in Python, but also some NodeJS and Rust
• 🌱 I’m currently learning Rust
• 🌎 Contact me if you want to join me in decarbonizing the planet
• 🥋 I'm currently training in Brazilian Jiu-Jitsu for sport and Tai Chi to help recover from a bike injury
• 📫 How to reach me:
  • mastodon
  • LinkedIn
  • twitter
• ⚡ Fun fact: I met my wife at PyCon US in 2010. She is awesome, her GH account is @audreyfeldroy

cached-property's Issues

Wrong cache timing for long-running functions

I'm not sure if this actually a bug, or intended behaviour, but cached_property_with_ttl doesn't consider the time that the cached function takes to execute, only the time between attribute accesses.
This means that, for long-running functions, the cache misses when it shouldn't (my use case is caching the result of HTTP requests).

Here's code that demonstrates the issue

from cached_property import cached_property_with_ttl
import time

class C(object):
    @cached_property_with_ttl(ttl=1)
    def value(self):
        print("computing")
        time.sleep(2)

c = C()
c.value
c.value

The function is called twice, despite the fact that the second attribute access happens "immediately"

This can be solved with the following patch

            if not ttl_expired:
                return value

+      now = time()
        value = self.func(obj)
        obj_dict[name] = (value, now)
        return value

Contribute implementation to Python

See http://bugs.python.org/issue21145.
All we need to add is the TTL feature that other implementations like pip and werkzeug and Django have.

This package already has some basic testing and it would be nice to start from there.

async/await support doesn't work with Twisted

I'm not sure I expect cached-property to do anything about this, but it appears to be an undocumented limitation.

My (limited) understanding is that this is part cached_property limitation and part a limitation of Twisted's Deferred (it does not support being awaited from an asyncio coroutine which is what cached_property is trying to do).

Hopefully one of the active Twisted developers (or someone else) can chime in with how cached_property might handle this.

Versions:

• Python 3.8.2
• Twisted==20.3.0
• cached-property==1.5.1

Reproduction:

from cached_property import cached_property

class Monopoly(object):

    def __init__(self):
        self.boardwalk_price = 500

    @cached_property
    async def boardwalk(self):
        self.boardwalk_price += 50
        return self.boardwalk_price

async def print_boardwalk():
    monopoly = Monopoly()
    print(await monopoly.boardwalk)
    print(await monopoly.boardwalk)
    print(await monopoly.boardwalk)

print("With asyncio")
print("------------")
import asyncio
asyncio.get_event_loop().run_until_complete(print_boardwalk())
print("------------")

print("With twisted")
print("------------")
from twisted.internet.defer import ensureDeferred
from twisted.internet.task import react
react(lambda _: ensureDeferred(print_boardwalk()))
print("------------")

Results:

With asyncio
------------
550
550
550
------------
With twisted
------------
main function encountered error
Traceback (most recent call last):
  File ".ve/lib/python3.8/site-packages/twisted/internet/task.py", line 909, in react
    finished = main(_reactor, *argv)
  File "<stdin>", line 29, in <lambda>
    
  File ".ve/lib/python3.8/site-packages/twisted/internet/defer.py", line 911, in ensureDeferred
    return _cancellableInlineCallbacks(coro)
  File ".ve/lib/python3.8/site-packages/twisted/internet/defer.py", line 1529, in _cancellableInlineCallbacks
    _inlineCallbacks(None, g, status)
--- <exception caught here> ---
  File ".ve/lib/python3.8/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "<stdin>", line 15, in print_boardwalk
    
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/coroutines.py", line 127, in coro
    res = yield from res
builtins.RuntimeError: await wasn't used with future

Add timed cached_property

Must work like this:

class Monopoly(object):

    def __init__(self):
        self.boardwalk_price = 500
        self.parkplace_price = 450

    # No time specified so cache is indefinite
    @timed_cached_property
    def boardwalk(self):
        self.boardwalk_price += 50
        return self.boardwalk_price

    # Time of 300 specified, making cache last for 5 minutes before invalidation
    @timed_cached_property(ttl=300)
    def parkplace(self):
        self.timed_cached_property += 50
        return self.timed_cached_property

Notes:

• This might supplant the current cached_property decorator.
• The example at https://wiki.python.org/moin/PythonDecoratorLibrary#Cached_Properties breaks if you don't pass a TTL (time-to-leave) argument.

Locking strategy is wrong

Now that the locking cached_property was adapted into Python standard library, I'd note here too that the threaded_cached_property locking strategy is wrong for many use cases, and I've written an issue in bugs.python.org.

The reason is that a descriptor is not instantiated for every object instance - they indeed exist only once for each class. Thus if there are multiple instances of the class, then all these would share the same one lock, even though the caches are independent. And if there is only one instance of a class then it is rather Pythonic to not write a class at all...

Convert all RST files to GFM

Justifications

• Writing in Markdown is easier/faster than RST, facilitating improved documentation
• Markdown has a lower barrier of entry
• Advancements in Twine it's checkable before deploying a new release to PyPI. I'm really tired of having x.x.x releases simply because the RST doesn't compile.

Tasks

• Use Pandoc for conversion on all files. Example: pandoc README.rst -t gfm -o README.md
• Remove sourceCode label from generated markdown files
• Remove .rst files
• Change rst file references in setup.py and in other places.

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 561: ordinal not in range(128)

Building with python3.4 on Debian:

I: pybuild base:170: python3.4 setup.py clean
Traceback (most recent call last):
File "setup.py", line 15, in
history = open('HISTORY.rst').read().replace('.. :changelog:', '')
File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
i : 'ascii' codec can't decode byte 0xe2 in position 561: ordinal not in range(128)

cached_property is settable while property isn't

A property without __set__ raises an AttributeError.
Cached property does not raise an AttributeError when being set.
Is that an implementation defect or intentional?
Shouldn't it behave like a read-only property behaves in terms of writeability?

Support Doctests

Running Python 3.7.3 on Ubuntu 18.04 x64

For some reason, when using this decorator outside of the module it is declared in, it causes doctests to disappear at runtime.

For some reason, modifying the cached_property class to inherit from property fixes this. I got this idea from werkzeug.

I'm not going to pretend to understand why either is happening (your class preserves __doc__ like it should, but some magic in property makes it accessible where it isn't otherwise).

Test 2 lines of code

See

cached-property/cached_property.py

Lines 24 to 25 in dc22684

	if obj is None:
	return self

Tests don't cover this case yet.

Does not support case where __doc__ is missing

The code uses self.__doc__ = getattr(func, '__doc__') in a few places, but that’s strictly equivalent to self.__doc__ = func.__doc__. If the goal is to avoid AttributeError, a default is needed: getattr(func, '__doc__', None). Only dict.get has a default default :)

Invalidating the entire cache.

I want to invalidate the entire cache with one command instead of needing to individually invalidating them. So instead of:

del monopoly.__dict__['boardwalk']
del monopoly.__dict__['parkplace']

it would be like:

del monopoly.__dict__['all']

I tried to clear the dictionary in this way:

monopoly.__dict__ = dict.fromkeys(monopoly.__dict__)

However, this also deletes the non-property variables ( e.g. monopoly.boardwalk_price) because they are all stored in the same place.

What's the best way to do this?

Can cached_property be a subclass of property?

There are certain pieces of code in some libraries, linters and static analyzers that check things like

if isinstance(attr, property):
    do_something()

This is especially important for use in metaclasses.

Obviously this fails with cached property; would it be possible to derive it from property so isinstance/issubclass checks work as expected?

How does this implementation differ from the functools' cached_property?

How does this implementation differ from the functools' cached_property? When should I use which one?

Proposal: inheritance structure and annotate/validate behaviour

while trying to implement a cached property variant that implements a annotate/validate behavior

i noticed that 2 implementation details could be shared in mixins

• all thread support is implemented via adding a lock property and using it in __set__
• ttl is using the current time as "annotation" and uses timedelta > ttl as validation

this would allow for a interesting hierarchy ill propose at the end

the annotate/validate behavior can be considered as a generalization of the ttl model
where ttl is just the annotation/validation based on time/ttl,
while annotate/validate could also do more extended checks like validating database connections
or doing inexpensive cache/db queries for values that where very expensive to query for

Hierachy

• CachedProperty implementing the base behaviour
• _ThreadSafeMixin implementign only the thread lock
• ThreadSafeCachedPropery inheriting fromCachedProperty,_ThreadSafeMixin`
• _AnnotatedCachedproperty implementing annotation/validation behaviour
  • annotation must result in a single object which is stored alongside the actual object
  • validation will receive only the annotation object, and must return a truth value
  • annotation may return the actual object in order to allow validation to use it
• CachedPropertyWithTTL inheriting from _AnnotatedCachedproperty implementing ttl annotation/validation
• AnnotatedCachedProperty. inheriting from _AnnotatedCachedproperty implementing exposure of the annotate/validate api
• the thread-safe variants would just mix in the _ThreadSafeMixin again

since this is a larger structural change, i wanted to discuss before taking a stake at completely implementing it

on a sidenote - i already tried the thread safety unification,
but sharing the code for pessimistic retrieval from __dict__ caused the threading test for the normal cached property to fail

i suspect the gil is involved and i stepped back to discuss the idea at that point since debugging/verifying the gil involvement would eat way too much time right now

add a cached property with idempotent invalidation

the fast cached property has one major drawback,
cache validation is not idempotent

to enable ease of use i propose adding a idempotent variant of the cached property,
which handles idempotent deletion

example code i did only write, not execute:

from cached_property import
class Example(object):
  @cached_property
  def foo(self):
    pass

del Example().foo

should not fail if one requests a idempotent cached property

a potential option to help with the performance hit would be using a cython spec to translate the cached property to c code for speed

Rename the package?

Would it make sense to rename the package to cached_property or cachedproperty?

Isn't there a way to empty cached results on according save operations?

Hey,

I am wondering, why it wouldn't be smarter to let the decorator (somehow) clear related caches whenever a related model changes. Use Case:

I have a big model with many related models and I need to filter them for a ListView Page. To display those values I would use cached_properties for data from other models like images. Since the related instances have not changed, we could use the cached_property. Now if we upload a new image, a signal could clear the cached_property used in the other model. The same goes for update or delete operations. That way the system would always be up to date, but max out the potential of caching where ever it makes sense.

Is that clear? I am not an expert like you, but since I am currently learning about all this, I was curious.

Caching GeneratorType results

Please consider the example below:

class Foo(object):
    def __init__(self, items):
        self.items = items

    @cached_property
    def squares(self):
        for item in self.items:
            yield item * item


foo = Foo([1, 2, 3])
print list(foo.squares)
print list(foo.squares)

It will produce the following output:

[1, 4, 9]
[]

To fix this we have modified cached_property decorator:

class cached_property(object):

    def __init__(self, func):
        self.func = func

    def __get__(self, obj, cls):
        value = self.func(obj)
        if isinstance(value, GeneratorType):
            value = list(value)
        obj.__dict__[self.func.__name__] = value
        return value

But now we loose the laziness of the method and also change its return type. Do you have in mind any elegant way to implement cached decorator that will cache items yielded so far, so when the property will be called second time it will first yield cached items and then continue with items yielded by the original (cached) method?

Access all caches

Please add the possiblity to get the caches defined at a class.
What I actually need is a way to invalidate all caches at once and I don't want to name them all because I may forget to add it to the list if I add a new cached property.
But I think a way to access the class members names that are cached would be of more use for other use cases.

cached_property works wrong with inheritance

class Parent(object):
  @cached_property
  def test(self):
    return 1


class Child(Parent):
  @cached_property
  def test(self):
    result = super(Child, self).test
    result = result / 0
    return result


child_instance = Child()
print(child_instance.test)
# gives a traceback

print(child_instance.test)
# prints "1"

Here is a quick-fix solution:

diff --git a/cached_property.py b/cached_property.py
index 67fa01a..6122eab 100644
--- a/cached_property.py
+++ b/cached_property.py
@@ -32,7 +32,13 @@ class cached_property(object):
         if asyncio and asyncio.iscoroutinefunction(self.func):
             return self._wrap_in_coroutine(obj)
 
-        value = obj.__dict__[self.func.__name__] = self.func(obj)
+        try:
+            value = self.func(obj)
+        except Exception:
+            obj.__dict__.pop(self.func.__name__, None)
+            raise
+
+        obj.__dict__[self.func.__name__] = value
         return value
 
     def _wrap_in_coroutine(self, obj):
@@ -69,7 +75,12 @@ class threaded_cached_property(object):
 
             except KeyError:
                 # if not, do the calculation and release the lock
-                return obj_dict.setdefault(name, self.func(obj))
+                try:
+                    value = self.func(obj)
+                except Exception:
+                    obj_dict.pop(name, None)
+                    raise
+                return obj_dict.setdefault(name, value)
 
 
 class cached_property_with_ttl(object):
@@ -108,7 +119,11 @@ class cached_property_with_ttl(object):
             if not ttl_expired:
                 return value
 
-        value = self.func(obj)
+        try:
+            value = self.func(obj)
+        except Exception:
+            obj_dict.pop(name, None)
+            raise
         obj_dict[name] = (value, now)
         return value
 
diff --git a/tests/test_cached_property.py b/tests/test_cached_property.py
index 5082416..e3499b2 100644
--- a/tests/test_cached_property.py
+++ b/tests/test_cached_property.py
@@ -88,6 +88,23 @@ class TestCachedProperty(unittest.TestCase):
         # rather than through an instance.
         self.assertTrue(isinstance(Check.add_cached, self.cached_property_factory))
 
+    def test_error_on_inheritance(self):
+        class CheckWithInheritance(CheckFactory(self.cached_property_factory)):
+            @self.cached_property_factory
+            def add_cached(inst):
+                return super(CheckWithInheritance, inst).add_cached / 0
+
+        check = CheckWithInheritance()
+        stub_to_check = object()
+
+        result = stub_to_check
+        with self.assertRaises(ZeroDivisionError):
+            result = check.add_cached
+        with self.assertRaises(ZeroDivisionError):
+            result = check.add_cached
+
+        self.assertEqual(result, stub_to_check)
+
     def test_reset_cached_property(self):
         Check = CheckFactory(self.cached_property_factory)
         check = Check()

versions >= 1.0.0: can't clear cached properties

Hi

I've using cached_property since 0.1.5 - I see it's now declared 1.0.0 - I'm using it with python 2.7.8

Well I had some regressions since I started using 1.0.0 - I can't seem to invalidate the cache - either through the way listed on pypy:

del Instance[attribute]

or the old tried and true way which worked when the data was not cached yet:

try:
     delattr(instance, pname)
except AttributeError:
     pass

As of right now, the only way I could get it to invalidate the cached entries, whether or not they are populated is this:

     try:
        del instance._cache[pname]
    except KeyError as e:
         pass

To be clear I haven't been able to clear the cache at all in any other way than the above direct mangling with _cache.

Is this a regression on your side? Is the documentation up to date with how you expect to clear the cache?

I think with something this big, if valid, needs to have the pypy package updated asap!

Update documentation?

cached generator?

I tried to wrap a generator property with this and was a bit confused, got *** TypeError: 'generator' object is not subscriptable back, I presume this is because the cached_property doesn't have a branch to do the necessary to capture and forward a generator and spawn a new generator on subsequent requests. [[Really as long as the output is iterable, I shoudln't really care if a list is yielded from the cache or a generator.]]

Would supporting generators be soemthing in-scope? I really like to use generators for slow-to-walk things (like say a paginated http rest api, where) caching the generated list is desirable.

I've not done my homework on this library, you have my apologies if this is documented and I just missed it in my rush. 🙇‍♂️
In my rush I am probably going to wrap generator methods, flattening them to lists as a cached property variant - though to me big extra-value in this decorator is the lack-of-need to designate cached and non-cached variants of the same property. Idk Thank you for a super helpful and super fast cache layer!

No compatible with `__slots__`

Hi. Very interresting project.
I was about to develop the same thing, but made a research first and found this

The problem is that it doesn't support the use of __slots__

Demonstration:

>>> from cached_property import cached_property
>>> class C:
...     __slots__ = 'x',
...     @cached_property
...     def y(self):
...         return 'y'
... 
>>> c = C()
>>> c.y
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.6/site-packages/cached_property.py", line 26, in __get__
    value = obj.__dict__[self.func.__name__] = self.func(obj)
AttributeError: 'C' object has no attribute '__dict__'

I have a little wrapper for this named slot_wrapper somewhere.

Here is a simplified version:

def slot_wrapper(attrname, slot, cls):

    class InnerDescriptor:

        def __get__(self, obj, cls):
            return slot.__get__(obj, cls)

        def __set__(self, obj, val):
            slot.__set__(obj, val)

return InnerDescriptor(attrname)

It takes the descriptor under the slot and replace it (__slots__ underly special descriptors, but they remains descriptors after all).
It needs adjustments with your code in order to work... I use it with metclass so it got instaciate after the creation of a class wich have __slots__.

My demand is between a bug report, a feature request and an enhacement proposal!

threaded_cached_property can raise confusing exception

When the function decorated by threaded_cached_property raises an exception, rather than the exception from the user code being front-and-center, users are presented first with the cached property KeyError, only to see their real error further down the line.

Take for example the following code and resulting traceback.

class Monopoly(object):

    def __init__(self):
        self.boardwalk_price = 500

    @threaded_cached_property
    def boardwalk(self):
        self.boardwalk_price += 1/0
        return self.boardwalk_price

m = Monopoly()
m.boardwalk

Traceback (most recent call last):
  File "/Users/spencer.young/repos/cached-property/cached_property.py", line 70, in __get__
    return obj_dict[name]
KeyError: 'boardwalk'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "t.py", line 15, in <module>
    m.boardwalk
  File "/Users/spencer.young/repos/cached-property/cached_property.py", line 74, in __get__
    return obj_dict.setdefault(name, self.func(obj))
  File "t.py", line 11, in boardwalk
    self.boardwalk_price += 1/0
ZeroDivisionError: division by zero

The traceback presents first, a KeyError which could be confusing to the user. Instead, it would be nice if cached-property would avoid this in such a way that only the relevant ZeroDivisionError would surface.

Some test failing on python2 (Debian)

I had to disable the tests on python 2.7 (building foor Debian) because of syntax errors in some tests. Could those tests be made aware of used Python version?

Thanks,
Mathias

The relevant bits:

dh_auto_test -O--buildsystem=pybuild
I: pybuild base:217: cd /<>/.pybuild/cpython2_2.7_cached-property/build; python2.7 -m unittest discover -v
tests.test_coroutine_cached_property (unittest.loader.ModuleImportFailure) ... ERROR
tests.test_async_cached_property (unittest.loader.ModuleImportFailure) ... ERROR
test_cached_property (tests.test_cached_property.TestCachedProperty) ... ok
test_none_cached_property (tests.test_cached_property.TestCachedProperty) ... ok
test_reset_cached_property (tests.test_cached_property.TestCachedProperty) ... ok
test_set_cached_property (tests.test_cached_property.TestCachedProperty) ... ok
test_threads (tests.test_cached_property.TestCachedProperty) ... ok
test_cached_property (tests.test_cached_property.TestCachedPropertyWithTTL) ... ok
test_none_cached_property (tests.test_cached_property.TestCachedPropertyWithTTL) ... ok
test_reset_cached_property (tests.test_cached_property.TestCachedPropertyWithTTL) ... ok
test_set_cached_property (tests.test_cached_property.TestCachedPropertyWithTTL) ... ok
test_threads (tests.test_cached_property.TestCachedPropertyWithTTL) ... ok
test_threads_ttl_expiry (tests.test_cached_property.TestCachedPropertyWithTTL) ... ok
test_ttl_expiry (tests.test_cached_property.TestCachedPropertyWithTTL) ... ok
test_cached_property (tests.test_cached_property.TestThreadedCachedProperty) ... ok
test_none_cached_property (tests.test_cached_property.TestThreadedCachedProperty) ... ok
test_reset_cached_property (tests.test_cached_property.TestThreadedCachedProperty) ... ok
test_set_cached_property (tests.test_cached_property.TestThreadedCachedProperty) ... ok
test_threads (tests.test_cached_property.TestThreadedCachedProperty) ... ok
test_cached_property (tests.test_cached_property.TestThreadedCachedPropertyWithTTL) ... ok
test_none_cached_property (tests.test_cached_property.TestThreadedCachedPropertyWithTTL) ... ok
test_reset_cached_property (tests.test_cached_property.TestThreadedCachedPropertyWithTTL) ... ok
test_set_cached_property (tests.test_cached_property.TestThreadedCachedPropertyWithTTL) ... ok
test_threads (tests.test_cached_property.TestThreadedCachedPropertyWithTTL) ... ok
test_threads_ttl_expiry (tests.test_cached_property.TestThreadedCachedPropertyWithTTL) ... ok
test_ttl_expiry (tests.test_cached_property.TestThreadedCachedPropertyWithTTL) ... ok

======================================================================
ERROR: tests.test_coroutine_cached_property (unittest.loader.ModuleImportFailure)

ImportError: Failed to import test module: tests.test_coroutine_cached_property
Traceback (most recent call last):
File "/usr/lib/python2.7/unittest/loader.py", line 254, in _find_tests
module = self._get_module_from_name(name)
File "/usr/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name
import(name)
File "tests/test_coroutine_cached_property.py", line 60
value = yield from check.add_control()
^
SyntaxError: invalid syntax

======================================================================
ERROR: tests.test_async_cached_property (unittest.loader.ModuleImportFailure)

ImportError: Failed to import test module: tests.test_async_cached_property
Traceback (most recent call last):
File "/usr/lib/python2.7/unittest/loader.py", line 254, in _find_tests
module = self._get_module_from_name(name)
File "/usr/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name
import(name)
File "tests/test_async_cached_property.py", line 34
async def add_control(self):
^
SyntaxError: invalid syntax

Support mypy / static typing

Right now static typing isn't really supported, and we are basically forced to turn it off via

from cached_property import cached_property  # type: ignore

What seems to work pretty well is to pretend to mypy that cached_property is the same as property, that is, write a cached_property.pyi stub file with just

cached_property = property

and then manually point mypy at it. Ideally that would be part of the out-of-the-box experience. For that we would need to either push support into typeshed or:

• Rewrite cached_property.py into a package (code goes into cached_property/__init__.py or import it from there)
• Add the cached_property/__init__.pyi stub
• Add the cached_property/py.typed marker as in https://www.python.org/dev/peps/pep-0561/

With #26 we wouldn't need the stub pyi file but the py.typed marker and refactoring into a package would still be required.

Mention functools' cached_property in the readme

This will help remove ambiguity over what to use.

Problems with threads

Hi, I wanted to extend your project so I created a Dockerfile to run tests in container. I did some modifications and when I ran tests they failed from time to time. So I reverted my changes and ran it again against current HEAD.

See results here http://pastebin.centos.org/34206/

Simply put, tests for threading randomly fail - in this case in 6th run. I am not very familiar with multi threaded applications, but will be happy to assist with debugging.

cached_property doesn't cache

I can't understand why it's not working

from cached_property import cached_property_with_ttl, cached_property

class Item(models.Model):

@cached_property
def test(self):
     return timezone.now()

{{ item.test }}

Every time I see different time.

cached_property 1.4.3
django 1.11.15
python 2.7.12

Feature Request: cached_classproperty

I would like to have a cached_classproperty.

What do you think?

Misbehavior with async def methods

When decorating an async method the coroutine is cached instead of the value returned by the coroutine. This is certainly not the programmer's intent, and moreover pointless since coroutines can only be awaited once:

$ ipython3
Python 3.6.4 (default, Feb  1 2018, 11:06:09) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import asyncio
   ...: from cached_property import cached_property
   ...:     
   ...: class Foo(object):
   ...: 
   ...:     @cached_property
   ...:     async def bar(self):
   ...:         return 'value'
   ...: 
   ...:     async def baz(self):
   ...:         await self.bar
   ...:         await self.bar
   ...: 
   ...: 
   ...: import asyncio
   ...: asyncio.get_event_loop().run_until_complete(Foo().baz())          
   ...: 
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-1-a740a5aac079> in <module>()
     14 
     15 import asyncio
---> 16 asyncio.get_event_loop().run_until_complete(Foo().baz())

/usr/lib64/python3.6/asyncio/base_events.py in run_until_complete(self, future)
    465             raise RuntimeError('Event loop stopped before Future completed.')
    466 
--> 467         return future.result()
    468 
    469     def stop(self):

<ipython-input-1-a740a5aac079> in baz(self)
     10     async def baz(self):
     11         await self.bar
---> 12         await self.bar
     13 
     14 

RuntimeError: cannot reuse already awaited coroutine

When caching an async method, the value should be stored and repeated access should always return a new coroutine (which returns the cached value).

This may be related to #7, though its not quite clear to me what that ticket is about

Test failure in test_threads_ttl_expiry due to updated freezegun

I'm getting the following test failure on Python 3.7:

============================= test session starts ==============================
platform linux -- Python 3.7.1, pytest-3.9.3, py-1.7.0, pluggy-0.8.0
rootdir: /build/source, inifile:
collected 30 items

tests/test_async_cached_property.py ...                                  [ 10%]
tests/test_cached_property.py ...............F........                   [ 90%]
tests/test_coroutine_cached_property.py ...                              [100%]

=================================== FAILURES ===================================
______________ TestCachedPropertyWithTTL.test_threads_ttl_expiry _______________

self = <tests.test_cached_property.TestCachedPropertyWithTTL testMethod=test_threads_ttl_expiry>

    def test_threads_ttl_expiry(self):
        Check = CheckFactory(self.cached_property_factory(ttl=100000), threadsafe=True)
        check = Check()
        num_threads = 5

        # Same as in test_threads
        check.run_threads(num_threads)
        self.assert_cached(check, num_threads)
        self.assert_cached(check, num_threads)

        # The cache expires in the future
        with freeze_time("9999-01-01"):
            check.run_threads(num_threads)
>           self.assert_cached(check, 2 * num_threads)

tests/test_cached_property.py:207:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/test_cached_property.py:69: in assert_cached
    self.assertEqual(check.add_cached, expected)
E   AssertionError: 6 != 10
===================== 1 failed, 29 passed in 8.39 seconds ======================

Do you have any idea why this could happen?

Add asyncio_cached_property

Only for users of Python 3.4 or higher, or those with Python 3.3 and asyncio installed.

Better multithreading support

Example at https://gist.github.com/Tinche/e092adeb8171fd0843dd will print out "Cached method called 10 times" where ideally it would only be called once.

I'd suggest documenting this fact or doing something about it :)

Deprecation warning due to usage of asyncio.coroutine.

tests/test_coroutine_cached_property.py:56
  /root/checked_repos/cached-property/tests/test_coroutine_cached_property.py:56: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
    def assert_control(self, check, expected):

tests/test_coroutine_cached_property.py:65
  /root/checked_repos/cached-property/tests/test_coroutine_cached_property.py:65: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
    def assert_cached(self, check, expected):

tests/test_coroutine_cached_property.py:76
  /root/checked_repos/cached-property/tests/test_coroutine_cached_property.py:76: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
    def test_cached_property(self):

tests/test_coroutine_cached_property.py:98
  /root/checked_repos/cached-property/tests/test_coroutine_cached_property.py:98: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
    def test_reset_cached_property(self):

tests/test_coroutine_cached_property.py:115
  /root/checked_repos/cached-property/tests/test_coroutine_cached_property.py:115: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
    def test_none_cached_property(self):

tests/test_async_cached_property.py::TestCachedProperty::test_cached_property
tests/test_async_cached_property.py::TestCachedProperty::test_none_cached_property
tests/test_async_cached_property.py::TestCachedProperty::test_reset_cached_property
  /root/checked_repos/cached-property/tests/test_async_cached_property.py:12: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
    coro = asyncio.coroutine(f)

tests/test_async_cached_property.py: 8 warnings
tests/test_coroutine_cached_property.py: 8 warnings
  /root/checked_repos/cached-property/cached_property.py:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
    def wrapper():

tests/test_coroutine_cached_property.py::TestCachedProperty::test_cached_property
tests/test_coroutine_cached_property.py::TestCachedProperty::test_none_cached_property
tests/test_coroutine_cached_property.py::TestCachedProperty::test_reset_cached_property
  /root/checked_repos/cached-property/tests/test_coroutine_cached_property.py:17: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
    coro = asyncio.coroutine(f)

tests/test_coroutine_cached_property.py::TestCachedProperty::test_cached_property
tests/test_coroutine_cached_property.py::TestCachedProperty::test_reset_cached_property
  /root/checked_repos/cached-property/tests/test_coroutine_cached_property.py:37: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
    def add_control(self):

tests/test_coroutine_cached_property.py::TestCachedProperty::test_cached_property
tests/test_coroutine_cached_property.py::TestCachedProperty::test_reset_cached_property
  /root/checked_repos/cached-property/tests/test_coroutine_cached_property.py:43: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
    def add_cached(self):

tests/test_coroutine_cached_property.py::TestCachedProperty::test_none_cached_property
  /root/checked_repos/cached-property/tests/test_coroutine_cached_property.py:122: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
    def add_cached(self):

cached_property decorator doesn't work with mangled method names

See also https://code.djangoproject.com/ticket/29478#ticket

import itertools

from cached_property import cached_property

count = itertools.count()
count2 = itertools.count()
count3 = itertools.count()


class Foo:
    @cached_property
    def __foo(self):
        return next(count)

    @cached_property
    def foo2(self):
        return next(count2)

    @property
    def foo3(self):
        return next(count3)

    def run(self):
        print('foo', self.__foo)
        print('foo', self.__foo)
        print('foo', self.__foo)
        print('foo', self.__foo)
        print('foo2', self.foo2)
        print('foo2', self.foo2)
        print('foo2', self.foo2)
        print('foo2', self.foo2)
        print('foo2', self.foo2)
        print('foo3', self.foo3)
        print('foo3', self.foo3)
        print('foo3', self.foo3)
        print('foo3', self.foo3)
        print('foo3', self.foo3)


Foo().run()


"""
python cached_property_test.py 
foo 0
foo 1
foo 2
foo 3
foo2 0
foo2 0
foo2 0
foo2 0
foo2 0
foo3 0
foo3 1
foo3 2
foo3 3
foo3 4
"""

Cache invalidation

Hello!

Is it planned to implement cache invalidation? I mean, if property's value depends on something, which can be modified from the outside, is it possible to tell to the property, that its value is not valid any more.

For, example:

class Foo(object):

    name = None

    def set_name(self, value):
        self.name = value
        self.greeting.invalidate_cache()

    @cached_property
    def greeting(self):
        name = self.name or 'anonymous'
        return 'Hello, {0}'.format(name)

`x = cached_property(y)` doesn't work properly

from cached_property import cached_property


class A:
    def __init__(self, x):
        self._x = x

    def get_x_len(self):
        print('...')
        return len(self._x)

    x_len = cached_property(get_x_len)


a = A([1, 2, 3])
print(a.get_x_len())
print(a.x_len)
print(a.get_x_len())

This examples fails with TypeError: 'int' object is not callable. The problem is, cached_property replaces get_x_len with 3 while it actually semantically should replacing x_len with 3.

[bug] threads lock wont work

for example:

class A:
    def __init__(self):
        self.a = 1
    @threaded_cached_property
    def b(self):
        return 2

if one thread writing/deleting a, and another thread getting b, this may corrupt __dict__. (some python don't have GIL)

Make the threaded_cached_property the default cached_property decorator?

Why not make everything thread safe?

is:issue is:open propagate cached property behavior to child class with abstract method

This is a feature request

Code:

import numpy as np
import abc
from cached_property import cached_property


class A(abc.ABC):

    @abc.abstractmethod
    @cached_property
    def v1(self):
        ...


class B:

    @cached_property  # things dont work if we dont add this
    def v1(self):
        return np.random.randint(133)


np.random.seed(123)
a = B()
print(a.v1)
print(a.v1)
print(a.v1)

Output:

109
109
109

Can we have this code run where there is no need to have @cached_property decorator for child class B. Or do you think this is not useful to implement as it violates some coding principles as the property v1() will now appear as a method in child class B to all IDEs

Support caching generators

Many times I'll have a function such as

def x():
    result = some_long_function_that_eventually_calculates_a_list()
    yield from (result.name for result in result)
    yield from (result.path for result in result)

I can't cache this function using regular cached_property because the generator will be exhausted.
I want to calculate it once, then whenever it is being called again return the items I retrieved in the order I retrieved them.

Python builtin help() does not show docstring of cached_property

from cached_property import cached_property

class Monopoly(object):

    def __init__(self):
        self.boardwalk_price = 500

    @cached_property
    def boardwalk(self):
        """my docstring
        Again, this is a silly example. Don't worry about it, this is
         just an example for clarity.
       """
        self.boardwalk_price += 50
        return self.boardwalk_price

help(Monopoly)

Above code shows:

Help on class Monopoly in module __main__:

class Monopoly(__builtin__.object)
 |  Methods defined here:
 |  
 |  __init__(self)
 |  
 |  boardwalk = <cached_property.cached_property object>

Same code without @cached_property decorator, help() shows actual content of docstring:

Help on class Monopoly in module __main__:

class Monopoly(__builtin__.object)
 |  Methods defined here:
 |  
 |  __init__(self)
 |  
 |  boardwalk(self)
 |      my docstring
 |      Again, this is a silly example. Don't worry about it, this is
 |       just an example for clarity.

This is python 2.7.10.

Is there any possibility to fix this?

Following stackoverflow questions seem to consider this:
http://stackoverflow.com/questions/17160117/python-using-decorator-py-to-preserve-a-method-docstring
http://stackoverflow.com/questions/1782843/python-decorator-problem-with-docstrings

Credit where it's due

The README currently says that the Django implementation is used, but I believe the implementation was originally the work of Marcel Hellkamp (@defnull) in bottle.py.

While it will always be tricky to pin down the "true source" of a piece of code, the git histories of bottle and django may be of some help. The cached_property decorator appeared in bottle a month before it appeared in django.

in bottle, Aug 2011: bottlepy/bottle@fa7733e

in django, Sept 2011: django/django@5009e45

Also, your docstring and variables names are exactly identical to Marcel's.

And one last friendly reminder: the licenses of both bottle and django (MIT and BSD, respectively) require a copyright notice to be posted.

Add a way to check if a property has already been cached

Sometimes it's useful to be able to check if a property has been cached already. Currently I'm using something like:

    @cached_property
    def cachable(self): pass

    def a_function(self):
        if 'cachable' in self.__dict__:
            # Already cached.
        else:
            # Not cached yet.

(Haven't tested this, but I think it'ill work) Can we think of a better API?

cached_property_with_ttl decorator doesn't work with async methods

Can we also support coroutines for @cached_property_with_ttl decorator?

Cached properties error with named tuples.

Hello there. Currently there's no way to use cached properties with namedTuples (or other immutable classes implemented the same way)

Trace:

AttributeError                            Traceback (most recent call last)
<ipython-input-3-7deeb8fbc12c> in <module>
----> 1 r.undirected

~/.local/share/virtualenvs/myvenv/lib/python3.7/site-packages/cached_property.py in __get__(self, obj, cls)
     33             return self._wrap_in_coroutine(obj)
     34 
---> 35         value = obj.__dict__[self.func.__name__] = self.func(obj)
     36         return value
     37 

AttributeError: 'MyClass' object has no attribute '__dict__'

Reproduced in https://github.com/Qu4tro/cachedproperty_immutable_classes

There was a change in python 3.5.1 and NamedTuple no longer has the __dict__ attribute. More information here: https://stackoverflow.com/a/34166617

I totally understand why it's failing, so my question is:

• Are you interested in supporting this use-case?
• If yes - Any implementation ideas flying around?
• If not - Maybe a small note on the README would be useful.

Lmk what you think.

Invalidating cache from within object does not work

It seems to me that resetting the cache from within my object (which uses the cached_property) does not work. I am getting the error KeyError: 'trans1':

class MyClass:

    def __init__(self, parameter):
        self.parameter = parameter

    def fit(self, data):
        print('fit')
        self.intermediate_data_ = data + self.parameter + 2
        del self.__dict__['trans1']
        del self.__dict__['trans2']
        return self
    
    @cached_property
    def trans1(self):
        print('trans 1 - calc')
        return self.intermediate_data_ / self.parameter / 2

    @cached_property
    def trans2(self):
        print('trans 2 - calc')
        return self.intermediate_data_ / self.parameter / 5