Login

Extended cacheable callables and properties

Author:
gsakkis
Posted:
June 6, 2010
Language:
Python
Version:
1.2
Score:
0 (after 0 ratings)

There are several snippets that provide a basic caching decorator for functions and methods (e.g. #202, #1130, etc.). The Cacheable class in this snippet extends them by (if you don't see an unordered list below, the Markdown on this site is still broken...):

  • Specifying how cache keys are determined in one of two general, flexible ways.
  • Exposing a few extra methods for (re)setting and deleting the cached value transparently. Given these methods, cache key management should follow DRY: keys are specified once (and only once) at instantiation time, without having to be repeated at later interactions with the cache.
  • Adding a variant for cacheable properties.

Unlike some other snippets, the way cache keys are generated must be explicitly specified by the client; there is no automagic key generation for arbitrary func(*args, **kwds) calls (but can be added if desired). The reason is that all usual serialization approaches (func.__name__/func.__module__, repr, pickle, etc.) are generally fragile, unsafe, inefficient or all of the above. Explicit is better than implicit in this case.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
from copy import copy
from django.core.cache import cache


class Cacheable(object):
    '''A higher level API for caching results of callables.

    This class provides a simple API for caching and retrieving transparently
    the result of a call using Django's cache, as well as (re)setting and
    deleting the cached value.

    In addition to being themselves callable, ``Cacheable`` instances are
    non-data descriptors and work seamlessly as methods.

    Example::

        class Foo(object):
            @Cacheable.decorate(key='bar_{1}')
            def bar(self, x):
                print('Computing bar(%s)..' % x)
                return x*x

        >>> foo = Foo()
        >>> foo.bar(4)
        Computing bar(4)..
        16
        >>> foo.bar(4)              # result is now cached
        16
        >>> foo.bar.delete_cache(4) # delete the cache for this param
        >>> foo.bar(4)              # recompute (and cache) it
        Computing bar(4)..
        16
    '''

    def __init__(self, func, key, timeout=None):
        '''Initializes a cacheable wrapper of ``func``.

        :param func: The callable to be wrapped. It may take any number of
            positional and/or keywords arguments.
        :param key: Specifies how to determine the cache key for a given
            ``func(*args, **kwds)`` call; it can be:

            - A callable: The cache key is computed as ``key(*args, **kwds)``.
              Obviously ``key`` must have the same (or compatible) signature
              with ``func``.
            - A string: A format string in PEP 3101 syntax. The cache key is
              determined as ``key.format(*args, **kwds)``.
        :param timeout: If given, that timeout will be used for the key;
            otherwise the default cache timeout will be used.
        '''
        self._func = func
        self._timeout = timeout
        self._obj = None    # for bound methods' im_self object
        if callable(key):
            self._key = key
        elif isinstance(key, basestring):
            self._key = key.format
        else:
            raise TypeError('%s keys are invalid' % key.__class__.__name__)


    @classmethod
    def decorate(cls, *args, **kwds):
        '''A decorator for wrapping a callable into a :class:`Cacheable` instance.'''
        return lambda func: cls(func, *args, **kwds)

    def __call__(self, *args, **kwds):
        '''Returns the cached result of ``func(*args, **kwds)`` or computes and
        caches it if it's not already cached.

        This method alone covers the majority of use cases, allowing clients to
        use ``Cacheable`` instances as plain callables.

        :returns: The cached or computed result.
        '''
        if self._obj is not None:
            args = (self._obj,) + args
        key = self._key(*args, **kwds)
        value = cache.get(key)
        if value is None:
            value = self._func(*args, **kwds)
            cache.set(key, value, timeout=self._timeout)
        return value

    def set_cache(self, *args, **kwds):
        '''Computes ``func(*args, **kwds)`` and stores it in the cache.

        Unlike :meth:`__call__`, this method sets the cache unconditionally,
        without checking first if a key corresponding to a call with the same
        parameters already exists.

        :returns: The newly computed (and cached) result.
        '''
        if self._obj is not None:
            args = (self._obj,) + args
        key = self._key(*args, **kwds)
        value = self._func(*args, **kwds)
        cache.set(key, value, timeout=self._timeout)
        return value

    def delete_cache(self, *args, **kwds):
        '''Deletes the cached result (if any) corresponding to a call with
        ``args`` and ``kwds``.
        '''
        cache.delete(self.get_cache_key(*args, **kwds))

    def get_cache_key(self, *args, **kwds):
        '''Returns the cache key corresponding to a call with ``args`` and ``kwds``.

        This is mainly for debugging and for interfacing with external services;
        clients of this class normally don't need to deal with cache keys explicitly.
        '''
        if self._obj is not None:
            return self._key(self._obj, *args, **kwds)
        return self._key(*args, **kwds)

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        new = copy(self)
        new._obj = obj
        return new


class CacheableProperty(property, Cacheable):
    '''A :class:`Cacheable` that is also a property.

    Example::

        class Foo(object):
            @CacheableProperty.decorate(key='bar_{0}')
            def bar(self):
                print('Computing bar()..')
                return 42

        >>> foo = Foo()
        >>> foo.bar       # just like a regular property
        Computing bar()..
        42
        >>> foo.bar       # result is now cached
        42
        >>> # Cacheable methods are available through the class
        >>> Foo.bar.delete_cache(foo)
        >>> foo.bar
        Computing bar()..
        42
    '''

    def __init__(self, func, key, timeout=None, fset=None, fdel=None, doc=None):
        Cacheable.__init__(self, func, key, timeout=timeout)
        property.__init__(self, fget=self.__call__, fset=fset, fdel=fdel)
        self.__doc__ = doc or getattr(func, '__doc__', None)

More like this

  1. Template tag - list punctuation for a list of items by shapiromatron 2 months ago
  2. JSONRequestMiddleware adds a .json() method to your HttpRequests by cdcarter 2 months, 1 week ago
  3. Serializer factory with Django Rest Framework by julio 9 months, 1 week ago
  4. Image compression before saving the new model / work with JPG, PNG by Schleidens 9 months, 4 weeks ago
  5. Help text hyperlinks by sa2812 10 months, 3 weeks ago

Comments

Please login first before commenting.