Login

Scoped Cache Compatible with Django Caching Helpers

Author:
axiak
Posted:
February 28, 2008
Language:
Python
Version:
.96
Score:
2 (after 2 ratings)

Have you ever felt the need to run multiple Django projects on the same memcached server? How about other cache backends? To scope the cache keys, you simply need to prefix. However, since a lot of Django's internals rely on django.core.cache.cache, you cannot easily replace it everywhere.

This will automatically upgrade the django.core.cache.cache object if settings.CACHE_PREFIX is set to a string and the Middleware contains ScopeCacheMiddleware.

A thread discussing the merging of this functionality into Django is available on the dev mailing list.

However, (as of now) nowhere in the thread does anyone mention the reason why this sort of treatment is needed: Many of Django's internal caching helpers use django.core.cache.cache, and will then conflict if multiple sites run on the same cache stores.

Example Usage:

>>> from django.conf import settings
>>> from django.core.cache import cache
>>> from scoped_caching import prefix_cache_object
>>> settings.CACHE_PREFIX
'FOO_'
# Do this once a process (e.g. on import or Middleware)
>>> prefix_cache_object(settings.CACHE_PREFIX, cache)
>>> cache.set("pi", 3.14159)
>>> cache.get("pi")
3.14159
>>> cache.get("pi", use_global_namespace=True)
>>> cache.get("FOO_pi", use_global_namespace=True)
3.14159
>>> cache.set("FOO_e", 2.71828, use_global_namespace=True)
>>> cache.get("e")
2.71828

To Install: Simply add ScopeCacheMiddleware as a middleware and define settings.CACHE_PREFIX and enjoy!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
from django.conf import settings
from django.core.cache import cache

class ScopeCacheMiddleware(object):
    """ Middleware which adds scoping to all of the
    cache methods.
    If settings.CACHE_PREFIX is defined, replace
    all django.core.cache.cache methods with new
    methods that automatically prefix the keys
    appropriately.

    Warning: May not work with psyco!
    """
    def process_request(self, request):
        if getattr(settings, 'CACHE_PREFIX', False):
            prefix_cache_object(settings.CACHE_PREFIX, cache)
        return

def prefix_cache_object(cache_prefix, cache_obj):
    """ Prefix the Django cache object (passed in) with the given cache prefix.
    Example usage:
    >>> from django.conf import settings
    >>> from django.core.cache import cache
    >>> from scoped_caching import prefix_cache_object
    >>> settings.CACHE_PREFIX
    'FOO_'
    # Do this once a process (e.g. on import or Middleware)
    >>> prefix_cache_object(settings.CACHE_PREFIX, cache)
    >>> cache.set("pi", 3.14159)
    >>> cache.get("pi")
    3.14159
    >>> cache.get("pi", use_global_namespace=True)
    >>> cache.get("FOO_pi", use_global_namespace=True)
    3.14159
    >>> cache.set("FOO_e", 2.71828, use_global_namespace=True)
    >>> cache.get("e")
    2.71828
    """
    if not cache_prefix:
        return

    cache_class = cache_obj.__class__

    if getattr(cache_class, '_PREFIXED', False):
        # If we're already prefixed, return.
        return

    def prefix_key(method):
        def _new_func(self, key, *args, **kwargs):
            use_global = kwargs.get('use_global_namespace', False)
            if 'use_global_namespace' in kwargs:
                del kwargs['use_global_namespace']
            if use_global:
                return method(self, key, *args, **kwargs)
            return method(self, cache_prefix + key,
                          *args, **kwargs)

        _new_func.__doc__ = method.__doc__
        _new_func.__name__ = method.__name__
        return _new_func

    def prefix_many(method):
        def _new_func(self, keys, *args, **kwargs):
            use_global = kwargs.get('use_global_namespace', False)
            if 'use_global_namespace' in kwargs:
                del kwargs['use_global_namespace']
            if use_global:
                return method(self, keys, *args, **kwargs)
            keys = [cache_prefix + key for key in keys]
            return method(self, keys, *args, **kwargs)

        _new_func.__doc__ = method.__doc__
        _new_func.__name__ = method.__name__
        return _new_func

    cache_class._PREFIXED = True
    cache_class.get = prefix_key(cache_class.get)
    cache_class.set = prefix_key(cache_class.set)
    cache_class.delete = prefix_key(cache_class.delete)
    cache_class.has_key = prefix_key(cache_class.has_key)
    cache_class.get_many = prefix_many(cache_class.get_many)

More like this

  1. Template tag - list punctuation for a list of items by shapiromatron 11 months, 2 weeks ago
  2. JSONRequestMiddleware adds a .json() method to your HttpRequests by cdcarter 11 months, 3 weeks ago
  3. Serializer factory with Django Rest Framework by julio 1 year, 6 months ago
  4. Image compression before saving the new model / work with JPG, PNG by Schleidens 1 year, 7 months ago
  5. Help text hyperlinks by sa2812 1 year, 8 months ago

Comments

crime_minister (on July 23, 2008):

Hi, I got bit by the issue that this snippet fixes when I built a bunch of Django applications that all used the same memcached instance. This snippet, or similar functionality, gets my vote for inclusion in mainline, FWIW!

#

Please login first before commenting.