Login

Model manager with row caching

Author:
[email protected]
Posted:
June 18, 2008
Language:
Python
Version:
.96
Score:
4 (after 4 ratings)

RowCacheManager is a model manager that will try to fetch any 'get' (i.e., single-row) requests from the cache. ModelWithCaching is an abstract base model that does some extra work that you'll probably want if you're using the RowCacheManager. So to use this code, you just need to do two things: First, set objects=RowCacheManager() in your model definition, then inherit from ModelWithCaching if you want the invalidation for free. If you are unusually brave, use the metaclass for ModelWithCaching and you won't even need the "objects=RowCacheManager()" line.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
from django.conf import settings
from django.core.cache import cache
from django.db import models

CACHE_PREFIX = settings.CACHE_MIDDLEWARE_KEY_PREFIX
CACHE_EXPIRE = 30

def cache_key(model, id):
    return ('%s:%s:%s' % (CACHE_PREFIX, model._meta.db_table, id)).replace(' ', '')

class RowCacheManager(models.Manager):
    """Manager for caching single-row queries. To make invalidation easy,
    we use an extra layer of indirection. The query arguments are used as a
    cache key, whose stored value is the unique cache key pointing to the
    object. When a model using RowCacheManager is saved, this unique cache
    should be invalidated. Doing two memcached queries is still much faster
    than fetching from the database."""
    def get(self, *args, **kwargs):
        # TODO: skip the layer of indirection if 'kwargs' contains id or pk?
        id = repr(kwargs)        
        pointer_key = cache_key(self.model, id)
        model_key = cache.get(pointer_key)
        if model_key is not None: 
            model = cache.get(model_key)
            if model is not None:
                return model
        # One of the cache queries missed, so we have to get the object from the database:
        model = super(RowCacheManager, self).get(*args, **kwargs)
        if not model_key:
            model_key = cache_key(model, model.pk)
            cache.set(pointer_key, model_key, CACHE_EXPIRE)
        cache.set(model_key, model, CACHE_EXPIRE) # you can use 'add' instead of 'set' here
        return model
 
class ModelWithCaching(models.Model):
    def save(self, *args, **kwargs):
        super(ModelWithCaching, self).save()
        if kwargs.pop('invalidate_cache', True):
            cache.delete(cache_key(self, self.id))
    class Meta:
        abstract = True
############################# Cross this line at your own risk
    __metaclass__ = MetaCaching
        
ModelBase = type(models.Model)

class MetaCaching(ModelBase):
    """Sets ``objects'' on any model that inherits from ModelWithCaching to
    be a RowCacheManager. This is tightly coupled to Django internals, so it
    could break if you upgrade Django. This was done partially as a proof-of-
    concept. It is advised to only use code above the comment line."""
    def __new__(*args, **kwargs):
        new_class = ModelBase.__new__(*args, **kwargs)
        new_manager = RowCacheManager()
        new_manager.contribute_to_class(new_class, 'objects')
        new_class._default_manager = new_manager
        return new_class

More like this

  1. Template tag - list punctuation for a list of items by shapiromatron 3 months, 1 week ago
  2. JSONRequestMiddleware adds a .json() method to your HttpRequests by cdcarter 3 months, 1 week ago
  3. Serializer factory with Django Rest Framework by julio 10 months, 1 week ago
  4. Image compression before saving the new model / work with JPG, PNG by Schleidens 10 months, 4 weeks ago
  5. Help text hyperlinks by sa2812 11 months, 3 weeks ago

Comments

BenJackson (on June 17, 2010):

I added use_for_related_fields = True to enable caching of the automatic queries caused by accessing ForeignKey fields.

#

Please login first before commenting.