Login

Model manager with row caching

Author:
jobs@flowgram.com
Posted:
June 18, 2008
Language:
Python
Version:
.96
Tags:
get model manager model-inheritance caching
Score:
4 (after 4 ratings)

RowCacheManager is a model manager that will try to fetch any 'get' (i.e., single-row) requests from the cache. ModelWithCaching is an abstract base model that does some extra work that you'll probably want if you're using the RowCacheManager. So to use this code, you just need to do two things: First, set objects=RowCacheManager() in your model definition, then inherit from ModelWithCaching if you want the invalidation for free. If you are unusually brave, use the metaclass for ModelWithCaching and you won't even need the "objects=RowCacheManager()" line.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
from django.conf import settings
from django.core.cache import cache
from django.db import models

CACHE_PREFIX = settings.CACHE_MIDDLEWARE_KEY_PREFIX
CACHE_EXPIRE = 30

def cache_key(model, id):
    return ('%s:%s:%s' % (CACHE_PREFIX, model._meta.db_table, id)).replace(' ', '')

class RowCacheManager(models.Manager):
    """Manager for caching single-row queries. To make invalidation easy,
    we use an extra layer of indirection. The query arguments are used as a
    cache key, whose stored value is the unique cache key pointing to the
    object. When a model using RowCacheManager is saved, this unique cache
    should be invalidated. Doing two memcached queries is still much faster
    than fetching from the database."""
    def get(self, *args, **kwargs):
        # TODO: skip the layer of indirection if 'kwargs' contains id or pk?
        id = repr(kwargs)        
        pointer_key = cache_key(self.model, id)
        model_key = cache.get(pointer_key)
        if model_key is not None: 
            model = cache.get(model_key)
            if model is not None:
                return model
        # One of the cache queries missed, so we have to get the object from the database:
        model = super(RowCacheManager, self).get(*args, **kwargs)
        if not model_key:
            model_key = cache_key(model, model.pk)
            cache.set(pointer_key, model_key, CACHE_EXPIRE)
        cache.set(model_key, model, CACHE_EXPIRE) # you can use 'add' instead of 'set' here
        return model
 
class ModelWithCaching(models.Model):
    def save(self, *args, **kwargs):
        super(ModelWithCaching, self).save()
        if kwargs.pop('invalidate_cache', True):
            cache.delete(cache_key(self, self.id))
    class Meta:
        abstract = True
############################# Cross this line at your own risk
    __metaclass__ = MetaCaching
        
ModelBase = type(models.Model)

class MetaCaching(ModelBase):
    """Sets ``objects'' on any model that inherits from ModelWithCaching to
    be a RowCacheManager. This is tightly coupled to Django internals, so it
    could break if you upgrade Django. This was done partially as a proof-of-
    concept. It is advised to only use code above the comment line."""
    def __new__(*args, **kwargs):
        new_class = ModelBase.__new__(*args, **kwargs)
        new_manager = RowCacheManager()
        new_manager.contribute_to_class(new_class, 'objects')
        new_class._default_manager = new_manager
        return new_class

More like this

  1. Ordered items in the database - alternative by Leonidas 7 years, 10 months ago
  2. Cache Manager by jerzyk 7 years, 4 months ago
  3. Custom Django manager that excludes subclasses by sciyoshi 6 years, 8 months ago
  4. Get actual child model in multi-table inheritance by bronger 4 years, 9 months ago
  5. Binding signals to abstract models by andreterra 2 years, 11 months ago

Comments

BenJackson (on June 17, 2010):

I added use_for_related_fields = True to enable caching of the automatic queries caused by accessing ForeignKey fields.

#

Please login first before commenting.