- December 10, 2007
- cache model manager
- 1 (after 1 ratings)
I had a problem: too many fetches from the DB. So, how to reduce load on the database without major changes to the code? Cache Manager is the answer. I've managed to reduce number of DB hits as much as 80% in some cases (dictionaries, complex relations). It is using standard cache mechanisms. I'm using it with mathopd.
This is a very simple solution, instead of standard Manager, put this in your model definition as:
objects = CacheManager()
Then everythere elase in the code instead of all() or get(...) call all_cached() or get_cached().
I've kept original methods intact, to have an dual access, when you really, really must have frest data from the DB, and you can't wait for cache to expire.
This is much easier to work with, then manually getting fetched data from the cache.No change to your existing code 9except model) and voila!
Additionally if you have some data, you would like to store with your serialized object (e.g. related data, dynamic dictionaries), you can do this in the model method '_init_instance_cache').
Drop me an email if you find this useful. :)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
from django.core.cache import cache from django.db import models class CacheManager(models.Manager): def _getCacheKey(self, id, subset='s'): return "%s_%s_%s" % (self.model._meta.object_name, subset, id) def all_cached(self): if self.__class__.__name__ == 'RelatedManager': key = "".join(["%s%s" % el for el in self.core_filters.items()]) cacheKey = self._getCacheKey(0, key) else: cacheKey = self._getCacheKey(0, 'all') qs = cache.get(cacheKey) if qs is None: qs = list(self.get_query_set()) for element in qs: if hasattr(element,'_init_instance_cache'): element._init_instance_cache() cache.set(cacheKey, qs) return qs def get_cached(self, *args, **kwargs): cacheKey = cat = None if self.__class__.__name__ == 'RelatedManager': key = "".join(["%s%s" % el for el in self.core_filters.items()]) else: key = "" lst = map(lambda x:(x,x.pk if isinstance(x,models.Model) else str(x)),kwargs.items()) key += "_" + "_".join(list(map(unicode,args))+list(map(lambda x:"%s:%s"%x,lst))) key = key.replace(" ","") cacheKey = self._getCacheKey(key) # check if we have a result already ached if cacheKey is not None: element = cache.get(cacheKey) # no results, try to get it from cache if element is None: element = self.get_query_set().get(*args, **kwargs) if hasattr(element,'_init_instance_cache'): element._init_instance_cache() # if this should be cached, write it to the cache if cacheKey is not None: cache.set(cacheKey, element) return element
More like this
- Model manager with row caching by email@example.com 7 years, 1 month ago
- Lightweight querysets by sardarnl 2 years, 1 month ago
- Improved generic foreign key manager 2 by Nomalz 5 years, 9 months ago
- Manager for something like __inall by apollo13 7 years, 8 months ago
- Fetching top items by ubernostrum 8 years, 5 months ago