- Author:
- bradbeattie
- Posted:
- November 21, 2012
- Language:
- Python
- Version:
- 1.4
- Score:
- 0 (after 0 ratings)
This is caching mechanism augmented to help address a number of common pitfalls in caching: cache stampeding, simultaneous expiry, cache invalidation, and the storing of None as a legitimate value.
No doubt the automatic key generation could stand to be simplified, but it works.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 | # Based on http://djangosnippets.org/snippets/564/
# Modified to stagger cache durations
# Modified to prevent cache stampeding (on cache miss, inserts a placeholder while regenerating a hit)
# Modified to allow correct storage of None results without triggering cache expiry
# Modified to incorporate object updated attributes to help expire keys
from django.core.cache import cache
from django.db.models import Model
from hashlib import sha1
from random import randint
import math
def cache_result(seconds=3600, expiry_variance=0.2, override_key=None):
def doCache(f):
def x(*args, **kwargs):
# Don't bother if seconds is set to 0
if not seconds:
return f(*args, **kwargs)
# Generate the key from the function name and given arguments
key = sha1(override_key or u"//".join((
unicode(f),
u"//".join(objectToString(a) for a in args),
u"//".join(unicode(a.updated) for a in args if hasattr(a, "updated")),
u"//".join(unicode(k) + objectToString(v) for k, v in kwargs.iteritems()),
u"//".join(unicode(v.updated) for k, v in kwargs.iteritems() if hasattr(v, "updated")),
)).encode("utf-8")).hexdigest()
flag = key + "flag"
# If a cached result exists, return it.
skip_cache_read = kwargs.pop("skip_cache_read", False)
result = cache.get(key) if not skip_cache_read and cache.get(flag) else None
if result:
return result[0]
# If no result exists, generate one and return it. While generating a result,
# postpone further regenerations to prevent cache stampeding.
cache.set(flag, True, int(math.log(seconds)))
result = f(*args, **kwargs)
cache.set(flag, True, seconds + randint(-int(seconds * expiry_variance), int(seconds * expiry_variance)))
cache.set(key, (result, ), max(seconds * 10, 86400 * 7))
return result
return x
def objectToString(obj):
if isinstance(obj, Model):
object_class = type(obj)
return ".".join((
object_class.__module__,
object_class.__name__,
unicode(obj.pk),
))
else:
return unicode(obj)
return doCache
|
More like this
- Template tag - list punctuation for a list of items by shapiromatron 10 months, 2 weeks ago
- JSONRequestMiddleware adds a .json() method to your HttpRequests by cdcarter 10 months, 3 weeks ago
- Serializer factory with Django Rest Framework by julio 1 year, 5 months ago
- Image compression before saving the new model / work with JPG, PNG by Schleidens 1 year, 6 months ago
- Help text hyperlinks by sa2812 1 year, 6 months ago
Comments
Please login first before commenting.