- March 10, 2011
- middleware protection security spam refresh django-newcache hammering
- 2 (after 2 ratings)
A middleware which will protect from page hammering using flexible spanning time windows using the cache backend. Please read the Docstring of the class for details.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85
from django.conf import settings from django.core.cache import cache from collections import deque class HammeringMiddleware(object): """ A middleware which will protect from page hammering using flexible spanning time windows using the cache backend. One IP is allowed to do SPAM_REQUEST_WINDOW_PER_IP requests per SPAM_TIME_WINDOW seconds. If more requests are done, the middleware will simply redirect to a warning page specified in SPAM_WINDOW_EXCEEDED_VIEW and block requests to pages other than SPAM_WINDOW_EXCEEDED_VIEW for HAMMER_WINDOW_EXCEEDED_BLOCK seconds. The last SPAM_REQUEST_WINDOW_PER_IP request timestamps will be logged in the cache in a list (deque). This cache key by default lasts for a maximum of SPAM_REQUEST_WINDOW_PER_IP * SPAM_TIME_WINDOW and can be configured using SPAM_CACHE_TIMEOUT. Notes on Ajax requests: If you have a website that does many subrequests per page view, such as websites with much Ajax-based content, you may have to consider either lowering HAMMER_TIME_WINDOW or increasing HAMMER_REQUEST_WINDOW_PER_IP. The time window threshold may hit often on those mentioned sites and users within a network (with the same IP). You also have the option to turn HammeringMiddleware off for Ajax-based requests by enabling HAMMER_EXEMPT_AJAX in settings. This is not recommended, though. Note on django-newcache: If you are using newcache as your backend, which supports thundering herd mitigation, set HAMMER_CACHE_HAS_HERDING to True. This will append the kwarg herd=False to cache.set calls. Notes: - HammeringMiddleware should be the very first entry in your MIDDLEWARE_CLASSES tuple - This middleware should be used with a memory-based (locmem/memcached) cache backend but works well with others """ maxreq = getattr(settings, "HAMMER_REQUEST_WINDOW_PER_IP", 25) window = getattr(settings, "HAMMER_TIME_WINDOW", 5) ctimeout = getattr(settings, "HAMMER_CACHE_TIMEOUT", window * maxreq) blockFor = getattr(settings, "HAMMER_WINDOW_EXCEEDED_BLOCK", window) tooFastView = getattr(settings, "HAMMER_WINDOW_EXCEEDED_VIEW", "/hammering/") exemptAjax = getattr(settings, "HAMMER_EXEMPT_AJAX", False) skipHerd = getattr(settings, "HAMMER_CACHE_HAS_HERDING", False) def reject(self): return HttpResponseRedirect(self.tooFastView) def setCache(self, key, value, timeout): if self.skipHerd: cache.set(key, value, timeout, herd=False) else: cache.set(key, value, timeout) def process_request(self, request): # get ident from forwarded IP, else REMOTE_ADDR addr = request.META.get("HTTP_X_FORWARDED_FOR", request.META["REMOTE_ADDR"]) ident, identBlocked = "rsm_%s" % addr, "rsmb_%s" % addr # we don't want to hammer ourselves with our HttpResponseRedirect, if we are on the warning page. # if settings tells us we should exempt ajax requests, don't go futher, either. if (request.is_ajax() and self.exemptAjax) or request.path.startswith(self.tooFastView): return # user is blocked? don't handle the request if cache.get(identBlocked, False): return self.reject() reqinfo = cache.get(ident, deque(, self.maxreq+1)) cTime = time.time() # append the current timestamp to the list - we are using deque here, with maxitems set, so it will be capped at all times. reqinfo.append(cTime) # remove all timestamps from our list that are older than our time window for stamp in list(reqinfo): if cTime - stamp > self.window: reqinfo.popleft() self.setCache(ident, reqinfo, self.ctimeout) itemcount = len(reqinfo) if itemcount == self.maxreq and itemcount > 1: if reqinfo[-1] - reqinfo < self.window: # if we hit the threshold, redirect (the first item in the list, which is the oldest timestamp, # compared to the newest timestamp in the list, exceeds the time window) # block him also self.setCache(identBlocked, 1, self.blockFor) return self.reject()