I've been working with a data set where a single object won't fit into memcached's 1 Mb slab limit. The two functions have been useful to me for debugging the size of the data structure once pickled, and if said pickled data structure is greater than 1 Mb.
These functions assume CACHE_BACKEND is memcached, obviously.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | import memcache
try:
from cStringIO import StringIO
except ImportError:
from StringIO import StringIO
try:
import cPickle as pickle
except ImportError:
import pickle
def data_size(data):
"""Returns the size in bytes of the data passed in."""
fp = StringIO()
pickler = pickle.Pickler(fp, protocol=0)
pickler.dump(data)
val = fp.getvalue()
return len(val)
def data_too_large(data):
"""Returns True if the data passed in is too large for memcached."""
return data_size(data) >= memcache.SERVER_MAX_VALUE_LENGTH
|
More like this
- Template tag - list punctuation for a list of items by shapiromatron 11 months, 2 weeks ago
- JSONRequestMiddleware adds a .json() method to your HttpRequests by cdcarter 11 months, 3 weeks ago
- Serializer factory with Django Rest Framework by julio 1 year, 6 months ago
- Image compression before saving the new model / work with JPG, PNG by Schleidens 1 year, 7 months ago
- Help text hyperlinks by sa2812 1 year, 7 months ago
Comments
Please login first before commenting.