Login

Cacheable resources

Author:
jbrisbin
Posted:
August 18, 2008
Language:
Python
Version:
.96
Tags:
md5 cacheable cacheing
Score:
1 (after 1 ratings)

This snippet provides a template tag that automatically replaces references to any resource you want cached forever with a version of the file that is based on the MD5 sum. For an image, you would use something like:

{% load utils %}
<img src="{% cacheable "/media/images/logo.png" %}"/>

To install it, put a setting in your settings.py file called "DOCUMENT_ROOT", put the python code into a templatetag-friendly file (e.g. app/templatetags/utils.py), load that template tag, then use either a string literal, as above, or a variable name to refer to your resource:

<img src="{% cacheable my_media_file %}"/>

The cacheable resource will be used when DEBUG = False, but in DEBUG mode, the path you give it will be passed back untouched (so you don't have a proliferation of cacheable files as you develop).

Django will need write access to the directory you've specified as "DOCUMENT_ROOT" (so it can copy the original file into a forever-cacheable version).

You'll also need to set up your webserver to serve files called "MYMD5SUMNAME.cache.(js|css|png|gif|jpg) with an expires header that is far into the future. The goal here is to create a version of your file that will never have to be downloaded again. If you ever change the original file, the MD5 sum will change and the changed file's cacheable name will reflect that.

Besides simply changing the name of resources, if the file is a JavaScript or CSS file, and you've specified MINIFY = True, the file will be minified using YUI compressor.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
import os.path, hashlib, subprocess, re, shutil
from django import template
from django.conf import settings
from django.core.cache import cache

register = template.Library()

js_or_css = re.compile("(.*)\.(js|css)")
is_path = re.compile("\"(.*)\"")

# Locations of things for YUI compressor
JAVA = "/usr/bin/java"
YUI_COMPRESSOR_JAR = "/path/to/yuicompressor-2.3.6/build/yuicompressor-2.3.6.jar"
MINIFY = True

class CacheableNode(template.Node):
  
  def __init__(self, rpath, path_literal=False):
    self.path_literal = path_literal
    if path_literal:
      self.rpath = self.cache_resource(rpath)
    else:
      self.rpath = template.Variable(rpath)
        
  def cache_resource(self, rpath):
    if not settings.DEBUG:
      cached_res = cache.get(rpath, False)
      if not cached_res:
        if rpath[0] == "/":
          rpath = rpath[1:]
        res_path = os.path.join(settings.DOCUMENT_ROOT, rpath)
        if os.path.exists(res_path):
            # Open the file for hashing
          fin = open(res_path, "r")
          s = fin.read()
          fin.close()
          # Update the MD5
          hash = hashlib.md5()
          hash.update(s)
          
          (path, fname) = os.path.split(res_path)
          cache_name = os.path.join(path, hash.hexdigest() + ".cache" + os.path.splitext(fname)[1])
          if not os.path.exists(cache_name):
            if js_or_css.match(fname) and MINIFY:        
              # Write out the cachable file
              minified = open(cache_name, "w")
              yuic = subprocess.Popen((JAVA, "-jar", YUI_COMPRESSOR_JAR, res_path), stdout=minified)
              rtncode = yuic.wait()
              minified.write("\n")
              minified.close()
              # If an error happened during minification, just copy the file over
              if rtncode != 0:
                print "ERROR Minifying %s" % res_path
                shutil.copyfile(res_path, cache_name)
            else:
              shutil.copyfile(res_path, cache_name)
          
          # Strip DOCUMENT_ROOT off file, leaving relative path
          cache_path = cache_name.replace(settings.DOCUMENT_ROOT, "")
          cache.set(rpath, cache_path, 3600) # Cache mapping for an hour
          if cache_path[0] != "/":
            cache_path = "/%s" % cache_path
          return cache_path
        else:
          return rpath
      # Return mapped path to cachable resource
      return cached_res
    return rpath
  
  def render(self, context):
    if self.path_literal:
      return self.rpath
    else:
      rpath = self.cache_resource(self.rpath.resolve(context))
      return rpath

@register.tag
def cachable(parser, token):
  try:
    tag_name, rpath = token.split_contents()
    if is_path.match(rpath):
      path_literal = True
    else:
      path_literal = False
    rpath = rpath.replace("\"", "")
  except ValueError:
    raise template.TemplateSyntaxError, "%r tag requires a valid path relative to settings.DOCUMENT_ROOT" % token.contents.split()[0]
  
  return CacheableNode(rpath, path_literal=path_literal)

More like this

  1. Template Tag of Django Image Thumb Creator by ayang23 4 years, 2 months ago
  2. Auto Generate/Save Thumbnails using Template Filter (scale max_x, max_y, or both) by ThisbeTom 6 years, 4 months ago
  3. [UPDATE]Filter to resize a ImageField on demand by rafacdb 6 years, 8 months ago
  4. Template tag for compressed CSS & JS (GAE version) by jeffar 6 years, 10 months ago
  5. staticview for app by limodou 7 years, 10 months ago

Comments

jbrisbin (on August 19, 2008):

Actually, I think this template tag is quite a bit different than versioned_media. Besides YUI compressor support, I wanted forever caching based on the file contents' MD5 sum, not the filesystem's modification time, which can get updated at any time, even when the file's contents don't change. This also provides clean urls rather than using query string parameters.

I say forever caching, but that's not entirely true because it is strongly suggested that HTTP/1.1 responses not have future expire dates longer than 1 year. Despite that, a version of your file that is based on the MD5 sum will still be cacheable if you upgrade servers, move directories, use it in load-balancing situations, or other such server shenanigans. I'm using MD5 sums throughout my application for ETag support, so it was natural to choose that method for the template tag.

#

Please login first before commenting.