Login

Better Static Image Serving With Fallback

Author:
menendez
Posted:
March 21, 2010
Language:
Python
Version:
1.1
Score:
2 (after 2 ratings)

Serves images from a local directory, but if it doesn't find it locally, then go look for it on a server. This is useful for sites with very large amounts of static content.

Instead of copying all your prod images to dev every time you update the database, simply use this in your URL patterns to serve the images:

(r'^(?P<path>.*)$', 'static_fallback.serve', {'document_root' : '/path/to/my/files/'})

By default, it caches the images locally in the path it was supposed to find them. The fallback server URL can be specified in urls.py or settings as FALLBACK_STATIC_URL.

Thanks to Johnny Dobbins for the idea. Based on the Nginx pass through image proxy concept.

For more info http://menendez.com/blog/using-django-as-pass-through-image-proxy/

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
"""
Views and functions for serving static files. These are only to be used
during development, and SHOULD NOT be used in a production setting.

file: static_fallback.py
"""

import mimetypes
import os
import posixpath
import urllib
import urllib2

import django

from django.conf import settings
from django.http import Http404, HttpResponse

def serve(request, path, document_root=None, show_indexes=False, cache=True,
          fallback_server=None):
    """
    Serve static files using Django's static file function but if it returns a
    404, then attempt to find the file on the fallback_server. Optionally and by
    default, cache the file locally.
    
    To use, put a URL pattern such as::

        (r'^(?P<path>.*)$', 'static_fallback.serve', {'document_root' : '/path/to/my/files/'})

    in your URLconf. You must provide the ``document_root`` param (required by 
    Django). You may also set ``show_indexes`` to ``True`` if you'd like to 
    serve a basic index of the directory. These parameters are passed through
    directly to django.views.static.serve. You should see the doc_string there 
    for details.
    
    Passing cache to True (default) copies the file locally.
    
    Be sure to set settings.FALLBACK_STATIC_URL to something like:
    
    FALLBACK_STATIC_URL = 'http://myprodsite.com'
    
    Alternatively, you can also tell it the fallback server as a parameter
    sent in the URLs.
    
    Author: Ed Menendez ([email protected])
    Concept: Johnny Dobbins
    """
    
    # This was mostly copied from Django's version. We need the filepath for 
    # caching and it also serves as an optimization. If the file is not found
    # then there's no reason to go through the Django process.
    try:
        fallback_server = settings.FALLBACK_STATIC_URL
    except AttributeError:
        print u"You're using static_fallback.serve to serve static content " + \
               "however settings.FALLBACK_STATIC_URL has not been set."
    
    # Save this for later to pass to Django.
    original_path = path
    
    path = posixpath.normpath(urllib.unquote(path))
    path = path.lstrip('/')
    newpath = ''
    for part in path.split('/'):
        if not part:
            # Strip empty path components.
            continue
        drive, part = os.path.splitdrive(part)
        head, part = os.path.split(part)
        if part in (os.curdir, os.pardir):
            # Strip '.' and '..' in path.
            continue
        newpath = os.path.join(newpath, part).replace('\\', '/')
    if newpath and path != newpath:
        return HttpResponseRedirect(newpath)                    # RETURN
    fullpath = os.path.join(document_root, newpath)
    # End of the "mostly from Django" section.

    try:
        # Don't bother trying the Django function if the file isn't there.
        if not os.path.isdir(fullpath) and not os.path.exists(fullpath):
            raise Http404, '"%s" does not exist' % fullpath     # RAISE
        else:
            # Pass through cleanly to Django's verson
            return django.views.static.serve(                   # RETURN
                        request, original_path, document_root, show_indexes)
    except Http404:
        if fallback_server:
            # Attempt to find it on the remote server.
            fq_url = '%s%s' % (fallback_server, request.path_info)
            try:
                contents = urllib2.urlopen(fq_url).read()
            except urllib2.HTTPError:
                # Naive to assume a 404 - ed
                raise Http404, '"%s" does not exist' % fq_url   # RAISE
            else:
                # Found the doc. Return it to response.
                mimetype = mimetypes.guess_type(fq_url)
                response = HttpResponse(contents, mimetype=mimetype)
                
                # Do we need to cache the file?
                if cache:
                    f = open(fullpath, 'wb+')
                    f.write(contents)
                    f.close()
                
                # Success! We have the file. Send it back.
                return response                                 # RETURN
        else:
            # No fallback_server was defined. So, it's really a 404 now.
            raise Http404                                       # RAISE

More like this

  1. Template tag - list punctuation for a list of items by shapiromatron 3 months, 1 week ago
  2. JSONRequestMiddleware adds a .json() method to your HttpRequests by cdcarter 3 months, 2 weeks ago
  3. Serializer factory with Django Rest Framework by julio 10 months, 2 weeks ago
  4. Image compression before saving the new model / work with JPG, PNG by Schleidens 11 months ago
  5. Help text hyperlinks by sa2812 12 months ago

Comments

Please login first before commenting.