Login

Templatetag for JS merging and compression

Author:
msaelices
Posted:
September 5, 2007
Language:
Python
Version:
.96
Tags:
javascript merging compression tunning
Score:
7 (after 7 ratings)

Javascript merging and compression templatetag

One of the most important things for improving web performance is to reduce the number of HTTP requests. This is a templatetag that merges several javascript files (compressing its code) into only one javascript.

It checks if this merged file is up to date, comparing modification time with the other javascripts to merge.

Usage:

{% load jsmerge %}
{% jsmerge mergedfile js/file1.js js/file2.js js/file3.js %}

The previous code will:

  1. Search in settings.MEDIA_ROOT for all files passed by parameter.
  2. Create a /path/to/media_root/mergedfile.js (if it doesn't exist or it's not up to date). This file is a merging plus compression of all javascripts.
  3. Return this HTML fragment: <script type="text/javascript" src="/media_url/mergedfile.js"></script>
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
import os, re
from os import path

from django.template import Library, Node, TemplateSyntaxError
from django.conf import settings

register = Library()

class JSPacker:
    """ JS compressor. This commpress a javascript content
        This class code is adapted from http://plone.org/products/resourceregistries """

    def __init__(self):
        # protect this strings:
        #  match a single quote
        #  match anything but the single quote, a backslash and a newline "[^'\\\n]"
        #  or match a null escape (\0 not followed by another digit) "\\0(?![0-9])"
        #  or match a character escape (no newline) "\\[^\n]"
        self.patterns = []
        self.protect(r"""('(?:[^'\\\n]|\\0(?![0-9])|\\x[0-9a-fA-F]{2}|\\u[0-9a-fA-F]{4}|\\[^\n])*?'|"""
                     r""""(?:[^"\\\n]|\\0(?![0-9])|\\x[0-9a-fA-F]{2}|\\u[0-9a-fA-F]{4}|\\[^\n])*?")""")

        # protect regular expressions
        self.protect(r"""\s+(\/[^\/\n\r\*](?:\\/|[^\n\r])*\/g?i?)""")
        self.protect(r"""([^\w\$\/'"*)\?:]\/[^\/\n\r\*](?:\\/|[^\n\r])*\/g?i?)""")

        # protect IE conditional compilation
        self.protect(r'(/\*@.*?(?:\*/|\n|\*/(?!\n)))', re.DOTALL)

        # remove multiline comments
        self.sub(r'/\*.*?\*/', '', re.DOTALL)

        # strip whitespace at the beginning and end of each line
        self.sub(r'^[ \t\r\f\v]*(.*?)[ \t\r\f\v]*$', r'\1', re.MULTILINE)

        # after an equal sign a function definition is ok
        self.sub(r'=\s+(?=function)', r'=')

        # whitespace before some special chars
        self.sub(r'\s+([={},&|\?:\.()<>%!/\]])', r'\1')

        # whitespace before plus chars if no other plus char before i
        self.sub(r'(?<!\+)\s+\+', '+')

        # whitespace after plus chars if no other plus char after it
        self.sub(r'\+\s+(?!\+)', '+')
        # whitespace before minus chars if no other minus char before it
        self.sub(r'(?<!-)\s+-', '-')
        # whitespace after minus chars if no other minus char after it
        self.sub(r'-\s+(?!-)', '-')
        # remove redundant semi-colons
        self.sub(r';+\s*([};])', r'\1')
        # remove any excessive whitespace left except newlines
        self.sub(r'[ \t\r\f\v]+', ' ')
        # excessive newlines
        self.sub(r'\n+', '\n')
        # first newline
        self.sub(r'^\n', '')

    def protect(self, pattern, flags=None):
        if flags is None:
            self.patterns.append((re.compile(pattern), None))
        else:
            self.patterns.append((re.compile(pattern, flags), None))

    def sub(self, pattern, replacement, flags=None):
        if flags is None:
            self.patterns.append((re.compile(pattern), replacement))
        else:
            self.patterns.append((re.compile(pattern, flags), replacement))

    def copy(self):
        result = Packer()
        result.patterns = self.patterns[:]
        return result

    def _repl(self, match):
        # store protected part
        self.replacelist.append(match.group(1))
        # return escaped index
        return "\x00%i\x00" % len(self.replacelist)

    def pack(self, input):
        # list of protected parts
        self.replacelist = []
        output = input
        for regexp, replacement in self.patterns:
            if replacement is None:
                output = regexp.sub(self._repl, output)
            else:
                # substitute
                output = regexp.sub(replacement, output)
        # restore protected parts
        replacelist = list(enumerate(self.replacelist))
        replacelist.reverse() # from back to front, so 1 doesn't break 10 etc.
        for index, replacement in replacelist:
            # we use lambda in here, so the real string is used and no escaping
            # is done on it
            before = len(output)
            regexp = re.compile('\x00%i\x00' % (index+1))
            output = regexp.sub(lambda m:replacement, output)
        # done
        return output

# singleton object.
jspacker = JSPacker()


class JSMergeNode(Node):
    def __init__(self, js_name, js_files):
        self.js_name = '%s.js' % js_name
        self.js_files = js_files
        self.merge_filename = path.join(settings.MEDIA_ROOT, self.js_name)

    def render(self, context):
        if not path.exists(self.merge_filename) or not self.is_merge_updated():
            # we merge all javascript files
            merge_file = open(self.merge_filename, 'w')
            for js in self.js_files:
                jspath = path.join(settings.MEDIA_ROOT, js)
                if not path.isfile(jspath):
                    continue
                self.merge_js(js, jspath, merge_file)
            merge_file.close()
        return self.js_tag()

    def is_merge_updated(self):
        """ compares modification time of all js with merged js """
        last_mtime = 0 # last modification time of a js file 
        for js in self.js_files:
            jspath = path.join(settings.MEDIA_ROOT, js)
            jsstat = os.stat(jspath)
            mtime = jsstat[-2]
            if last_mtime < mtime:
                last_mtime = mtime
        merge_mtime = os.stat(self.merge_filename)[-2]
        return merge_mtime > last_mtime

    def merge_js(self, jsname, jspath, fd):
        """ do merging and compressing of javascript """
        global jspacker
        jsfile = open(jspath)
        jscontent = jsfile.read()
        jscontent = jspacker.pack(jscontent)
        fd.write('/* -- %s -- */\n\n%s\n\n\n' % (jsname, jscontent))

    def js_tag(self):
        """ write js tag for merged file inclusion """
        js_url = '%s%s' % (settings.MEDIA_URL, self.js_name)
        return '<script type="text/javascript" src="%s"></script>' % js_url


def do_jsmerge(parser, token):
    """
    This will merge javascript files in only one compressed javascript.

    Usage::

        {% load jsmerge %}
        {% jsmerge jsname [jsfile1] [jsfile2] .. %} 

    Example::

        {% load jsmerge %}
        {% jsmerge jsmergefile  js/file1.js js/file2.js js/file3.js %}

    This will create (if not exists) a /media/jsmergefile.js with three files merged. The HTML output for this will be::

        <script type="text/javascript" src="/media/jsmergefile.js"></script>
    """
    tokens = token.contents.split()
    if len(tokens) < 2:
        raise TemplateSyntaxError(u"'%r' tag requires at least 1 arguments." % tokens[0])
    js_name = tokens[1]
    return JSMergeNode(js_name, tokens[2:])

register.tag('jsmerge', do_jsmerge)

More like this

  1. Template tag for compressed CSS & JS (GAE version) by jeffar 6 years, 9 months ago
  2. head inclusion middleware by bowdengm 6 years, 2 months ago
  3. Google Closure support in django-compress by fabrice.bonny 5 years, 5 months ago
  4. YUI Loader as Django middleware by akaihola 6 years, 12 months ago
  5. A dict template tag by Batiste 6 years, 12 months ago

Comments

amirouche (on September 5, 2007):

What do you want to do is adding some "logic" between the static data and the result, just as we do with data in a database. It's not recommended to use Django to serve static files so how do you do ?

#

cdome (on September 8, 2007):

Excellent snippet. Looking forward for css compressor snippet

#

msaelices (on September 15, 2007):

CSS compressor it's easy too. I'll code it when i have free time

#

Please login first before commenting.