Django snippets: caching parsed templates

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from django.template import loader
from django.conf import settings


template_cache = {} 
original_get_template = loader.get_template
def cached_get_template(template_name):
    global template_cache
    t = template_cache.get(template_name,None)
    if not t or settings.DEBUG:
        template_cache[template_name] = t = original_get_template(template_name)
    return t
loader.get_template = cached_get_template

More like this

server with debugging backdoor by teepark 3 years, 8 months ago
Page numbers with ... like in Digg by Ciantic 3 years, 1 month ago
Template filter that divides a list into exact columns by davmuz 3 months, 3 weeks ago
SOAP web service with soaplib 0.9+ by wRAR 1 year, 7 months ago
Integer list to pattern function by marinho 4 years, 5 months ago

Put this code somewhere in one of your INSTALLED_APPS __init__.py file. This code will replace the django.template.loader.get_template with cached version. Standard get_template function from django reads and parses the template code every time it's called. This version calls (if DEBUG set to False) it only once per template. After that it gets a Template object from template_cache dictionary. On django http server with template code like that:

{% extends "index.html" %}
{% block content %}
{% if form.has_errors %}
<p>Your username and password didn't match. Please try again.</p>
{% endif %}

<form method="post" action=".">
<table>
<tr><td><label for="id_username">Username:</label></td><td>{{ form.username }}</td></tr>
<tr><td><label for="id_password">Password:</label></td><td>{{ form.password }}</td></tr>
</table>

<input type="submit" value="login" />
<input type="hidden" name="next" value="{{ next }}" />
</form>

{% endblock %}

ab -n 100 on mac os x 10.5 core 2 duo 2 ghz with 2 GB of RAM gives

forge-macbook:~ forge$ ab -n 100 http://127.0.0.1:8000/login/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient).....done

Server Software:        WSGIServer/0.1
Server Hostname:        127.0.0.1
Server Port:            8000

Document Path:          /login/
Document Length:        934 bytes

Concurrency Level:      1
Time taken for tests:   0.432934 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      120200 bytes
HTML transferred:       93400 bytes
Requests per second:    230.98 [#/sec] (mean)
Time per request:       4.329 [ms] (mean)
Time per request:       4.329 [ms] (mean, across all concurrent requests)
Transfer rate:          270.25 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     3    3   1.5      4      12
Waiting:        3    3   1.2      3      12
Total:          3    3   1.5      4      12

Percentage of the requests served within a certain time (ms)
  50%      4
  66%      4
  75%      4
  80%      4
  90%      4
  95%      5
  98%     10
  99%     12
 100%     12 (longest request)

without template caching, and

forge-macbook:~ forge$ ab -n 100 http://127.0.0.1:8000/login/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient).....done

Server Software:        WSGIServer/0.1
Server Hostname:        127.0.0.1
Server Port:            8000

Document Path:          /login/
Document Length:        934 bytes

Concurrency Level:      1
Time taken for tests:   0.369860 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      120200 bytes
HTML transferred:       93400 bytes
Requests per second:    270.37 [#/sec] (mean)
Time per request:       3.699 [ms] (mean)
Time per request:       3.699 [ms] (mean, across all concurrent requests)
Transfer rate:          316.34 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     3    3   0.9      3       9
Waiting:        2    3   0.9      3       8
Total:          3    3   0.9      3       9

Percentage of the requests served within a certain time (ms)
  50%      3
  66%      3
  75%      3
  80%      3
  90%      3
  95%      5
  98%      8
  99%      9
 100%      9 (longest request)

with caching enabled. In both cases DEBUG is set to False.

Author:: forgems
Posted:: December 12, 2007
Language:: Python
Django Version:: .96
Tags:: template cache performance optimization
Score:: 9 (after 9 ratings)

Tools

Comments

zgoda (on December 13, 2007):

Be aware that changes in template require server restart to become visible. Anyway, very nice feature.

alexdong (on May 8, 2008):

There is a little problem with this since Python 2.4's cache framework couldn't pickle a compiled template object. Without that, a production box will barely benefit from the cached template by fetching it from memcached.

An alternative is to patch the template loader source code with this ticket.

Alex from haokanbu.com

fredz (on August 12, 2009):

Excellent !

I don't understand why this mecanism is not integrated in django.

The two only responses of django team about high load is to take care of sql queries and use cache, but there are dynamic sites (real estates for us) in which the cache is irrelevant (too many combinaisons of pages to use the cache). So bypassing the template parse for each query is a real optimisation (we have a 20-25% gain).

I don't really agree with the need of memcached, cause a site has a limited amount of templates, and it is more simple (and faster) to store the templates instance in a dict attached to each apache process, and reload the server when templates sources are modified.

Thank you.

fredz (on August 14, 2009):

See http://code.djangoproject.com/ticket/6262

mulianto (on April 7, 2012):

hi, is this snippet already in django distribute 1.3 ? i compare with per site cache and this one, the performance is faster with this snippet..

Django snippets

caching parsed templates

More like this

Tools

Comments

Snippets

About