Login

caching parsed templates

Author:
forgems
Posted:
December 12, 2007
Language:
Python
Version:
.96
Tags:
template cache performance optimization
Score:
9 (after 9 ratings)

Put this code somewhere in one of your INSTALLED_APPS __init__.py file. This code will replace the django.template.loader.get_template with cached version. Standard get_template function from django reads and parses the template code every time it's called. This version calls (if DEBUG set to False) it only once per template. After that it gets a Template object from template_cache dictionary. On django http server with template code like that:

{% extends "index.html" %}
{% block content %}
{% if form.has_errors %}
<p>Your username and password didn't match. Please try again.</p>
{% endif %}

<form method="post" action=".">
<table>
<tr><td><label for="id_username">Username:</label></td><td>{{ form.username }}</td></tr>
<tr><td><label for="id_password">Password:</label></td><td>{{ form.password }}</td></tr>
</table>

<input type="submit" value="login" />
<input type="hidden" name="next" value="{{ next }}" />
</form>

{% endblock %}

ab -n 100 on mac os x 10.5 core 2 duo 2 ghz with 2 GB of RAM gives

forge-macbook:~ forge$ ab -n 100 http://127.0.0.1:8000/login/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient).....done

Server Software:        WSGIServer/0.1
Server Hostname:        127.0.0.1
Server Port:            8000

Document Path:          /login/
Document Length:        934 bytes

Concurrency Level:      1
Time taken for tests:   0.432934 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      120200 bytes
HTML transferred:       93400 bytes
Requests per second:    230.98 [#/sec] (mean)
Time per request:       4.329 [ms] (mean)
Time per request:       4.329 [ms] (mean, across all concurrent requests)
Transfer rate:          270.25 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     3    3   1.5      4      12
Waiting:        3    3   1.2      3      12
Total:          3    3   1.5      4      12

Percentage of the requests served within a certain time (ms)
  50%      4
  66%      4
  75%      4
  80%      4
  90%      4
  95%      5
  98%     10
  99%     12
 100%     12 (longest request)

without template caching, and

forge-macbook:~ forge$ ab -n 100 http://127.0.0.1:8000/login/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient).....done

Server Software:        WSGIServer/0.1
Server Hostname:        127.0.0.1
Server Port:            8000

Document Path:          /login/
Document Length:        934 bytes

Concurrency Level:      1
Time taken for tests:   0.369860 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      120200 bytes
HTML transferred:       93400 bytes
Requests per second:    270.37 [#/sec] (mean)
Time per request:       3.699 [ms] (mean)
Time per request:       3.699 [ms] (mean, across all concurrent requests)
Transfer rate:          316.34 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     3    3   0.9      3       9
Waiting:        2    3   0.9      3       8
Total:          3    3   0.9      3       9

Percentage of the requests served within a certain time (ms)
  50%      3
  66%      3
  75%      3
  80%      3
  90%      3
  95%      5
  98%      8
  99%      9
 100%      9 (longest request)

with caching enabled. In both cases DEBUG is set to False.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
#!/usr/bin/env python
# -*- coding: utf-8 -*-

from django.template import loader
from django.conf import settings


template_cache = {} 
original_get_template = loader.get_template
def cached_get_template(template_name):
    global template_cache
    t = template_cache.get(template_name,None)
    if not t or settings.DEBUG:
        template_cache[template_name] = t = original_get_template(template_name)
    return t
loader.get_template = cached_get_template

More like this

  1. Yet another SQL debugging facility by miracle2k 7 years, 8 months ago
  2. Effective content caching for mass-load site using redirect feature by nnseva 3 years, 9 months ago
  3. enani by andisthermal 4 years, 3 months ago

Comments

zgoda (on December 13, 2007):

Be aware that changes in template require server restart to become visible. Anyway, very nice feature.

#

fredz (on August 12, 2009):

Excellent !

I don't understand why this mecanism is not integrated in django.

The two only responses of django team about high load is to take care of sql queries and use cache, but there are dynamic sites (real estates for us) in which the cache is irrelevant (too many combinaisons of pages to use the cache). So bypassing the template parse for each query is a real optimisation (we have a 20-25% gain).

I don't really agree with the need of memcached, cause a site has a limited amount of templates, and it is more simple (and faster) to store the templates instance in a dict attached to each apache process, and reload the server when templates sources are modified.

Thank you.

#

mulianto (on April 7, 2012):

hi, is this snippet already in django distribute 1.3 ? i compare with per site cache and this one, the performance is faster with this snippet..

#

Please login first before commenting.