Login

All snippets written in Python

Snippet List

Random object IDs using an abstract base model

To put obfuscated primary keys in any class, simply inherit from this one. For example: class Offer(ObfuscatedPKModel) You can match for these bigint primary keys in your urls.py like this: '^offer/(?P<offer_pk>[0-9\-]+)$'

  • models
  • model
  • random
  • abstract
  • primary-key
  • obfuscation
  • obfuscated
  • ID
Read More

Transparently encrypt ORM fields using OpenSSL (via M2Crypto)

Sometimes you need to store information that the server needs in unencrypted form (e.g. OAuth keys and secrets), but you don't really want to leave it lying around in the open on your server. This snippet lets you split that information into two parts: * a securing passphrase, stored in the Django settings file (or at least made available via that namespace) * the actual secret information, stored in the ORM database Obviously, this isn't as secure as using a full blown key management system, but it's still a significant step up from storing the OAuth keys directly in the settings file or the database. Note also that these fields will be displayed unencrypted in the admin view unless you add something like the following to ``admin.py``: from django.contrib import admin from django import forms from myapp.fields import EncryptedCharField class MyAppAdmin(admin.ModelAdmin): formfield_overrides = { EncryptedCharField: {'widget': forms.PasswordInput(render_value=False)}, } admin.site.register(PulpServer, PulpServerAdmin) If Django ever acquires a proper binary data type in the default ORM then the base64 encoding part could be skipped. This snippet is designed to be compatible with the use of the South database migration tool *without* exposing the passphrase used to encrypt the fields in the migration scripts. (A migration tool like South also allows you to handle the process of *changing* the passphrase, by writing a data migration script that decrypts the data with the old passphrase then writes it back using the new one). Any tips on getting rid of the current ugly prefix hack that handles the difference between deserialising unencrypted strings and decrypting the values stored in the database would be appreciated! Sources of inspiration: [AES encryption with M2Crypto](http://passingcuriosity.com/2009/aes-encryption-in-python-with-m2crypto/) [EncryptedField snippet](http://djangosnippets.org/snippets/1095/) (helped me improve several Django-specific details)

  • fields
  • encryption
Read More

django app name & breadcrumbs l10n

This approach allows you to rename you app name & app breadcrumbs within admin interface. Paste the code in admin.py of your application and change AppLabelRenamer.main params.

  • admin
  • applications
  • admin-interface
  • breadcrumbs
Read More

Dynamically add css-classes to formfields

This example assumes you have a form and want to highlight one of two fields by setting <class="highlight"> in the html dynamically. This is an alternative to <https://docs.djangoproject.com/en/1.3/ref/forms/widgets/#customizing-widget-instances>, but now you're not limited to assigning the class to the fields html-output, instead you can also assign it to a div around the field like done here. After assigning a css-attribute to a field, we access the css via a templatefilter *{{ field|css }}* that looks up *field.form.fields[field.name].css* and not simply *field.css*, since the latter would try to access a non-existing css-attribute on a BoundField-instance EDIT: The templatefilter is unnecessary. There is a much easier way, since the original field itself is an attribute of the BoundField named 'field'. So in the template, we can access the css via {{ field.field.css }}. Thanks to Tom Evans for pointing me at this.

  • css
  • form
  • class
  • dynamic-form
  • formfield
  • dynamic-css
  • css-class
Read More

PropertyBasedContext

This approach allows you to avoid code duplication to produce same context data for different views. It could be usefull when you are using templates inheritace.

  • context
  • contextprocessor
Read More

Template loader to target a specific template

This is http://djangosnippets.org/snippets/1376/ rewritten as a new class-style Loader, and slightly improved. Allows you to reference templates like this: app_label:some/template/name.html This makes it possible to insert customizations at any point in a template hierarchy. For example, you could replace a block within the base admin template: {% extends "admin:admin/base.html" %} {% block breadcrumbs %}Custom Breadcrumbs-Style{% endblock %}

  • templates
  • loader
Read More

View all log entries in the admin

By default, you can only see the action log ("History") for particular model instances and a list of your own actions on the admin's index. This adds a fully-fledged admin view for the LogEntry model, where you can filter actions by user, content type, action type, browse by change date, and also search in the change message. Add the code to any of your apps' admin.py. The entries will be visible only to superusers and won't be editable.

  • admin
  • logentry
Read More

Prettify HTML5 middleware

Slightly updated version of http://djangosnippets.org/snippets/172/ . Supports new HTML5 tags (even though tidy doesn't).

  • middleware
  • tidy
  • standard
  • html5
Read More

Effective content caching for mass-load site using redirect feature

Hi All, I would like to share my idea and experience in using http redirect feature to organize effective content caching and avoid site blocking on mass requests to not yet cached content. First of all, I should explain that I don't mean usual template-based pages, whose caching in django is enough in almost all cases. My speech is concerning to content such as dynamically created images, external files and other content which may require almost unpredictable long time period to be ready, up to about 10 seconds. Anyway, the content which is already in cache, can be returned immediately. The question is - what *should* happen if the requested content is *not yet present* in the cache? The HTML content is not a problem in this case - we can return intermediate page with meta information to ask a client to retry the content after some period of time. But non-HTML content, such as image, or json, is a problem. The usual *blocking* behavior of the django caching subsystem is not appropriate in a case when the content preparation process takes *more than several fractions* of a second. The whole period while the content is preparing (painting, getting from the external resource and so on) *hundreds* of users will *ask not yet cached* content until all available application server connections will be *occupied* by waiting requests and new user will wait for the content even if the content requested by him is actually ready to be returned. The celery package resolves this problem *partially*, allowing to schedule cache update before cache expiration in asynchronous manner (I am using threadpool package for this purpose instead), and avoiding blocking on content preparation procedure, but the problem with not yet available content remains unresolved. We might schedule periodic cache update to avoid such a situation, but this way leads to unnecessary resource load and may be inappropriate at all in some cases, f.e. for the service like CloudMade or Google "staticmap" painting map content basing on the request parameters. My solution is *using HTTP Redirect feature* . Ideal solution would be available if most of HTTP clients supported Retry-After header in the HTTP Redirect response. Unfortunately, no one of most popular clients (neither Mozilla, nor IE, nor Chrome) does it. Really they ask redirected URL immediately after receiving the redirection response, and ignore Retry-After header at all. Anyway, I hope that they *will* support this feature, but this is not enough right now. So, instead of delegating waiting job to the client side, we need to organize waiting on the server. For this purpose I have created small separate application server with it's *own connection pool*, and delegated waiting task to him (see the snippet). This server needs one URL (/retry_after in the example) looking to the view above, installed in the url.py, and additional DEFAULT_RETRY_TIMEOUT parameter in the settings.py. The view uses request GET parameters, so Django will not try to cache it's results. The only task for this server is to *suspend* redirected requests until requested time is *really arrived*, and then redirect the request to the original URL. To make possible to process many requests simultaneously, it breaks waiting and redirects too long time requests to the self basing on the default timeout. Using this server is simple. We will instantiate the 'retry' server on the separate URL as a *separate application server* and will redirect all requests required to be delayed to this server, passing original URL and the time moment when the requested content should be ready, as parameters. The main server which prepares and returns the content, is *never waiting*. It returns prepared content from the cache, if it is found there, or *starts* content preparation process in *asynchronous* manner using threads, celery package, or any other appropriate technique, and *immediately redirects* the request to the 'retry' server The following is a fragment of the code using 'retry' server: ... # Create your views here. def jams(request,z,x,y): jams_timeout = settings.JAMS_TIMEOUT retry_after_server = settings.RETRY_AFTER_SERVER try: jams_ts,jams_png = request_jams(z,x,y) except Http404,ex: return HttpResponseNotFound() if not jams_png: if settings.DEBUG: logger.debug("JAMS NOT YET READY FOR: %s" % str((z,x,y))) # Tell a client to retrieve jams later time_after = time.time() + settings.DEFAULT_RETRY_TIMEOUT if settings.DEBUG: logger.debug("SEND RETRY AFTER FOR: %s" % str((z,x,y))) loc = reverse(jams,args=(z,x,y)) if request.get_host(): loc = request.build_absolute_uri(loc) r = HttpResponseRedirect( retry_after_server + '?' + urlencode({'redirect': loc,'retry_after':int(time_after)}) ) r['Retry-After'] = settings.DEFAULT_RETRY_TIMEOUT return r # return a png got from the cache if settings.DEBUG: logger.debug("RETURNING JAMS FOR: %s" % str((z,x,y,http_date(jams_ts),http_date(time.time())))) r = HttpResponse(jams_png) r['Content-Type'] = "image/png" r['Content-Length'] = len(jams_png) r['Content-Location'] = request.path # extra data for cache negotiation # Common and ConditionalGet Django Middleware will do the rest r['ETag'] = '"%s"' % md5_constructor(jams_png).hexdigest() r['Last-Modified'] = http_date(jams_ts) r['Expires'] = http_date(jams_ts + jams_timeout) return r ... At the application level, the request_jams() call tries to get content from the cache, and starts content update process asynchronously if required. It returns content got from the cache, and timestamp when the content has been created last time, or pair of None,None values, if the requested content is not available immediately. The line starting from 'r = HttpResponseRedirect' redirects the request to the 'retry' server, if the content is not available immediately. I am using nginx as a 'front-end' proxy server which passes requests to two separate fastcgi daemons - 'retry' and main django application servers separating requests by prefix. The following is a fragment of the nginx.conf configuration file for the nginx server: server { ... location /jams { fastcgi_pass unix:/var/tmp/jams.sock; ... } location /retry_after { fastcgi_pass unix:/var/tmp/retry.sock; ... } ... } As you can see, two locations are passed to *different* application server instances in the production environment, so they *don't block each other*. Really they both are implemented inside one django project, and started from the same directory as fastcgi daemon instances with different starting parameters (sockets to listen, log files etc.). I've used such structure to configure and debug my server easy, either just starting one developer server for both locations, or to have a possibility to start production server in described 'split' mode on the other hand. You can see that the both servers (jams and retry) use the same DEFAULT_RETRY_TIMEOUT setting from the settings.py file. The DEFAULT_RETRY_TIMEOUT plays role of load balancing parameter for requests which wait finishing asynchronous job. After the retry timeout is passed, the control returns to the client by the redirection response (or, for smart clients, the client itself waits the retry timeout before requesting the resource again), so the server may cleanup resources temporary acquired by these requests. Additional bonus of using such technique is stability of server processor load. Even in prime time, the server load is stable and controlled: number of the application instances and number of threads in the thread pool (evaluating asynchronous jobs) for each instance is fixed by configuration, nginx load per request is minimal, and concurrent requests are balanced (between server, network, and clients) by the retry timeout and redirect responses. You can see the result of the work on the our site, f.e. [on this page](http://www.doroga.tv/nnov/apps/map/), where all (transparent version) jam tiles are painting such a manner. The only difference for the production server environment is that some top zooms of most popular regions are refreshed by the special daemon (also written using django as a 'management command') which forces tiles to be refreshed in the cache even no one client requests it right now — to decrease response time for the jams map after long 'sleep' period.

  • cache
  • load
  • redirect
  • nginx
  • content
  • mass
Read More

tag to store a settings value as template variable

Get any value from settings.py as a template variable. The variable can then be used in conditional tags. E.g. to show a link to a help page only if it the help page url is defined in settings.py {% load get_setting %} {% get_setting MY_HELP_URL as help_url %} {% if help_url %}<a href="{% help_url|safe %}">Help</a>{% endif %}

  • template
  • tag
  • templatetag
  • settings
Read More

Method Caching

A very simple decorator that caches both on-class and in memcached: @method_cache(3600) def some_intensive_method(self): return # do intensive stuff` Alternatively, if you just want to keep it per request and forgo memcaching, just do: @method_cache() def some_intensive_method(self): return # do intensive stuff`

  • memcache
  • cache
  • decorator
  • memcached
  • decorators
  • caching
Read More

Read only form & model field

These decorators can be used to make some model/form fields readonly. **Sample usage:** # Use this decorator on form with readonly fields.` @modelform_with_readonly_fields` class FooAdminForm(forms.ModelForm):` ... # This decorator shoud be used to protect selected fields ` # from modification after initial save.` @has_readonly_fields` class Foo(models.Model):` read_only_fields = ('name', )` ... **Result** will be the same as shown in this post: [Readonly field](http://stackoverflow.com/questions/324477/in-a-django-form-how-to-make-a-field-readonly-or-disabled-so-that-it-cannot-be/1424453#1424453) and [Readonly model field](http://www.djangozen.com/blog/read-only-fields-in-models)

  • forms
  • model
  • field
  • readonly
Read More

Loading templates by app path

Loading templates by path app.template_name -> app/templates/template_name.html app.subdir.template_name -> app/templates/subdir/template_name.html Usage: append in settings.TEMPLATE_LOADERS and using render_to('app.index', context)

  • template
  • loader
Read More

2955 snippets posted so far.