`dumpdata` without `MemoryErrors`, with progress notification. Most of the real work is done by snippets [1400](http://www.djangosnippets.org/snippets/1400/) and [1401](http://www.djangosnippets.org/snippets/1401/).
./manage.py dumpdata_stream --format=xml > big_dump.xml
This is basically the stock Django `dumpdata` with a few modifications. Django devs: it's hard to reuse parts of most Django management commands. A little refactoring could go a long way.
Trying `./manage.py dumpdata` on a huge database and getting `MemoryError`s? Here's part of your solution.
[Snippet 1400](http://www.djangosnippets.org/snippets/1400/) provides a queryset_foreach utility that we've found very useful. This snippet uses it on a serializer that can output to a stream, such as the XML serializer.
Management command coming momentarily...
Call a function for each element in a queryset (actually, any list).
Features:
* stable memory usage (thanks to Django paginators)
* progress indicators
* wraps batches in transactions
* can take managers or even models (e.g., `Assertion.objects`)
* warns about `DEBUG`.
* handles failures of single items without dying in general.
* stable even if items are added or removed during processing (gets a list of ids at the start)
Returns a `Status` object, with the following interesting attributes
* `total`: number of items in the queryset
* `num_successful`: count of successful items
* `failed_ids`: list of ids of items that failed
Allows you to include globs of IP addresses in your INTERNAL_IPS. It's shell-style glob syntax (the [fnmatch module](http://docs.python.org/library/fnmatch.html)).
This should be helpful with the [Debug Toolbar](http://github.com/robhudson/django-debug-toolbar/tree/master), among other things.
Like [#1362](http://www.djangosnippets.org/snippets/1362/), but with no external dependencies.
Another one like [980](http://www.djangosnippets.org/snippets/980/), but using re's instead.
I haven't benchmarked these, but my guess is that [981](http://www.djangosnippets.org/snippets/981/) is faster for strings of pure digits and this is faster for larger chunks of text that happen to contain digits. If you're generating the numbers yourself, I'd just use 981 on a number right when you generate it.
This is based on [980](http://www.djangosnippets.org/snippets/980/), removing the unnecessary use of StringIO. Hopefully the translation can be educational.
Whip up an AJAX API for your site in a jiffy:
class MySite(AJAXApi):
@AJAXApi.export
def hello(request):
return {'content': self.get_content()}
def get_content(self):
return 'Hello, world!'
urlpatterns += MySite().url_patterns()
(the example needs the JSON encoding middleware of [snippet 803](http://www.djangosnippets.org/snippets/803/) to work.)
The secret is that bound instance methods are callable too, so work as views. (Most Django people only use functions, or sometimes classes with `__call__`, as view functions.)
You get implicit type dispatch off that `self` object. So you could subclass `MySite`, change `get_content`, and still use the same `hello` method.
See (django-webapp)[http://code.google.com/p/django-webapp/] for a REST-ish Resource class using this same idea.
You can clearly do better than my `func_to_view`, and also make a better decorator than `exported` (so you can actually say `@exported('name') def function()` etc.). This is more of a proof of concept that should work for most people.
Caveat: I've changed a few things since I last really tested this.
(psst, this also works for non-AJAX views too.)
Handles exceptions from AJAX code, giving more useful errors and tracebacks.
(By the way, all these new snippets are extracts from [django-webapp](http://code.google.com/p/django-webapp/).)
This exact code is not well tested, but it's refactored from some code we use in production.
Another `JsonResponse` class, including comment wrapping. Extensions to other kinds of CSRF protection should be obvious. Good explanations of why such protections are needed would make excellent comments on this snippet.
This depends on the `json_encode` method in [snippet 800](http://www.djangosnippets.org/snippets/800/).
The Django JSON encoder already extends the `simplejson` encoder a little; this extends it more and gives an example of how to go about further extension.
Hopefully `newserializers` (see the community aggregator today) will supercede this, but until then, it's useful.
This is another example use of the [exception middleware](http://www.djangosnippets.org/snippets/638/). It shows how to log exceptions to a file. Someone wanted to do this to avoid DOS-ing the email server in case of a silly error.
(untested.)
This exception middleware abstracts the functionality of the builtin exception handling mechanisms, but makes them extensible by inheritance.
Just add it (or some subclass) to the top of your active middleware list.
You can use this to [make your admin emails more informative](http://www.djangosnippets.org/snippets/631/) or [log errors to a file](http://www.djangosnippets.org/snippets/639/).
This middleware makes the admin error emails a lot more informative: you get the same HTML response that you get with `DEBUG=True`.
It uses the base class defined in [#638](http://www.djangosnippets.org/snippets/638/).
You will probably want to apply the patch for [#6748](http://code.djangoproject.com/ticket/6748) to help avoid slowdowns caused by unintentional database queries. As the ticket (and django-developers thread) notes, it isn't foolproof; you may still find this executing database queries.
Use this decorator on a function that returns a dict to get a JSON view, with error handling.
Features:
* response always includes a 'result' attribute ('ok' by default)
* catches all errors and mails the admins
* always returns JSON even on errors