Login

Tag "database"

Snippet List

Transparent encryption for model fields

Here's a simple way to transparently encrypt a field (just repeat the idiom for more than one) so that it is not stored plaintext in the database. This might be useful for social security numbers, etc. The storage size for the ciphertext depends on which algorithm you use. Blowfish here requires 32 characters to store an encrypted 16 characters. Note also that Blowfish requires a block size of a multiple of 8, so that is what the repeat in the _set_ssn() is all about. The Crypto module is from http://www.amk.ca/python/code/crypto.html I make no claims as to how secure this scheme is overall, comments on this are welcome.

  • database
  • encryption
Read More

MySQL "Text" Type Model Field

Custom field for using MySQL's `text` type. `text` is more compact than the `longtext` field that Django assigns for `models.TextField` (2^16 vs. 2^32, respectively)

  • text
  • models
  • mysql
  • db
  • database
  • field
  • custom-field
Read More

dynamic model graph

This view assumes you have downloaded [modelviz.py](http://django-command-extensions.googlecode.com/svn/trunk/extensions/management/modelviz.py) and placed it in a python module called utils within my_project. You also must have graphviz installed and a subprocess-capable python. From there you can feed it a URL query list of the options you want to pass to generate_dot, and it will dynamically draw png or svg images of your model relationships right in the browser. All it wants is a nice form template for graphically selecting models. Most of this code and the main idea thereof was shamelessly plagiarized from [someone else](http://gundy.org/). Examples: `http://localhost:8000/model_graph/?image_type=svg&app_labels=my_app&include_models=Polls&include_models=Choices` `http://localhost:8000/model_graph/?image_type=png&all_applications&disable_fields&group_models`

  • models
  • graph
  • png
  • database
  • graphviz
  • modelviz
  • visualization
  • svg
Read More

JSON-compatible query filter specification

This function is designed to make it easier to specify client-side query filtering options using JSON. Django has a great set of query operators as part of its database API. However, there's no way I know of to specify them in a way that's serializable, which means they can't be created on the client side or stored. `build_query_filter_from_spec()` is a function that solves this problem by describing query filters using a vaguely LISP-like syntax. Query filters consist of lists with the filter operator name first, and arguments following. Complicated query filters can be composed by nesting descriptions. Read the doc string for more information. To use this function in an AJAX application, construct a filter description in JavaScript on the client, serialize it to JSON, and send it over the wire using POST. On the server side, do something like: > `from django.utils import simplejson` > `filterString = request.POST.get('filter', '[]')` > `filterSpec = simplejson.loads(filterString)` > `q = build_query_filter_from_spec(filterSpec)` > `result = Thing.objects.filter(q)` You could also use this technique to serialize/marshall a query and store it in a database.

  • filter
  • ajax
  • json
  • database
  • query
Read More

Export Database and Media_Root via Admin Interface

This is the view-code to export a database-dump (hardcoded for mysql) or a media_root dump via the admin interface. just add 2 urls patterns to call the views and a small template with a simple form to send a http-post to the views. Note: The downloads are sort of streaming. I have successfully exportet a 2GB media_root as tar-ball without major increase of ram-usage of the django-process.

  • admin
  • export
  • database
  • media-root
Read More

freshdb management command

This is useful especially during the model-creation stage, when things are in constant flux. The `freshdb` command will drop the project's database, then create a new one. A common use case: manage.py freshdb manage.py syncdb

  • db
  • database
  • management
Read More

Pickled Object Field

*Incredibly useful for storing just about anything in the database (provided it is Pickle-able, of course) when there isn't a 'proper' field for the job:* A field which can store any pickleable object in the database. It is database-agnostic, and should work with any database backend you can throw at it. Pass in any object and it will be automagically converted behind the scenes, and you never have to manually pickle or unpickle anything. Also works fine when querying. **Please note that this is supposed to be two files, one fields.py and one tests.py (if you don't care about the unit tests, just use fields.py)**

  • model
  • db
  • orm
  • database
  • pickle
  • object
  • field
  • pickled
Read More

Yet another SQL debugging facility

Inspired by http://www.djangosnippets.org/snippets/159/ This context processor provides a new variable {{ sqldebug }}, which can be used as follows: {% if sqldebug %}...{% endif %} {% if sqldebug.enabled %}...{% endif %} This checks settings.SQL_DEBUG and settings.DEBUG. Both need to be True, otherwise the above will evaluate to False and sql debugging is considered to be disabled. {{ sqldebug }} This prints basic information like total number of queries and total time. {{ sqldebug.time }}, {{ sqldebug.queries.count }} Both pieces of data can be accessed manually as well. {{ sqldebug.queries }} Lists all queries as LI elements. {% for q in sqldebug.queries %} <li>{{ q.time }}: {{ q }}</li> {% endfor %} Queries can be iterated as well. The query is automatically escaped and contains <wbr> tags to improve display of long queries. You can use {{ q.sql }} to access the unmodified, raw query string. Here's a more complex example. It the snippet from: http://www.djangosnippets.org/snippets/93/ adjusted for this context processor. {% if sqldebug %} <div id="debug"> <p> {{ sqldebug.queries.count }} Quer{{ sqldebug.queries|pluralize:"y,ies" }}, {{ sqldebug.time }} seconds {% ifnotequal sql_queries|length 0 %} (<span style="cursor: pointer;" onclick="var s=document.getElementById('debugQueryTable').style;s.display=s.display=='none'?'':'none';this.innerHTML=this.innerHTML=='Show'?'Hide':'Show';">Show</span>) {% endifnotequal %} </p> <table id="debugQueryTable" style="display: none;"> <col width="1"></col> <col></col> <col width="1"></col> <thead> <tr> <th scope="col">#</th> <th scope="col">SQL</th> <th scope="col">Time</th> </tr> </thead> <tbody> {% for query in sqldebug.queries %}<tr class="{% cycle odd,even %}"> <td>{{ forloop.counter }}</td> <td>{{ query }}</td> <td>{{ query.time }}</td> </tr>{% endfor %} </tbody> </table> </div> {% endif %}

  • sql
  • debug
  • queries
  • db
  • database
  • contextprocessor
Read More

Db Mock

I hate when my unittest hits database. Especially when each test case needs different dataset. So I wrote this db mock, that's local to specific test and uses sqlite3 in-memory db. Usage (nosetests): class TestMainNoData(DbMock): 'testing main function with no meaningful data' def test_no_posts(self): 'there are no posts' assert models.Post.objects.count() == 0, 'should be no feeds'

  • testing
  • unittest
  • database
  • test
Read More

Pull ID from arbitrary sequence

Django does not currently allow one to pull ID values from arbitrarily named sequences. For example, if you did not create your ID column using the serial data type in PostgreSQL, you likely will not be able to use your sequences. This is quite a problem for those integrating with legacy databases. While ultimately the best place to fix this is django proper, this decorator will help people get by for now. Note that in this case, all of my sequences are named "pk_TABLENAME". You'll likely have a different convention and should update the decorator appropriately. While I could have made the pattern a parameter, it didn't seem like that would gain much here.

  • database
  • sequence
  • postgresql
Read More

autocompleter with database query

This is an improvement of snippet 253 in that it supports database queries. Implementing autocompletion for foreign keys takes a few steps: 1) Put the snippet above into <app>/widgets/autocomplete.py. 2) Create a view of your foreign key model (here: Donator) in <app>/donator/views.py: from models import Donator from widgets.autocomplete import autocomplete_response def autocomplete(request): return autocomplete_response( request.REQUEST['text'], Donator, ( 'line_1', 'line_2', 'line_3', 'line_4', 'line_5', 'line_6', 'line_7', 'line_8', '^zip_code', 'location' ) ) This view returns the autocompletion result by searching the fields in the tuple. Each word from the form field must appear at least in one database field. 3) Create a URLconf that points to this new view. 4) In the form where you need the autocompletion, define the widget of the foreign key field as an instance of AutoCompleteField: from widget.autocomplete import AutoCompleteField field.widget = AutoCompleteField( url='/donator/autocomplete/'), options={'minChars': 3} ) The url parameter is the URL connected to the view in step 3), the options dict is passed on to the Ajax.Autocompleter JavaScript object. Links: * [Snippet 253](http://www.djangosnippets.org/snippets/253/) * [Django and scriptaculous integration](http://wiki.script.aculo.us/scriptaculous/show/IntegrationWithDjango) * [Ajax.Autocompleter](http://wiki.script.aculo.us/scriptaculous/show/Ajax.Autocompleter)

  • ajax
  • selection
  • database
  • autocomplete
  • query
  • foreign-key
  • prototype
  • scriptaculous
  • protoculous
Read More

Ordered items in the database - alternative

Every now and then you need to have items in your database which have a specific order. As SQL does not save rows in any order, you need to take care about this for yourself. No - actually, you don't need to anymore. You can just use this file - it is designed as kind-of plug-in for the Django ORM. Usage is (due to use of meta-classes) quite simple. It is recommended to save this snippet into a separate file called `positional.py`. To use it, you only have to import `PositionalSortMixIn` from the `positional` module and inherit from it in your own, custom model (but *before* you inherit from `models.Model`, the order counts). Usage example: Add this to your `models.py` from positional import PositionalSortMixIn class MyModel(PositionalSortMixIn, models.Model): name = models.CharField(maxlength=200, unique=True) Now you need to create the database tables: `PositionalSortMixIn` will automatically add a `postition` field to your model. In your views you can use it simply with `MyModel.objects.all().order_by('position')` and you get the objects sorted by their position. Of course you can move the objects down and up, by using `move_up()`, `move_down()` etc. In case you feel you have seen this code somewhere - right, this snippet is a modified version of [snippet #245](/snippets/245/) which I made earlier. It is basically the same code but uses another approach to display the data in an ordered way. Instead of overriding the `Manager` it adds the `position` field to `Meta.ordering`. Of course, all of this is done automatically, you only need to use `YourItem.objects.all()` to get the items in an ordered way. Update: Now you can call your custom managers `object` as long as the default manager (the one that is defined first) still returns all objects. This Mix-in absolutely needs to be able to access all elements saved. In case you find any errors just write a comment, updated versions are published here from time to time as new bugs are found and fixed.

  • db
  • orm
  • database
  • plugin
  • mixin
Read More

47 snippets posted so far.