Login

Tag "database"

Snippet List

Friendly ID(Python 3.X)

This is just modified version of [friendly id](https://djangosnippets.org/snippets/1249/) for make this script compatible with python 3.x Invoice numbers like "0000004" are a little unprofessional in that they expose how many sales a system has made, and can be used to monitor the rate of sales over a given time. They are also harder for customers to read back to you, especially if they are 10 digits long. This is simply a perfect hash function to convert an integer (from eg an ID AutoField) to a unique number. The ID is then made shorter and more user-friendly by converting to a string of letters and numbers that wont be confused for one another (in speech or text). To use it: import friendly_id class MyModel(models.Model): invoice_id = models.CharField(max_length=6, null=True, blank=True, unique=True) def save(self, *args, **kwargs): super(MyModel, self).save(*args, **kwargs) # Populate the invoice_id if it is missing if self.id and not self.invoice_id: self.invoice_id = friendly_id.encode(self.id) self.save() if self.id and not self.invoice_id When an object from this model is saved, an invoice ID will be generated that does not resemble those surrounding it. For example, where you are expecting millions of invoices the IDs generated from the AutoField primary key will be: obj.id obj.invoice_id 1 TTH9R 2 45FLU 3 6ACXD 4 8G98W 5 AQ6HF 6 DV3TY ... 9999999 J8UE5 The functions are deterministic, so running it again sometime will give the same result, and generated strings are unique for the given range (the default max is 10,000,000). Specifying a higher range allows you to have more IDs, but all the strings will then be longer. You have to decide which you need: short strings or many strings :-) This problem could have also been solved using a random invoice_id generator, but that might cause collisions which cost time to rectify, especially when a decent proportion of the available values are taken (eg 10%). Anyhow, someone else has now already written this little module for you, so now you don't have to write your own :-)

  • database
  • field-id
  • invoice-id
  • invoice
Read More

Django chunked queryset iterator

The function slices a queryset into smaller querysets containing chunk_size objects and then yield them. It is used to avoid memory error when processing huge queryset, and also database error due to that the database pulls whole table at once. Concurrent database modification wouldn't make some entries repeated or skipped in this process.

  • django
  • python
  • database
  • queryset
  • iterator
  • memoryerror
Read More

Testing for pending migrations in Django

DB migration support has been added in Django 1.7+, superseding South. More specifically, it's possible to automatically generate migrations steps when one or more changes in the application models are detected. Definitely a nice feature! I've written a small generic unit-test that one should be able to drop into the tests directory of any Django project and that checks there's no pending migrations, ie. if the models are correctly in sync with the migrations declared in the application. Handy to check nobody has forgotten to git add the migration file or that an innocent looking change in models.py doesn't need a migration step generated. Enjoy!

  • testing
  • unittest
  • database
  • migration
Read More

Multi-DB Reconnecting Persistent Postgres Connection

This is a modification of http://djangosnippets.org/snippets/1707/ that handles the database going down or PG Bouncer killing the connection. This also works in things like Twisted to make sure the connection is alive before doing a real query. Thanks @mike_tk for the original post! EDIT: Updated the wrapper to handle multi-db. Before it was using the first connection it made, now it creates an attribute name for the connection based on the name of the database.

  • database
  • multi-db
  • twisted
  • connection
  • persistent
  • multiple-databases
  • socket
  • web-socket
Read More

Database backup with admin command

Detect type of database (MySQL, PostgreSQL or SQLite) and make backup. In this moment ONLY WORK in GNU/Linux, NOT WIN.

  • database
  • admin-actions
  • backup
  • MySQL
  • admin-command
  • SQLite
  • PostgreSQL
Read More
Author: jhg
  • 1
  • 3

Validation for full e-mails (e.g. "Joe Hacker <[email protected]>")

Out of the box, Django e-mail fields for both database models and forms only accept plain e-mail addresses. For example, `[email protected]` is accepted. On the other hand, full e-mail addresses which include a human-readable name, for example the following address fails validation in Django: Joe Hacker <[email protected]> This package adds support for validating full e-mail addresses. **Database model example** from django import models from full_email.models import FullEmailField class MyModel(models.Model): email = FullEmailField() **Forms example** from django import forms from full_email.formfields import FullEmailField class MyForm(forms.Form): email = FullEmailField(label='E-mail address') I maintain this code in a [GitHub gist](https://gist.github.com/1505228). It includes some unit tests as well.

  • forms
  • model
  • email
  • validation
  • orm
  • database
Read More

ByteSplitterField

When you want to save integers to the db, you usually have the choice between 16-, 32- and 64-bit Integers (also 8- and 24-bit for MySQL). If that doesn't fit your needs and you want to use your db-memory more efficient, this field might be handy to you. Imagine you have 3 numbers, but need only 10 bit to encode each (i.e. from 0 to 1000). Instead of creating 3 smallint-fields (48 bit), you can create one 'ByteSplitterField' which implements 3 'subfields' and automatically encodes them inside a 32 bit integer. You don't have to take care how each 10-bit chunk is encoded into the 32-bit integer, it's all handled by the field (see also field's description). Additionally, the Field offers opportunity to use decimal_places for each of your subfields. These are 'binary decimal places', meaning the integer-content is automatically divided by 2, 4, 8, etc. when you fetch the value from the field. You can also specify how values are rounded ('round' parameter) and what happens when you try to save a value out of range ('overflow' parameter) Not implemented (maybe in the future if I should need it sometime): * signed values. All values are positive right now! * real (10-based) decimal places (actually you could probably directly use DecimalFields here) * further space optimization, i.e. saving into CharField that's length can be chosen byte-wise

  • model
  • db
  • database
  • field
  • custom
  • custom-model-field
  • IntegerField
  • multibit-field
  • model-field
Read More

Decoupling models with cross-database relations

The snippet enables decoupling model classes, associated with a ForeignKey, for the purpose of separating them into two databases. Looking at the following example: class Reporter(models.Model): ... class Article(models.Model): reporter = models.ForeignKey(Reporter) We want to separate the `Reporter` and `Article` into two separate databases, but this won't work in Django because of the [cross model relations](http://docs.djangoproject.com/en/dev/topics/db/multi-db/#cross-database-relations). The solution is to use an ID field and not a `ForeignKey` one. This makes the access very uncomfortable. The above class will make this a bit less awkward. It doesn't support the [RelatedManager](http://docs.djangoproject.com/en/dev/ref/models/relations/#django.db.models.fields.related.RelatedManager), but it will support accessing the related field as you normally would. The article class will look like this (assuming the reporter model id field is an `IntegerField`): class Article(DecoupledModel): reporter_id = models.IntegerField() _linked_fields = { 'reporter': Reporter, } Once you have an article object, you can access the reporter as you normally would for both read and writing. For example: my_reporter = article1.reporter article2.reporter = my_reporter

  • django
  • model
  • database
Read More

Improved Pickled Object Field (Fixed for Django 1.2)

Small changes to [Snippet 1694](http://djangosnippets.org/snippets/1694/) to that QueryAPI works for django 1.2 and higher. Changes: * Replaced `get_db_prep_value` with `get_prep_value`. * Replaced `get_db_prep_lookup` with modified `get_prep_lookup`.

  • model
  • db
  • orm
  • database
  • pickle
  • object
  • field
  • type
  • pickled
  • store
Read More

JsonObjectField

This fields.py file defines a new model field type, "JsonObjectField," which is designed to allow the storage of arbitrary Python objects in Django TextFields. It is intended primarily to allow the storage of Python dictionaries or list objects. As the name implies, it converts objects to JSON for storage; this conversion happens transparently, so from your model's perspective, the field stores and retrieves the actual objects.

  • model
  • json
  • database
  • object
Read More

Email queue in DB

This is what I use to send simple status emails from my sites. Instead of a django.core.mail.send_mail call, which can take an irrritatingly, nondeterministically long time to return (regardless of error state), you can stow the emails in the database and rely on a separate interpreter process send them off (using a per-minute cron job or what have you). You then also have a record of everything you've sent via email from your code without having to configure your outbound SMTP server to store them or anything like that. Usage notes are in the docstring; share and enjoy.

  • django
  • python
  • email
  • mail
  • database
  • queue
  • asynchronous
Read More

47 snippets posted so far.