This utility is useful when you want to safely retrieve a single object from the database without having to wrap get() in a try/except block every time. It also supports query optimization via select_related and prefetch_related, making it ideal for performance-conscious applications.
This is a Python 3 re-write for https://djangosnippets.org/snippets/2117/
""" Takes arguments & constructs Qs for filter() We make sure we don't construct empty filters that would return too many results We return an empty dict if we have no filters so we can still return an empty response from the view """
The function slices a queryset into smaller querysets containing chunk_size objects and then yield them. It is used to avoid memory error when processing huge queryset, and also database error due to that the database pulls whole table at once. Concurrent database modification wouldn't make some entries repeated or skipped in this process.
I want to create Mixins for QuerySet objects that will by default filter out certain records (e.g. filter out "deleted" records, or filter out "unapproved" records, etc). I'd like those to be separate independent mixins. So in each of those, I override all() to filter out deleted or unapproved, etc. But, I also want to offer a method in the queryset to remove those filters or remove some part of those filters. That's where this code comes in. After some examination of how QuerySets work, this seemed like the simplest method for "undoing" some filter in a queryset
How to use =========== Save the snippet to a file utils.py, and add the following view to your Django app: from django.http import HttpResponse from .utils import queryset_to_workbook def download_workbook(request): queryset = User.objects.all() columns = ( 'first_name', 'last_name', 'email', 'is_staff', 'groups') workbook = queryset_to_workbook(queryset, columns) response = HttpResponse(mimetype='application/vnd.ms-excel') response['Content-Disposition'] = 'attachment; filename="export.xls"' workbook.save(response) return response Note: you can use dotted notation (`'foreign_key.foreign_key.field'`) in the columns parameter to access fields that are accessible through the objects returned by the queryset (in that case you probably want to use `select_related` with your queryset).
Suppose you have two models A and B. The models have lots of large text/json fields and other heavy data. Your view is showing some list of `B.select_related(A)`, however, it is using just few lightweight fields. You want to defer all heavyweight fields in order to offload your DB and limit its traffic, but it is a tedious error prone work, that smells a bit with over-optimization. Real example, you have Product and Category objects. Both have large description fields with lots of text data. However, these fields are only needed on product detail page or on category detail page. In other cases (eg. side menu, some product suggestion block etc) you just need few lightweight things. Solution: add `LightweightMixin` to your models manager and specify your `heavyweight_fields` and `always_related_models`. # all visible products with necessary relations prefetched and heavyweight # fields deferred. qs = Product.objects.lightweight() # custom queryset with default defers applied. description and ingredients fields will # be normally deferred, but in this case they are explicitly selected qs = Product.objects.filter(my_filter="that", makes="sense") qs = Product.objects.as_lightweight(qs, select=('description', 'ingredients')) The best way to work with this snippet is to add the mixin to all managers in order to use `lightweight()` and `public()` calls. The `as_public()` is intended for your visibility, `select_related()` and `batch_select()` queryset settings. When you need full blown object, use `public()` queryset. When you need a lightweight list of objects, use `lightweight()` queryset. When your application is completed, check the database querylog for frequent unnecessary large and ugly queries. Group all queries that were made by `lightweight()` querysets, make a list of unnecessary heavy fields and add them to manager's `heavyweight_fields`. If your `as_public()` uses `select_related()` joining heavy objects, then you can also specify `always_related_models` to defer some fields of these relations too. Why? Because database IO will become your major bottleneck someday, just because you fetch 2Mb of data in order to render some tiny menu. With proper caching this is not a major issue, but proper queries may be sign of proper coding.
This is a function to take a Q object and construct a function which returns a boolean. This lets you use the exact same filter syntax that Django's managers use and apply it inside list comprehensions, or to non-persistent objects, or to objects of different types with the same attribute names.
SELECT FOR UPDATE, which does row-level locking in the database, was added by Django only in version 1.4. This snippet emulates that feature in older versions of Django. Tested in Django 1.2, but should work in newer versions as well. select_related is removed because it causes errors when used with RawQuerySet.
I have a model with a datetime field that I used as a timestamp. I’m in California’s timezone (“America/Los_Angeles”). The data is saved in UTC in MySQL (as confirmed by the ORM). I just want to do a query that looks like this: “give me all the information with day X’s timestamp” (24 hour period). But the timestamp is a datetime, not date. If you just do varname.date(), it’s still UTC’s date, not your local timezone’s date. Here’s what I did: 1. First construct the start and end time period covering the 24 hour period of that day you want 2. Make it an “aware” (not naive) datetime 3. Filter for the __range
Django allows you to specify your own ModelManager with custom methods. However, these methods are chainable. That is, if you have a method on your PersonManager caled men(), you can't do this: Person.objects.filter(birth_date__year=1978).men() Normally, this isn't a problem, however your app may be written to take advantage of the chainability of querysets. For example, you may have an API method which may return a filtered queryset. You would want to call with_counts() on an already filtered queryset. In order to overcome this, we want to override django's QuerySet class, and then make the Manager use this custom class. The only downside is that your functions will not be implemented on the manager itself, so you'd have to call `Person.objects.all().men()` instead of `Person.objects.men()`. To get around this you must also implement the methods on the Manager, which in turn call the custom QuerySet method.
This snippet is greatly inspired by [@jlorich](http://djangosnippets.org/users/jlorich/)'s useful [#2436](http://djangosnippets.org/snippets/2436/). The main difference is that I wanted to choose the names of my URL params instead of being forced into naming them "value1", "value2", etc. When reversing the URL you have to remember that the kwargs aren't friendly. By using the same names in the `filters` list, you don't have to change the way your otherwise write the URL pattern. Also it's clear throughout how you'll be filtering the QuerySet. The other change I made was "erroring early". This avoids running the QuerySet all over again inside `object_detail()` just to have it raise an exception we could have caught the first time.
Dehydrates objects that can be dictionaries, lists or tuples containing django model objects or django querysets. For each of those, it creates a smaller/dehydrated version of it for saving in cache or pickling. The reverse operation is also provided so dehydrated objects can also be re-hydrated. *Example:* >>> import pickle >>> users = list(User.objects.all()[:20]) >>> print users [<User: Indiana Jones>, <User: Bilbo Baggins>, ...] >>> pickled_users = pickle.dumps(users) >>> print len(pickled_users) 17546 >>> dehydrated_users = dehydrate(users) >>> pickled_dehydrated_users = pickle.dumps(dehydrated_users) >>> rehydrated_users = hydrate(pickle.loads(pickled_dehydrated_users)) >>> print rehydrated_users [<User: Indiana Jones>, <User: Bilbo Baggins>, ...] >>> print len(pickled_dehydrated_users) 1471
This manager is based on the economical method of randomization. It usable in related models. For support this behavior required to be defined as default manger.
Because select_related() only works on ForeignKeys that are not null or blank, when you are customizing the admin, even if you set "list_select_related=True" you can still end up with way too many querys in the Admin changelist. By adding this code to your model you can decrease the queries dramatically. I found I needed this when I was working with Django apps with a lot of legacy data and I couldn't set the ForeignKey null=False.
39 snippets posted so far.