Snippets by sardarnl

Lightweight querysets

Suppose you have two models A and B. The models have lots of large text/json fields and other heavy data. Your view is showing some list of `B.select_related(A)`, however, it is using just few lightweight fields. You want to defer all heavyweight fields in order to offload your DB and limit its traffic, but it is a tedious error prone work, that smells a bit with over-optimization. Real example, you have Product and Category objects. Both have large description fields with lots of text data. However, these fields are only needed on product detail page or on category detail page. In other cases (eg. side menu, some product suggestion block etc) you just need few lightweight things. Solution: add `LightweightMixin` to your models manager and specify your `heavyweight_fields` and `always_related_models`. # all visible products with necessary relations prefetched and heavyweight # fields deferred. qs = Product.objects.lightweight() # custom queryset with default defers applied. description and ingredients fields will # be normally deferred, but in this case they are explicitly selected qs = Product.objects.filter(my_filter="that", makes="sense") qs = Product.objects.as_lightweight(qs, select=('description', 'ingredients')) The best way to work with this snippet is to add the mixin to all managers in order to use `lightweight()` and `public()` calls. The `as_public()` is intended for your visibility, `select_related()` and `batch_select()` queryset settings. When you need full blown object, use `public()` queryset. When you need a lightweight list of objects, use `lightweight()` queryset. When your application is completed, check the database querylog for frequent unnecessary large and ugly queries. Group all queries that were made by `lightweight()` querysets, make a list of unnecessary heavy fields and add them to manager's `heavyweight_fields`. If your `as_public()` uses `select_related()` joining heavy objects, then you can also specify `always_related_models` to defer some fields of these relations too. Why? Because database IO will become your major bottleneck someday, just because you fetch 2Mb of data in order to render some tiny menu. With proper caching this is not a major issue, but proper queries may be sign of proper coding.