Login

Chain multiple querysets into one

Author:
mattdw
Posted:
October 1, 2008
Language:
Python
Version:
1.0
Score:
2 (after 2 ratings)

This class acts as a wrapper around multiple querysets. Use it if you want to chain multiple QSs together without combining them with | or &. eg., to put title matches ahead of body matches:

>>> qs1 = Event.objects.filter(## title matches ##)
>>> qs2 = Event.objects.filter(## matches in other fields ##)
>>> qs = MultiQuerySet(qs1, qs2)
>>> len(qs)
>>> paginator = Paginator(qs)
>>> first_ten = qs[:10]

It effectively acts as an immutable, sliceable QuerySet (with only a very limited subset of the QuerySet api)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
class MultiQuerySet(object):
    def __init__(self, *args, **kwargs):
        self.querysets = args
        self._count = None
    
    def count(self):
        if not self._count:
            self._count = sum(len(qs) for qs in self.querysets)
        return self._count
    
    def __len__(self):
        return self.count()
        
    def __getitem__(self, item):
        indices = (offset, stop, step) = item.indices(self.count())
        items = []
        total_len = stop - offset
        for qs in self.querysets:
            if len(qs) < offset:
                offset -= len(qs)
            else:
                items += list(qs[offset:stop])
                if len(items) >= total_len:
                    return items
                else:
                    offset = 0
                    stop = total_len - len(items)
                    continue

More like this

  1. Template tag - list punctuation for a list of items by shapiromatron 11 months, 2 weeks ago
  2. JSONRequestMiddleware adds a .json() method to your HttpRequests by cdcarter 11 months, 3 weeks ago
  3. Serializer factory with Django Rest Framework by julio 1 year, 6 months ago
  4. Image compression before saving the new model / work with JPG, PNG by Schleidens 1 year, 7 months ago
  5. Help text hyperlinks by sa2812 1 year, 8 months ago

Comments

ericflo (on October 1, 2008):

qs = Event.objects.filter(## title matches ##) | Event.objects.filter(## matches in other fields ##)

#

ericflo (on October 1, 2008):

just kidding (wish there was a comment delete)

#

mattdw (on October 1, 2008):

Heh... thanks, ericflo. Fair suggestion, and qs = qs1 | qs2 will probably cover 90% of cases. This is for the other 10% ;-).

#

dan90 (on October 2, 2008):

Hm. If you don't need full query set functionality, you might want to consider itertools, which has the nice added bonus you don't even need to use query_sets for the same model. e.g.

from itertools import chain
chained_qs = chain(Event.objects.all(), BlogPost.objects.all())
for record in chain_qs:
    print record.slug

(assuming BlogPosts and Events both have a field called 'slug')

that also gets you your original order query set as a special case, and it's very clean.

#

mattdw (on October 2, 2008):

Mine doesn't require querysets to be from the same model either. I looked at itertools.chain, but it doesn't support enough QuerySet functionality to work with django's pagination module, pagination being one of the requirements of this solution. (In particular, there's no way to a. find the length of a chained iterator, or b. slice it arbitrarily. This class supports both of those.)

#

carljm (on October 3, 2008):

Very handy! I've had this exact need before, always worked around it. Nicely done!

#

vbabiy (on March 9, 2009):

Is is exactly what I been looking for. Thanks

#

Please login first before commenting.