Login

JSON-compatible query filter specification

Author:
mhalle
Posted:
March 30, 2008
Language:
Python
Version:
.96
Score:
2 (after 2 ratings)

This function is designed to make it easier to specify client-side query filtering options using JSON. Django has a great set of query operators as part of its database API. However, there's no way I know of to specify them in a way that's serializable, which means they can't be created on the client side or stored.

build_query_filter_from_spec() is a function that solves this problem by describing query filters using a vaguely LISP-like syntax. Query filters consist of lists with the filter operator name first, and arguments following. Complicated query filters can be composed by nesting descriptions. Read the doc string for more information.

To use this function in an AJAX application, construct a filter description in JavaScript on the client, serialize it to JSON, and send it over the wire using POST. On the server side, do something like:

from django.utils import simplejson

filterString = request.POST.get('filter', '[]')

filterSpec = simplejson.loads(filterString)

q = build_query_filter_from_spec(filterSpec)

result = Thing.objects.filter(q)

You could also use this technique to serialize/marshall a query and store it in a database.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
from django.db.models.query import Q, QNot, QAnd, QOr, LOOKUP_SEPARATOR

def build_query_filter_from_spec(spec, field_mapping=None):
    """ 
    Assemble a django "Q" query filter object from a specification that consists 
    of a possibly-nested list of query filter descriptions.  These descriptions
    themselves specify Django primitive query filters, along with boolean 
    "and", "or", and "not" operators.  This format can be serialized and 
    deserialized, allowing django queries to be composed client-side and
    sent across the wire using JSON.
    
    Each filter description is a list.  The first element of the list is always 
    the filter operator name. This name is one of either django's filter 
    operators, "eq" (a synonym for "exact"), or the boolean operators
    "and", "or", and "not".

    Primitive query filters have three elements:

    [filteroperator, fieldname, queryarg]
    
    "filteroperator" is a string name like "in", "range", "icontains", etc. 
    "fieldname" is the django field being queried.  Any name that django
    accepts is allowed, including references to fields in foreign keys
    using the "__" syntax described in the django API reference. 
    "queryarg" is the argument you'd pass to the `filter()` method in
    the Django database API.

    "and" and "or" query filters are lists that begin with the appropriate 
    operator name, and include subfilters as additional list elements:

    ['or', [subfilter], ...]
    ['and', [subfilter], ...] 

    "not" query filters consist of exactly two elements:

    ['not', [subfilter]]

    As a special case, the empty list "[]" or None return all elements.

    If field_mapping is specified, the field name provided in the spec
    is looked up in the field_mapping dictionary.  If there's a match,
    the result is subsitituted. Otherwise, the field name is used unchanged
    to form the query. This feature allows client-side programs to use
    "nice" names that can be mapped to more complex django names. If
    you decide to use this feature, you'll probably want to do a similar
    mapping on the field names being returned to the client.

    This function returns a Q object that can be used anywhere you'd like
    in the django query machinery.

    This function raises ValueError in case the query is malformed, or
    perhaps other errors from the underlying DB code.

    Example queries:

    ['and', ['contains', 'name', 'Django'], ['range', 'apps', [1, 4]]]
    ['not', ['in', 'tags', ['colors', 'shapes', 'animals']]]
    ['or', ['eq', 'id', 2], ['icontains', 'city', 'Boston']]

    """
    if spec == None or len(spec) == 0:
        return Q()
    cmd = spec[0]

    if cmd == 'and' or cmd == 'or':
        # ["or",  [filter],[filter],[filter],...]
        # ["and", [filter],[filter],[filter],...]
        if len(spec) < 2:
            raise ValueError,'"and" or "or" filters must have at least one subfilter'

        if cmd == 'and':
            qop = QAnd
        else:
            qop = QOr

        resultq = None
        for arg in spec[1:]:
            q = build_query_filter_from_spec(arg)                
            if q != None:
                if resultq == None:
                    resultq = q
                else:
                    resultq = qop(resultq, q)

    elif cmd == 'not':
        # ["not", [query]]
        if len(spec) != 2:
            raise ValueError,'"not" filters must have exactly one subfilter'
        q = build_query_filter_from_spec(spec[1])
        if q != None:
            resultq = QNot(q)

    else:
        # some other query, will be validated in the query machinery
        # ["cmd", "fieldname", "arg"]

        # provide an intuitive alias for exact field equality
        if cmd == 'eq':
            cmd = 'exact'

        if len(spec) != 3:
            raise ValueError,'primitive filters must have two arguments (fieldname and query arg)'

        field_name = spec[1]
        if field_mapping:
            # see if the mapping contains an entry for the field_name
            # (for example, if you're mapping an external database name
            # to an internal django one).  If not, use the existing name.
            field_name = field_mapping.get(field_name, field_name)

        kwname = str("%s%s%s" % (field_name, LOOKUP_SEPARATOR, cmd))
        kwdict = {kwname : spec[2]}
        resultq =  Q(**kwdict)

    return resultq

More like this

  1. Template tag - list punctuation for a list of items by shapiromatron 11 months, 2 weeks ago
  2. JSONRequestMiddleware adds a .json() method to your HttpRequests by cdcarter 11 months, 3 weeks ago
  3. Serializer factory with Django Rest Framework by julio 1 year, 6 months ago
  4. Image compression before saving the new model / work with JPG, PNG by Schleidens 1 year, 7 months ago
  5. Help text hyperlinks by sa2812 1 year, 8 months ago

Comments

Please login first before commenting.