Login

A slightly better YAML serializer

Author:
wapcaplet
Posted:
June 10, 2011
Language:
Python
Version:
1.3
Score:
1 (after 1 ratings)

I find Django's default YAML serializer output too repetitive, so I came up with this customized version that avoids outputting the model name for every single object, and makes the primary key into an index for the object's fields. This allows many simple model instances to be serialized as one-liners.

See the module docstring for additional explanation and usage notes.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
# better_yaml.py

"""
Customized YAML serializer, with more condensed and readable output.
Rather than producing a flat list of objects with the same three attributes:

    - fields: {...}
      model: modelname
      pk: 123

This serializer nests the data, grouping by model name, then indexing by
primary key. For example, instead of this output, as produced by the default
YAML serializer:

    - fields: {name: blue}
      model: app.firstmodel
      pk: 3
    - fields: {name: red}
      model: app.firstmodel
      pk: 1
    - fields: {name: green}
      model: app.firstmodel
      pk: 2
    - fields: {name: crumbly}
      model: app.secondmodel
      pk: 2
    - fields: {name: squishy}
      model: app.secondmodel
      pk: 1

You'll get this output:

    app.firstmodel:
      1: {name: red}
      2: {name: green}
      3: {name: blue}
    app.secondmodel:
      1: {name: squishy}
      2: {name: crumbly}

To use this customized serializer and deserializer, save this file
somewhere in your Django project, then add this to your settings.py:

    SERIALIZATION_MODULES = {
        'yaml': 'path.to.better_yaml',
    }

Note that this serializer is NOT compatible with the default Django
YAML serializer; this one uses nested dictionaries, while the default
one uses a flat list of object dicts.

Requires PyYaml (http://pyyaml.org/), of course.
"""

from StringIO import StringIO
import yaml

from django.core.serializers.pyyaml import Serializer as YamlSerializer
from django.core.serializers.python import Deserializer as PythonDeserializer
from django.utils.encoding import smart_unicode

class Serializer (YamlSerializer):
    """
    Serialize database objects as nested dicts, indexed first by
    model name, then by primary key.
    """
    def start_serialization(self):
        self._current = None
        self.objects = {}

    def end_object(self, obj):
        model = smart_unicode(obj._meta)
        pk = obj._get_pk_val()

        if model not in self.objects:
            self.objects[model] = {}

        self.objects[model][pk] = self._current
        self._current = None


def Deserializer(stream_or_string, **options):
    """
    Deserialize a stream or string of YAML data,
    as written by the Serializer above.
    """
    if isinstance(stream_or_string, basestring):
        stream = StringIO(stream_or_string)
    else:
        stream = stream_or_string

    # Reconstruct the flat object list as PythonDeserializer expects
    # NOTE: This could choke on large data sets, since it
    # constructs the flattened data list in memory
    data = []
    for model, objects in yaml.load(stream).iteritems():
        # Add the model name back into each object dict
        for pk, fields in objects.iteritems():
            data.append({'model': model, 'pk': pk, 'fields': fields})

    # Deserialize the flattened data
    for obj in PythonDeserializer(data, **options):
        yield obj

More like this

  1. Template tag - list punctuation for a list of items by shapiromatron 2 months, 2 weeks ago
  2. JSONRequestMiddleware adds a .json() method to your HttpRequests by cdcarter 2 months, 3 weeks ago
  3. Serializer factory with Django Rest Framework by julio 9 months, 2 weeks ago
  4. Image compression before saving the new model / work with JPG, PNG by Schleidens 10 months, 1 week ago
  5. Help text hyperlinks by sa2812 11 months ago

Comments

Please login first before commenting.