Login

Command to make fixtures.

Author:
buriy
Posted:
July 27, 2008
Language:
Python
Version:
.96
Score:
9 (after 9 ratings)

"Make fixture" command. Highly useful for making test fixtures. Use it to pick only few items from your data to serialize, restricted by primary keys. By default command also serializes foreign keys and m2m relations. You can turn off related items serialization with --skip-related option.

How to use:

python manage.py makefixture

will display what models are installed

python manage.py makefixture User[:3]

or

python manage.py makefixture auth.User[:3]

or

python manage.py makefixture django.contrib.auth.User[:3]

will serialize users with ids 1 and 2, with assigned groups, permissions and content types.

python manage.py makefixture YourModel[3] YourModel[6:10]

will serialize YourModel with key 3 and keys 6 to 9 inclusively.

Of course, you can serialize whole tables, and also different tables at once, and use options of dumpdata:

python manage.py makefixture --format=xml --indent=4 YourModel[3] AnotherModel auth.User[:5] auth.Group
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
#save into anyapp/management/commands/makefixture.py
#or back into django/core/management/commands/makefixture.py
#v0.1 -- current version
#known issues:
#no support for generic relations 
#no support for one-to-one relations
from optparse import make_option
from django.core import serializers
from django.core.management.base import BaseCommand
from django.core.management.base import CommandError
from django.core.management.base import LabelCommand
from django.db.models.fields.related import ForeignKey
from django.db.models.fields.related import ManyToManyField
from django.db.models.loading import get_models

DEBUG = False

def model_name(m):
    module = m.__module__.split('.')[:-1] # remove .models
    return ".".join(module + [m._meta.object_name])

class Command(LabelCommand):
    help = 'Output the contents of the database as a fixture of the given format.'
    args = 'modelname[pk] or modelname[id1:id2] repeated one or more times'
    option_list = BaseCommand.option_list + (
        make_option('--skip-related', default=True, action='store_false', dest='propagate',
            help='Specifies if we shall not add related objects.'),
        make_option('--format', default='json', dest='format',
            help='Specifies the output serialization format for fixtures.'),
        make_option('--indent', default=None, dest='indent', type='int',
            help='Specifies the indent level to use when pretty-printing output'),
    )
    
    def handle_models(self, models, **options):
        format = options.get('format','json')
        indent = options.get('indent',None)
        show_traceback = options.get('traceback', False)
        propagate = options.get('propagate', True)
        
        # Check that the serialization format exists; this is a shortcut to
        # avoid collating all the objects and _then_ failing.
        if format not in serializers.get_public_serializer_formats():
            raise CommandError("Unknown serialization format: %s" % format)

        try:
            serializers.get_serializer(format)
        except KeyError:
            raise CommandError("Unknown serialization format: %s" % format)

        objects = []
        for model, slice in models:
            if isinstance(slice, basestring):
                objects.extend(model._default_manager.filter(pk__exact=slice))
            elif not slice or type(slice) is list:
                items = model._default_manager.all()
                if slice and slice[0]:
                    items = items.filter(pk__gte=slice[0])
                if slice and slice[1]:
                    items = items.filter(pk__lt=slice[1])
                items = items.order_by(model._meta.pk.attname)
                objects.extend(items)
            else:
                raise CommandError("Wrong slice: %s" % slice)
        
        all = objects
        if propagate:
            collected = set([(x.__class__, x.pk) for x in all]) 
            while objects:
                related = []
                for x in objects:
                    if DEBUG:
                        print "Adding %s[%s]" % (model_name(x), x.pk)
                    for f in x.__class__._meta.fields + x.__class__._meta.many_to_many:
                        if isinstance(f, ForeignKey):
                            new = getattr(x, f.name) # instantiate object
                            if new and not (new.__class__, new.pk) in collected:
                                collected.add((new.__class__, new.pk))
                                related.append(new)
                        if isinstance(f, ManyToManyField):
                            for new in getattr(x, f.name).all():
                                if new and not (new.__class__, new.pk) in collected:
                                    collected.add((new.__class__, new.pk))
                                    related.append(new)
                objects = related
                all.extend(objects)
        
        try:
            return serializers.serialize(format, all, indent=indent)
        except Exception, e:
            if show_traceback:
                raise
            raise CommandError("Unable to serialize database: %s" % e)

    def get_models(self):
        return [(m, model_name(m)) for m in get_models()]

    def handle_label(self, labels, **options):
        parsed = []
        for label in labels:
            search, pks = label, ''
            if '[' in label:
                search, pks = label.split('[', 1)
            slice = ''
            if ':' in pks: 
                slice = pks.rstrip(']').split(':', 1)
            elif pks: 
                slice = pks.rstrip(']')
            models = [model for model, name in self.get_models() 
                            if name.endswith('.'+search) or name == search]
            if not models:
                raise CommandError("Wrong model: %s" % search)
            if len(models)>1:
                raise CommandError("Ambiguous model name: %s" % search)
            parsed.append((models[0], slice))
        return self.handle_models(parsed, **options)

    def list_models(self):
        names = [name for _model, name in self.get_models()]
        raise CommandError('Neither model name nor slice given. Installed model names: \n%s' % ",\n".join(names))

    def handle(self, *labels, **options):
        if not labels:
            self.list_models()

        output = []
        label_output = self.handle_label(labels, **options)
        if label_output:
                output.append(label_output)
        return '\n'.join(output)

More like this

  1. Template tag - list punctuation for a list of items by shapiromatron 11 months, 1 week ago
  2. JSONRequestMiddleware adds a .json() method to your HttpRequests by cdcarter 11 months, 2 weeks ago
  3. Serializer factory with Django Rest Framework by julio 1 year, 6 months ago
  4. Image compression before saving the new model / work with JPG, PNG by Schleidens 1 year, 7 months ago
  5. Help text hyperlinks by sa2812 1 year, 7 months ago

Comments

phxx (on August 7, 2008):

This would be a cool candidate for django-command-extensions.

#

AdamKG (on August 27, 2008):

Doesn't seem to work with inherited models. Insanely useful, though.

#

peterbe (on May 13, 2009):

Doesn't work unless you specify a slice. I had to use

myapp.MyModel[:]

#

joe_generic (on January 25, 2011):

Useful. Could use a little refactoring and debugging.

Not sure if this matters, but slice is now a native part of the python language; so %s/\<slice>/model_slice/g for starters, (i.e. replace slice with model_slice, if you're not familiar with vim.)

Then insert around line 108 model_slice = model_slice if model_slice != '' else ['',''] after the other processing on model_slice has been done.

That's just a quick hack fix to the requirement for a slice.

#

andybak (on July 1, 2015):

Here's a cleaned-up version with the joe_generics suggestions implemented. I've also switched to using Django's Collector so I could easily follow reverse relationships. Pretty much untested at the moment: https://djangosnippets.org/snippets/10506/

#

Please login first before commenting.