Login

Tag "loaddata"

Snippet List

Loading initial data per model at table creation (useful with migrations)

A very simple way of automatically loading data on model creation. As I am using South I wanted an automatic way of loading initial data only when a new model is create during a migration. Django provides almost everything out of the box by providing the *post_syncdb* signal (triggered also by South migrate command) and the *loaddata* command. The code will simply look for a an existing fixture named `<model>_initial.*` and invoke the *loaddata* command to try to load it Note: beware *post_syncdb* signal is deprecated since version 1.7: it has been replaced by *post_migrate*.

  • loaddata
  • migration
  • south
Read More

MongoDB data load

This snippet loads data from JSON files into a MongoDB database. The code is related with the other snippet [MongoDB data dump](http://djangosnippets.org/snippets/2872/). To get it working, just create a ``MONGODB_NAME`` variable in settings, holding the name of your Mongo database. This can be edited to fit more your needs. The snippet requires ``Pymongo``, since it uses its bson module and the ``MongoClient``.

  • loaddata
  • fixtures
  • mongodb
Read More

Remove old fields on dumpdata generated json

A small script that takes a manage.py dumpdata generated json file, and removes fields of the specified models. I needed this because i kept my initial data on a json file and after I removed a field on one of my models, the script wouldn't work anymore.

  • json
  • loaddata
  • fixture
  • dumpdata
  • migrate
  • removed field
Read More

CSV to JSON Fixture

**This script converts a CSV file into a JSON file ready to be imported via `manage.py loaddata` like any other fixture data.** It can be used manually to do a one-time conversion (for placing into a /fixtures folder), or used in a fabric script that automatically converts CSV to JSON live then runs `loaddata` to import as fixture data. To run script: >`csv2json.py input_file_name model_name` > >e.g. csv2json.py airport.csv app_airport.Airport > >Note: input_file_name should be a path relative to where this script is. **Scripts depends on simplejson module.** The module can just be placed in a sub-folder to the script to make it easy to import. If you use the same Python binary that you use for your Django site, you could use the Django import instead: `from django.utils import simplejson` **File Input/Ouptut formats:** Assumes CSV files are saved with LF line endings, and that first line has field values. First column is the model's pk field. Sample CSV input: id,ident,name,city,state 1,00C,Animas Air Park,Durango,CO 6,00V,Meadow Lake,Colorado Springs,CO 7,00W,Lower Granite State,Colfax,WA 12,01J,Hilliard Airpark,Hilliard,FL Output file name is input name + ".json" extension. Sample JSON output: [ { "pk": 1, "model": "app_airport.Airport", "fields": { "name": "Animas Air Park", "city": "Durango", "ident": "00C", "state": "CO", } } ] **Debugging Conversion Problems** If JSON import errors out with "ValidationError: This value must be an integer", you probably have a blank in an Integer field within your CSV file, but if can't figure out, try setting a breakpoint in file: ./django/django/db/models/fields/__init__.py e.g. 688 try: 689 return int(value) 690 except (TypeError, ValueError): 691 import pdb; pdb.set_trace() 692 -> raise exceptions.ValidationError( 693 _("This value must be an integer.")) To figure out what field caused the error, while in the debugger: (Pdb) u <- to go UP the callstack (Pdb) field.name

  • json
  • loaddata
  • fixtures
  • csv
  • import
  • fixture
Read More

dumpdata/loaddata with MySQL and ForeignKeys

InnoDB tables within MySQL have no ability to defer reference checking until after a transaction is complete. This prevents most dumpdata/loaddata cycles unless the dump order falls so that referenced models are dumped before models that depend on them. This code uses [Ofer Faigon's](http://www.bitformation.com) topological sort to sort the models so that any models with a ForeignKey relationship are dumped after the models they reference. class Entry(models.Model): txt = .... class Comment(models.Model): entry = models.ForeignKey(Entry) This code will ensure that Entry always gets dumped before Comment. Fixtures are an important part of the django Unit Testing framework so I really needed to be able to test my more complicated models. **Caveats** 1. You use this snippet to dump the data and the built in manage.py loaddata to load the fixture output by this program. A similar solution could be applied to the XML processing on the loaddata side but this sufficed for my situations. 2. This code does not handle Circular or self-references. The loaddata for those needs to be much smarter.

  • mysql
  • loaddata
  • foreign-key
  • fixture
  • dumpdata
Read More

Export models as json

I use this script to export a group of models that I want to import later as initial data. It exports them as serialized json, which is perfect for importing later with the loaddata function in manage.py.

  • json
  • loaddata
  • export
  • fixtures
Read More

7 snippets posted so far.