CompressedTextField
I found my self doing data migration for a client, and also found that their old system (CSV) had 4 text fields that were 2 to 4MB each and after about 2 minutes of hammering the mysql server as I was parsing this and trying to insert the data the server would drop all connections. In my testing I found that if I didnt send those 4 fields the mysql server was happy to let me migrate all my data all (240GB of it). So I started thinking, "I should just store these fields compress anyways, a little over head to render the data, but thats fine by me." So thus was born a CompressedTextField. It bz2 compresses the contents then does a base64 encode to play nice with the server storage of text fields. Once I started using this my data migration, though a bit slower then without the field data, ran along all happy.
- text
- model
- field
- compress