Snippets by skoczen

Sorl Thumbnail + Amazon S3

**General notes:** - Set MEDIA_URL (or whatever you use for uploaded content to point to S3 (ie. MEDIA_URL = "http://s3.amazonaws.com/MyBucket/")) - Put django-storage in project_root/libraries, or change the paths to make you happy. - This uses the functionality of django-storage, but *not* as DEFAULT_FILE_STORAGE. The functionality works like so: **Getting stuff to S3** - On file upload of a noted model, a copy of the uploaded file is saved to S3. - On any thumbnail generation, a copy is also saved to S3. **On a page load:** 1. We check to see if the thumbnail exists locally. If so, we assume it's been sent to S3 and move on. 2. If it's missing, we check to see if S3 has a copy. If so, we download it and move on. 3. If the thumb is missing, we check to see if the source image exists. If so, we make a new thumb (which uploads itself to S3), and move on. 4. If the source is also missing, we see if it's on S3, and if so, get it, thumb it, and push the thumb back up, and move on. 5. If all of that fails, somebody deleted the image, or things have gone fubar'd. **Advantages:** - Thumbs are checked locally, so everything after the initial creation is very fast. - You can clear out local files to save disk space on the server (one assumes you needed S3 for a reason), and trust that only the thumbs should ever be downloaded. - If you want to be really clever, you can delete the original source files, and zero-byte the thumbs. This means very little space cost, and everything still works. - If you're not actually low on disk space, Sorl Thumbnail keeps working just like it did, except your content is served by S3. **Problems:** - My python-fu is not as strong as those who wrote Sorl Thumbnail. I did tweak their code. Something may be wonky. YMMV. - The relative_source property is a hack, and if the first 7 characters of the filename are repeated somewhere, step 4 above will fail. - Upload is slow, and the first thumbnailing is slow, because we wait for the transfers to S3 to complete. This isn't django-storage, so things do genuinely take longer.