Plupload S3 mixin for Django

Use this together with Plupload to upload files into your S3

Dependencies

django-mediagenerator – https://bitbucket.org/wkornewald/django-mediagenerator/src – this can be easily altered to suit your needs
djangotoolbox – https://bitbucket.org/wkornewald/djangotoolbox – for JSONResponse. This can be easily altered if you don’t want to install djangotoolbox
appengine-urlfetch – This was built for use in AppEngine however can be easily changed to use urllib

Installation

1. Change your model to inherit from s3mixin.models.S3Mixin instead of django.Model

1. Add in your urls.py in ‘urlpatterns’ array:
   url(r'^userprofile/s3policy/$', 'accounts.views.userprofile_s3policy', {}, name='s3policy' ), url(r'^skin/(?P<pk>add)/s3policy/$', 'accounts.views.skin_s3policy', {}, name='s3policy' ), url(r'^skin/(?P<pk>\d+)/add/s3policy/$', 'accounts.views.skin_s3policy', {}, name='s3policy' ),

1. Add in your views.py
   @login_required def skin_s3policy(request, pk): ret = s3mixin.views.s3policy(request, 'accounts/skin/%s' % (request.user.id)) skin = Skin() if pk=='add' else Skin.objects.get(id=pk) skin.file = settings.AWS_PREFIX + ret.filename skin.name = os.path.splitext(os.path.basename(ret.filename))[0].lower() skin.user = request.user skin.save() return ret

@login_required
def userprofile_s3policy(request):
ret = s3mixin.views.s3policy(request, ‘accounts/userprofile/%s’ % (request.user.id) )
profile = request.user.get_profile()
profile.file = settings.AWS_PREFIX + ret.filename
profile.save()
return ret

Example prefix might be a blog/forum id that the user has permission to.
A simplistic prefix might be the user-id to which the file will be linked to.
You may use the GET[‘file_size’] to create a more robust logic for upload size limits.

1. In your form, use S3FileWidget

1. In your template, make sure to include
   <script type="text/javascript" src="{% media_url 'js/plupload/plupload.full.min.js' %}"></script> <script type="text/javascript" src="{% media_url 'plupload-s3.js' %}"></script> <script type="text/javascript"> {{ form.fields.file.widget.javascript }} </script>

1. In your settings.py add:
   AWS_ACCESS_KEY_ID = '' AWS_SECRET_ACCESS_KEY = '' AWS_MAX_FILE_SIZE = '10MB' AWS_BUCKET = 'dev-bucket' if not on_production_server else 'prod-bucket' AWS_PREFIX = 'http://%s.s3.amazonaws.com/' % (AWS_BUCKET) AWS_CLOUDFRONT = 'http://%s.my-cloudfront.com/' % AWS_BUCKET # Your CNAME for Cloudfront
2. AWS_DNS_ROTATOR = 6 # Determines if CLOUDFRONT urls will be prefixed by m1.my-cloudfront.com, m2.my-cloudfront.com ….
3. THUMBNAIL_SERVICE = ‘http://d2e9m6sll3dt8o.cloudfront.net/’ if on_production_server else ‘http://1linedev.thumbnail-service.appspot.com/’


THUMBNAIL_SERVICE may be used in conjunction with https://github.com/burgalon/thumbnail-service to generate thumbnails at any size on the fly. Another suggestion is to use a proxy cache to avoid generating those thumbnails on every request – https://github.com/burgalon/SymPullCDN

1. Add crossdomain.xml to your AWS_BUCKET of the sort:
   
   
   

Also suggested is to set a robots.txt blocking crawlers from scanning the buckets.

Finally, when you think all setup is done, run the page through a browser, make sure that the widget’s javascript is rendering, and that nothing fails on the javascript

Comments

1. Notice that the code has resize : {width : 1920, height : 1080, quality : 90} which means that all uploaded JPG/PNG will be resized if larger than 1920×1080. This seems reasonable with our current global standards. Resizing photos disables progress reporting in Flash
2. S3Mixin has SERVER_TYPES attribute which allows setting file types which you might want to store on the server side (to allow reading the file directly from DB and not through S3). Currently it’s set to HTML files. You might want to change this.
3. Currently only the Flash runtime of Plupload is being used in order to keep things consistent and stable. S3 requires cross-domain upload and passing multi-part consistently in order to match the signature. Some of the runtimes are not passing the same parameters exactly as the Flash runtime and might require some tweaking in order to work.
4. Feel free to fork and help to turn this into a pluggable app which is generic enough

Example

You can see a working example on www.boorgle.com in the edit profile page after you’re registered