Upload progress handler using cache framework

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
from django.core.files.fileuploadhandler import FileUploadHandler
from django.core.cache import cache

class UploadProgressCachedHandler(FileUploadHandler):
    """
    Tracks progress for file uploads.
    The http post request must contain a header or query parameter, 'X-Progress-ID'
    which should contain a unique string to identify the upload to be tracked.
    """

    def __init__(self, request=None):
        super(UploadProgressCachedHandler, self).__init__(request)
        self.progress_id = None
        self.cache_key = None

    def handle_raw_input(self, input_data, META, content_length, boundary, encoding=None):
        self.content_length = content_length
        if 'X-Progress-ID' in self.request.GET :
            self.progress_id = self.request.GET['X-Progress-ID']
        elif 'X-Progress-ID' in self.request.META:
            self.progress_id = self.request.META['X-Progress-ID']
        if self.progress_id:
            self.cache_key = "%s_%s" % (self.request.META['REMOTE_ADDR'], self.progress_id )
            cache.set(self.cache_key, {
                'length': self.content_length,
                'uploaded' : 0
            })

    def new_file(self, field_name, file_name, content_type, content_length, charset=None):
        pass

    def receive_data_chunk(self, raw_data, start):
        if self.cache_key:
            data = cache.get(self.cache_key)
            data['uploaded'] += self.chunk_size
            cache.set(self.cache_key, data)
        return raw_data
    
    def file_complete(self, file_size):
        pass

    def upload_complete(self):
        if self.cache_key:
            cache.delete(self.cache_key)



# A view to report back on upload progress:

from django.core.cache import cache
from django.http import HttpResponse, HttpResponseServerError 

def upload_progress(request):
    """
    Return JSON object with information about the progress of an upload.
    """
    progress_id = ''
    if 'X-Progress-ID' in request.GET:
        progress_id = request.GET['X-Progress-ID']
    elif 'X-Progress-ID' in request.META:
        progress_id = request.META['X-Progress-ID']
    if progress_id:
        from django.utils import simplejson
        cache_key = "%s_%s" % (request.META['REMOTE_ADDR'], progress_id)
        data = cache.get(cache_key)
        return HttpResponse(simplejson.dumps(data))
    else:
        return HttpResponseServerError('Server Error: You must provide X-Progress-ID header or query param.')

More like this

  1. Upload, Progressbar with sessions by revolunet 5 years, 8 months ago
  2. Ajax progress bar by ebartels 6 years ago
  3. Fake File Uploads by rfk 5 years, 2 months ago
  4. upload handler decorators by Rozza 5 years, 9 months ago
  5. A custom 500 handler which is AJAX-aware by mallipeddi 6 years, 2 months ago

Comments

kayamb (on May 2, 2008):

Hi,

Thanks for those 2 snippets, they look very promising!

However, I can't quite figure out how to use them. In particular what kind of view should the html form's action attribute point to, and how would that view handle the form? Should that view wait for the completion of the upload before returning anything?

Here follows my first shot at making that view. Is that a good start, and if so, what is missing?

def frontpage(request):
    if request.method == 'POST':
        # [????] I don't know what to put there!...
    else:
        form = UploadForm()
    return render_to_response('frontpage.html', { 'form': form }, context_instance=RequestContext(request))

Thanks a lot!

Julien

#

kayamb (on May 3, 2008):

Never mind the comment above, I think I hadn't applied the patch properly.

#

revolunet (on July 10, 2008):

Hello and thanx a lot for these great two samples. Is it compatible with actual trunk ?

Can you show an example of the view who 'starts' the upload please ? i cannot figure it :(

can i use request.upload_handlers.insert(0, UploadProgressCachedHandler()) in my view instead of modify the settings.py ?

#

revolunet (on July 17, 2008):

hello

ok after some hours i finally made it work with some changes :

im using apache2+mod_wsgi with latest trunk

i use session instead of cache middleware, with a unique sessionkey for each upload ID. i have to use self.request.session.save() in the receive_data_chunk to update the data for the progress view.

instead of tweaking settings.py, i add in my view :

if request.method == 'POST':
    request.upload_handlers.insert(0, UploadProgressSessionHandler(request))

Thanks !

#

revolunet (on July 17, 2008):

in my upload_file_handler, i had to add

os.remove(f.temporary_file_path())

so the temporary file in /tmp can be removed

#

tuttle (on July 26, 2008):

Thanks for this snippet!

Anyway, I think it is worth mentioning that this will

1) NOT currently work with Django devel webserver (cannot serve multiple requests at once),

2) NOT work with a cache not sharing data between processes such as locmem://.

Finally I made this to work on prefork Apache with memcached://.

#

t3mp0 (on August 29, 2010):

To get this work with Django 1.2, line 1 needs to be changed to:

from django.core.files.uploadhandler import FileUploadHandler

#

williamcai (on December 11, 2010):

its great. but as tuttle's comments, it didn't work with Django develop web-server, which took me some time to find this. It's better to put this notice somewhere remarkable.

#

jhl (on December 12, 2010):

With a couple of exceptions, header parameter names are typically changed by the server. For example, X-Progress-ID would become HTTP_X_PROGRESS_ID. I haven't tested this, but the conditional clause

elif 'X-Progress-ID' in self.request.META:
    self.progress_id = self.request.META['X-Progress-ID']

should probably be changed to

elif 'HTTP_X_PROGRESS_ID' in self.request.META:
    self.progress_id = self.request.META['HTTP_X_PROGRESS_ID']

#

aaronfay (on May 13, 2011):

I think it's also worth noting that the GET is important, I tried to put the X-Progress-ID in my form and then access it with 'self.request.POST' but the POST data isn't ready yet! Infinite recursion is the bomb...

Aaron

#

jachetto (on September 21, 2012):

I'm not able to make it working in Admin. Looks like something goes wrong with csrf. After submission i get a csrf error protection (i'm just tryng to add a progress bar in my standard django admin ui).

Any idea?

#

maciej.m (on October 28, 2012):

The size of uploaded data differ from length of the file:

{"uploaded": 3997696, "length": 3954569}

#

(Forgotten your password?)