philippe::niquille | irregular niche market thoughts

Streaming upload in django (for large files)

Jun 17th 2007
No Comments
respond
trackback

Do you get Python errors such as the following (or MemoryError) when you try to upload large (>10MB) via POST requests?

Traceback (most recent call last):

File “/usr/lib/python2.4/site-packages/django/core/handlers/base.py”, line 77, in get_response
response = callback(request, *callback_args, **callback_kwargs)

File “/usr/lib/python2.4/site-packages/django/contrib/auth/decorators.py”, line 17, in _checklogin
return view_func(request, *args, **kwargs)

File “/home/wwwgroup/sgoc/sgoc.ch/sgoc/stockphoto/views.py”, line 55, in import_photos
if request.POST:

File “/usr/lib/python2.4/site-packages/django/core/handlers/modpython.py”, line 69, in _get_post
self._load_post_and_files()

File “/usr/lib/python2.4/site-packages/django/core/handlers/modpython.py”, line 50, in _load_post_and_files
self._post, self._files = http.parse_file_upload(self._req.headers_in, self.raw_post_data)

File “/usr/lib/python2.4/site-packages/django/core/handlers/modpython.py”, line 119, in _get_raw_post_data
self._raw_post_data = self._req.read()

IOError: Client read error (Timeout?)

Axiak put together a wonderful patch to add streaming upload support to django (see tickets #2070 and #4165). See his latest install notes here.Hopefully it will make it to SVN soon!

Basically just get the two patches (1, 2) and apply them to the django source tree.

cd /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/django
patch -p0 < 5343_cleaned_streaming_file_upload_tweaked.diff
patch -p0 < 5343_javascript_and_admin.diff

Notes:
* progress doesn’t work on the dev server
* be sure to have __init__.py files both in

./django/contrib/uploadprogress/
./django/contrib/uploadprogress/middleware

* check the permissions if you get an ImportError
* set LimitRequestBody 104857600 in your Apache vhost (for a 100MB max, don’t go to high since Apache somehow does not accept the high value..)



Explore Recent



my del.icio.us



Similar Posts

No Comments

Leave a Reply