«
»

Okay, I discovered this today when looking to increase the speed at which uploaded documents were saved to disk. Now, I can’t explain the inner workings of why it is fast(er), all I know is that with the exact same form upload test ran 100 times with a 25MB file over a 100Mbit/s network this method was on average a whole 2.3 seconds faster over traditional methods of write, writelines, etc..

How does this extend to real-world production usage over external networks, well no idea. Though I plan to find out. So you all will be the first to know as soon as I find some guinea pig site that does enough file uploads to implement this on.

# Minus some boiler plate for validity and variable setup.
import os
import shutil
memory_file = request.POST['upload']
disk_file = open(os.path.join(save_folder, save_name),'w')
shutil.copyfileobj(memory_file.file, disk_file)
disk_file.close()

This entry was posted by Wayne on 16 Jul 2008 at 18:27 under code, python. Tagged , . You can leave a response, or trackback from your own site. Follow any responses to this entry through the RSS 2.0 feed.

2 Comments

  1. pieceofpy.com » Blog Archive » Code: pycurl HTTP file upload says:

    [...] Simple as that. As for handling the file upload in the Python script, take a look at Code: Saving in memory file to disk [...]

    28 Dec 2008, 22:31

  2. Code: pycurl HTTP file upload « Unordinary Discharge says:

    [...] [/sourcecode] Simple as that. As for handling the file upload in the Python script, take a look at Code: Saving in memory file to disk « Prev：Code: Saving in memory file to disk Book: Test Driven Development by Example [...]

    17 May 2010, 15:03

Leave a Reply

You must be logged in to post a comment.