authored by Wayne Witzel III

I use hg (Mercurial) for version control. Since switching to hg I have adopted the following process. I also do this for my Git projects at work.

• I create a local branch to working.
• I setup my External Tools in Eclipse to run my test suite.
• The output of my test suite gets committed to my local branch.
• I squash the local branch messages when I merge in to master.
• I add some insightful commit message for my master commit. Like, I haz changes.

So yesterday, I roll up my sleeves and prepare to dive in to an older project that smells like rotten potatoes. The plan of attack is to take this project and bring it up-to-date with Python 2.6, Pylons 0.9.7, and SQLalchemy 0.5.2 in the process of doing it, re-factor and extend where needed, of course letting the tests drive. I start my work and wand waving and 2-3 hours in I've removed about 200 lines of cruft and copy paste inheritance extended flexibility by further encapsulating some behavior using the Strategy pattern. I've got 47 tests (including functional doctests) passing and I'm green bar and happy with my time spent. So now time to merge this baby back in to master. My test suite external tool performs the hg add . and I keep my .hgignore pretty up-to-date for Python projects, so I feel confident doing that. I open up the terminal to check out the change sets and start the merge and I notice I missed a binary format in my .hgignore. So I now have about 15 unwanted files staged for adding. Being lazy and knowing my last commit was when I just ran my test suite, I blindly run. [sourcecode lang="bash"] $ ^R hg revert (Ctrl-R, hg revert - shell previous command search) $ hg revert -a --no-backup # ...my work being destroyed because I was lazy and not paying attention # whimpering [/sourcecode] It is at this point my day goes from great to awful. I face palm as I watch the uncommitted changes I've been making over the last 3 hours get reverted. As I mentioned, this project was older, in fact, it was started before the migration to hg and I never updated the External Tools runnable for this project in Eclipse to do the new hg add / commits. So every time I thought I was committing when I was running the tests, I was in fact not. Fortunate for me, I did have some buffers open and was able to recover the end result in about 45 minutes of hacking, but I did lose all of my change history which was very very disappointing (not to mention scary). So if I had any advice after this it would be ensure your older projects are up-to-date with how you do things now and they follow your current development process before you start refactoring. I guess the oneliner could be; When refactoring a project start with the tool set first.

A friend having issues installing Pylons on Windows XP with Python 2.6 gave me the idea to do this quick write up. So here it is, the 6 step method for running Pylons on Windows XP.

• Download Python.
• Add Python to your path and launch a command prompt.
• Download ez_setup.py, python ez_setup.py
• Download simplejson, python setup.py --without-speedups install
• easy_install Pylons
• easy_install formbuild
• Do a quick test; paster create --template=pylons

And that is all she wrote. Pretty easy. The reason we install simplejson seperate is because the default behavior is to build with speedups and well .. by default, that behavior won't work on a standard Windows XP machine. So we install it seperate to avoid any conflicts.

In preparation for a production deployment of a new Pylons app, I've been looking in to different deployment methods. In an effort to to be /. safe and Diggable when the new application launches, we've decided on 4 server deployment.

• 1 nginx server
• 2 pylons (paster) servers
• 1 postgresql server

I built nginx from the source without issues. The default install location of /usr/local/nginx works for me. You'll need to make any init scripts and install them, see your platform doucmentation for how to do this. You'll also want to be sure to add the new log dir to any log stats/consolidating/trimming jobs you run. Here is the important parts of the nginx configuration for proxying to the Paster servers. Also be sure you adjust your keep alive and connection timeout settings, if you have just a standard Ajaxy Web 2.0 application, you'll want to kick that down to 5 5 or 5 10. They default is way to high unless you're doing constant streaming of live updates or something to that degree.

worker_processes  2;
events {
    worker_connections  1024;
}
http {
    client_body_timeout   5;
    client_header_timeout 5;
    keepalive_timeout     5 5;
    send_timeout          5;
    
    tcp_nodelay on;
    tcp_nopush  on;

    gzip              on;
    gzip_buffers      16 8k;
    gzip_comp_level   1;
    gzip_http_version 1.0;
    gzip_min_length   0;
    gzip_types        text/plain text/html text/css;
    gzip_vary         on;

    upstream pasters {
        server 10.3.0.5:5010;
        server 10.3.0.6:5011;
    }
    server {
        listen       80;
        server_name  localhost;

        location / {
            proxy_pass http://pasters;
            proxy_redirect default;
        }
    }

The paster servers are setup like this, I put them both in the same .ini and setup them up in the tpl. This lets me do an easy_install , setup-app based deployment without having to manually edit the ini to change the port numbers, which is error prone. This also lets you adjust and tune per server, instead of deploying 1 server section and changing it for each. Example would be if one server was way more powerful, you could tune it and then use the weighting in nginx to prefer that server. All without having to edit the ini after deployment.

[server:main]
use = egg:Paste#http
host = 0.0.0.0
port = 5010
use_threadpool = True
threadpool_workers = 10

[server:main2]
use = egg:Paste#http
host = 0.0.0.0
port = 5011
use_threadpool = True
threadpool_workers = 10

Using 10 1000 on Apache bench gave me some good results. 85 requests per second to either of the standalone Paster servers. 185 requests per second when balanced with nginx. For fun, I deployed a third on my database server and was pleased to see 250 requests per second. Then I deployed 3 per server. So a total of 9 paster instances and was able to see 1080 requests per second. I also increased the thread of each from 10 to 25 , this uses more memory, but enables a higher RPS. Getting closer to the estimated 2,500 needed to survive a /. and should survive the estimated 1,000 from a high Digg.