pieceofpy.com

Morning Tutorial

In the AM I attended the “py.Test: rapid testing with minimal effort“. I was planning to attend Python 401, but that filled up before I registered. I learn some new things about py.test that I didn’t know about, having never read the doc for it, it wasn’t hard.

I learned about the -k switch (loop on fail). Basically this continually runs the tests as source units change. It only reruns failing tests or new tests.

The generative tests using yield was something I had know about but never used and now I know exactly where I will be applying this. I have a program that takes a dozen or so different command line arguments and switches. I will generate a text file with all possible combinations. Then I can use that file to run through the command line tests. 

Afternoon Tutorial

In the afternoon I attended the “Advanced SQLAlchemy“ tutorial. I am not sure if I was the target audience for this tutorial or not. I have been using SQLAlchemy for a while now. The topics of coverage showed promise. Maybe I built it up too much in my head. A tutorial by Bayer himself, he wrote it. I should be blow away here. I wanted to leave this tutorial thinking, “Wow look how stupid I was, look at how much easier this is when you use FOO or BAR.” Or WOW, I never knew you could do that. Sadly, I have to say I had neither of those moments.

First let me say this, this isn’t a critique of either of the tutorial instructors or the content of it at an academia level. Both were quality.

I have to say the best thing I did see a nice shorthand way of doing somethings with the declarative_base. The coverage of inheritance mapping wasn’t really mind blow single inheritance was a sparse matrix approach using exclude and the join inheritance was just a Strategy pattern.

Transactions were covered for what seems like people who had never worked with transactions. The deadlock example that was given using SessionExtension was nice and practical and really the only thing that made me go “ahhhh” as I knew I could refactor the current way I was dealing with concurrency and databases with SQLAlchemy.

Summary

Overall, it was a good day. The coverage in the tutorials was very good. The dialog with people who were attending the tutorial was the best part really. Helping people work through the examples and answering some questions that people were either too shy or too embarrassed to ask to the whole class.

Just like with OOPSLA 2008, I like to keep a personal log of what I did at the conference each day. Today was my first day at Pycon  2009. We arrived early this morning (around 9 AM). I managed to get registered and pickup my fun bag which included a Pycon shirt, a Launchpad shirt, and an CD for Opensolaris amoung other things.

We ate at the convention center restaurant across the street from the Hyatt Regency O’Hare .. yeah don’t go there. After that, we took a nap, we had been up since the day before (our flight in to Chicago left Florida at 5 AM).

We stopped at Red Bar, the bar inside the hotel and had a few drinks and spoke with John Moulder (spelling?) who was also attending Pycon. We laughed a little since he works for the government and his last name was Moulder.

I have 2 tutorials tomorrow. An AM tutorial about py test and an afternoon tutorial on SQL Alchemy, Looking forward to both. Even though I am disappointed the Python 401 tutorial was full, I am sure the py test tutorial will be a fine substitute and equally as informative.

Heading to PyCon this year. Looking forward to the tutorials and the great line up of keynotes. I highly recommend attending this year, it looks like one of the best PyCon’s in a while. I’ll be attending the Advanced SQLAlchemy tutorial and the py Test tutorial. I was hoping to get in to the Python 401 tut , but registered late and it was already full.

They key notes I am looking forward

• Building tests for large, untested codebases by C. Titus Brown
• Metaprogramming with Decorators and Metaclasses by Bruce Eckel
• Topics of Interest by Ian Bicking

So if you are a Python hacker get over to http://us.pycon.org sign up and get yourself there! It is gonna be a great conference this year.

As I am sure most of you have heard Python 3.0 (final) has been released. For me, this means some nights getting some continuing development projects updated for the language changes and freezing some projects in maintence mode with their own copy of Python 2.6 (or in some cases 2.4).

Some highlights

• print is now a function: print(”5×5″, “is”, 5*5, sep=” “)
• annotations for methods (I create a lot of libraries, so this is great!)
• extended unpacking: x, y , *z = [1,2,3,4,5] now x is 1, y is 2, and z is 3-5
• <> removed, use != (personal favorite cause I hate <>)
• no longer can you from import * inside functions

See the whole list here: http://docs.python.org/3.0/whatsnew/3.0.html

Been reading around and I guess November is the official blog entry a day writers month or something. I’ll make a go again. In October, I tried the one post a day run for my blog. I did well, but fell short in the end. Though I’ve already missed the first two days in November, I’ll call that the margin of error.

Oh and I believe the posts should have some meat to them. Not just another “Hey look, a post, I win November.” Though as a last resort, I am not above that.

So for lack of anything better to write about, here is an oldie but goodie. A Python implementation of the proxy pattern (virtual proxy) with a real worldish feel to it.

First we define our ABC and subclass it for our needs.

class File(object):
    def load(self):
        pass

class RealFile(File):
    def __init__(self, name):
        self.name = name
        self.load()

    def load(self):
        print "Loading %s..." % (self.name)
    def process1(self):
        print "[phase1] Processing %s..." % (self.name)
    def process2(self):
        print "[phase2] Processing %s..." % (self.name)

Now, we can subclass File for our proxy. Now I know what you are saying. You don’t need the extra levels of abstraction because Python doesn’t have the levels of type sensitivity of other languages. Could you just implement this in the first RealFile subclass? Yes, but that isn’t the point of this. The verbosity helps define the example and keeps this implementation language independent (mostly).

class ProxyFile(File):
    def __init__(self, name):
        self.name = name
        self.file = None

    def process1(self):
        if not self.file:
            self.file = RealFile(self.name)
        self.file.process1()

    def process2(self):
        if not self.file:
            self.file = RealFile(self.name)
        self.file.process2()

So you can see, this hides away the details of loading the file. Allows the user to call process1 / process2 as the business logic determines and preforms lazy loading. The Proxy pattern is very powerful when combined with other patterns. Like Null Object and Lazy loading.

def main():
    f1 = ProxyFile("bigdb01.csv")
    f2 = ProxyFile("bigdb02.csv")
    f3 = ProxyFile("bigdb03.csv")

    f1.process1()
    # some busines logic
    f1.process2()
    # more BL
    f2.process2()
    # more BL
    f2.process1()
    # Hey, we found what we needed, skipped f3
    #f3.process()

if __name__ == '__main__':
    main()

You can view full source at: http://trac.pieceofpy.com/pieceofpy/browser/patterns/proxy.py

After a few drinks at the hotel bar, we head to bed to get ready for Day 3 of OOPSLA.

Keynote: Pyrimids and Software or something…
I showed up late for this, but from what I gathered, it was a comparison to the Pyrmids and software. It was really facinating and I enjoyed Mark Lehner’s talk about finding the lost city and everything .. but something this heavy at 8:30 in the morning is well .. yeah, neat.

Introducing New Ideas Into Your Organization
Linda Rising (http://lindarising.com for those of you living in a cave) gave a great tutorial on how people can get new idea into their organization. Everyone has had to deal with this at one time or another no matter where they have worked. People don’t like change. They don’t wanna change if they don’t have to, even if all the reasons are good and they agree with the new idea, in the end, if they don’t want to change, they won’t. As my wife always says. “I want you to WANT to take out the garbage.”

Quick side note, I was speaking with Ricardo Lopez on the day previous to Linda’s tutorial and his advice was to make sure you have enough in your savings for when you get fired. Now as funny as it sounds, his reason, which was being fearless and with standing pressures to make concessions was directly inline with the ideas and patterns Linda provided in her tutorial.

Back to the tutorial. I arrived early (mistake), so the other early arriver’s and myself ended up being actors in a play. The play of how to introduce new ideas into your organization. The play included Innovators, Early Adopters, Connectors, etc.. and how/when you can use these different types of people in the organization to our benefit and pitfalls to avoid so they don’t become enemies.

The Cliff notes version, get the easy ones on board first (innovators like new cool stuff), put a small step of the new idea in place with this group. Now take it to the early adopters (how depends on your corporate culture), get them involved and maybe add another step. Continue to promote this style growth until you have enough good results from this new thing to ask some of the good ol’ boys who might have middle managements ear, continue on this path with Brown Bags and cookie offerings. At this point the movement itself is viral and you’ll need to start tending to the delicate process of getting the upper management stamp of approval.

Anyway, the tutorial was a pattern language for introducing new ideas based of her latest book and I must say, I really enjoyed the tutorial and acting out my part. Though the pattern language should be adjusted and tailored to fit your corporate culture, I think it is an excellent start for anyone having problems getting no ideas in to their company, be it agile development or more water fountains. The tutorial finished with a set of group sessions tasked with solving a member of each groups problem. You can find more information about these patterns and her on Linda’s website mentioned at the start of this post.

Afterwards
After the keynote and tutorial, my wife and I ate at the hotel restaurant and called it a night. This conference is fun and loaded with great information and conversation, but leaves my brain so tired at the end of the day it is hard to to much else but eat some food and go to sleep.

After a goodnight sleep, I mange to NOT be late for this mornings tutorial sessions. Today I have planned to attend Advanced Software Architecture and Pattern-Oriented Software Architecture: A Pattern Language for Distributed Computing.

Advanced Software Architecture
Marwan Abu-Khalil was the presenter for this tutorial. This wasn’t as hands on as I usually like from my tutorial sessions, but with that being said, there was a lot of good knowledge being tossed around.

We cover the best practices and outlined a rough path one should follow when designing software. Now it was mentioned frequently and often, that this was just the most common path encountered and that each project would lend itself to a different, there inherit lines the hard part of software architecture. If you could just put it in a patterns book, everyone would be good at it. The quote frequently used was “All architecture is design, but not all design is architecture.”

Brainstorm session instigated by this tutorial session.
As a developer I myself tend to loose focus of the bigger picture as I dive deep in to the inner workings of some specific implementation or core sub-system. This course gives a nice high-level perspective of what the software architecture should focus on. For micro sized development teams; 1-5 developers, these things become even more important, since in most cases, every developer on a such a small team will be responsible for some of the most complicated areas of the system. This goes against the rule of thumb that architects, though they should be involved in the implementation, should not be involved in developing any of the core or complex systems of the architecture, so to prevent them from loosing focus in the details.

I agree with this, but as mentioned, in micro teams this can be very hard, if no impossible to do (1 man “team”). This is why introducing things like peer code review and daily standups are so important. Most people seem to have this idea that the smaller you are, the less you need to apply todays agile methods, on the countrary, staying agile with more developers as you can decouple subsystems and spread them across multiple developers or teams. Staying objective and keeping systems decoupled and preventing architectural drift in a one or two man team is even harder than say three, five man teams.

Pattern-Oriented Software Architecture
This tutorial was done by Doug Schmidt, side note, he’s been living in Nashville the last few years doing his research and work out of Vanderbilt University.

This was a great tutorial. The subtitle was A Pattern Language for Distributed Computing. But it wasn’t so much about completely new patterns, in fact any on fimilar with any of the GoF, PLOP, or POSA books would of received a lot of review. This was about using many proven patterns that we’ve used in stand-alone archictectures for a long time, take those and establish a pattern language for solving the problems we encoutned in todays distributed enviroments.

A pattern mentioned which I must of missed from the POSA 2 book, but new it immediately because I have implemented it under different names was the Active Object pattern. Quickly, Active Object decouples the execution from the invokation of a method. Good coverage of other patterns as well. The handout includes pros and cons in a distributed enviroment for all the patterns presented.

I did take issue with some of the cons being things like “Harder to test” and “Harder to debug”. I just kept thinking .. if you mock out the interactive components so that your tests are decoupled from any implementation, it seems no harder to test the Active Object pattern vs. any other piece of software. Questions like what if the request comes back out of order? What if the request processor goes offline? What do you do with the queue? What if you get a response for a request that is no longer valid? No more room in your queue? Etc, etc.. All of these conditions can and should be covered by tests. I don’t have issues with testing or debuging my most recent Active Object implementation.

There was also a nice closing discussion about utilizing multi-core, embedded devices giving us a new reason to remember all the cycle and memory conservation techniques every said we could forget and if existing functional languages would be the wave of the future to take advantage of advancing core technology. Schmidt conclusion was we will need a new language or a dramatic extention to deal with these new concurrency issues … I just kept thinking Stackless.

In the end, it was another standard issue Schmidt tutorial, it did not disappoint. Fast pace and lots of invited Q&A and viewer interaction. Even if you don’t like the guy or his ideas, it is worth it to see him give a talk at least once. Random side note .. he refered to Python once, following the words “novelty languages like…”

I didn’t proof this at all, corrections welcomed.

After the seven hour drive to Nashville, we settled in to our hotel and hit the bar. Getting the brain ready for todays activities. I’ll be updating at the end of each Tutorial or Keynote through out the day, but will only publish the entries at the end of day, so bare with me if the entrys seem like many posts in one.

Green Bar for C++
This was the first turotial I atteneded. The one I had picked was canceled and this was my runner up. The tutorial was handled by the author CUTE Peter Sommerlad. The presentation was good and packed full of content. I have been out of the C++ game for a while now, but his presentation of CUTE integrated in to the new Eclipse has sparked my interest once again and really made me want to take on another C++ project. Maybe an optimized parser this extends Python bindings .. just have to find something to parse.

They were very well setup for the course. Course was very interactive and hands on. I spent the second half of the course implementing the classic Numeric to Roman converted using Eclipse and CUTE with TDD practices. The practices themselves were all refresher for me, but getting a chance to put my hands on some C++ felt good. It had been over a year since I really got dirty in C++. They also had multi-platform installs of Eclipse and a CUTE package archive on a flash drive. Everyone in the tutorial was up and running in no time. I made not of this for sure, in case my any of my recent proposals to speak get accepted.

I was also able to see other peoples approach to the same problem which is always insigntful and thought provoking, no matter the language. I spoke with Peter breifly after the tutorial. The functionality of the CUTE framework itself only took about 2 days of development. The integration in to Eclipse is a different monster. Thanks for the free pen and the swiss chocolate.

Building SOAs with Web Services
Olaf from IBM gives us a great tutorial on using SOAs to build web services. The introduction was an abstract overview of SOA (Service Oriented Archiecture) with some touches on concepts and motivation behind SOAP and WSDLs.

He then spoke about some real world implementations and showed some examples and diagrams of those implementations. After which we began the interactive part of walking through a very simple implementation that tied a PHP script to a Java SOA via Tomcat/Axis.

Coverage was also given to user experiences and best practices and identifying when a given problem lends itself to the SOA (WSDL/SOAP) solution. Overall it was a good tutorial and getting a chance to see the Web Tools plugin for Eclipse in action generating bottom-up from existing POJO and making the WSDL contract was neat.

Afterwards
After the day at OOPSLA my wife and I at some St. Louis style BBQ down on Broadway. It was very good. I have a personal goal to eat nothing but BBQ for dinner during my time in Nashville.

I read through the guidelines and I didn’t see anything against me publishing my submission for Pycon. So I’d like any feedback you might have on this tutorial outline.

I have to ask myself “Who is the target audience?”. A developer with some programming, maybe not Python experience who does web programming. A developer with some basic familiarity with web frameworks, maybe having used rails, Turbogears, Django, or even Pylons before. A developer who uses Pylons but hasn’t really looked at what it can do for them?

I also keep hemming and hawing over the web services the “hard way” section. Why show someone the way not to do it? My argument for it is I find the positive impact of doing it the easy way, with the RESTful controller/routes template to be so much greater when it directly follows the common, home grown style I see all over.

I’ve kept it off the main page, since the proposal is rather long.
Continue reading ‘PyCon 2009 Submission’ »

I’ve enabled Gravatar for comments, if you don’t know what that is, you can find out all you need to know at http://gravatar.com

I have also setup a github account for all the source code from this blog. It is pretty easy to use and should a multi-contributor project stem from this blog, we’ll have a place for it. Find out more at http://github.com or browse the repo at:
http://github.com/wwitzel3/pieceofpy/tree/master

UPDATED
I took down the github repo and setup a mercurial repo with trac on top of it.
http://trac.pieceopfy.com/pieceofpy

See Blog themes and SCM for more details.