Python Beauty

by Zef Hemel

I’ve been playing with Python some more in the past few days (more on that in the close future). The more I use it, the more I love it. The syntax is rich but not absurdly rich like Perl or Ruby. It has a couple of concepts wich are applied throughout the language with no (or very few) special cases. I’ll give two examples of the power of Python: adding a new feature to the language (method synchronization) and some examples of functional-like programming.

I’ll start off with the synchronized decorator. If you want to avoid that multiple threads call a particular data-mutating function at the same time (which can cause data corruption) there are synchronisation features in Python, but they’re not very nice to use. Example:

import thread
someMethod_lock = thread.allocate_lock()
def someMethod():
   someMethod_lock.acquire(True) # Wait until lock is released
   try:
      # Do your stuff
   finally:
      someMethod_lock.release()

In Java this is easier, you can just mark a method as synchronized:

public synchronized void someMethod() {
   // Do your stuff
}

It doesn’t do exactly the same thing (as Java has one lock per object, instead one function/method as kind of implied by this example), but it’s close enough.

I wanted to have this Java feature in Python. I figured out a nice way to do this, in particulary in Python 2.4, which has decorators. First I defined a synchronized function:

def synchronized(fun):
    def caller(*params, **kparams):
        fun.lock.acquire(True) # Wait
        try:
            return fun(*params, **kparams)
        finally:
            fun.lock.release()
    fun.lock = thread.allocate_lock()
    return caller

Eh? If you’re not famiar with Python you may wonder what I meant with the simplicity-of-Python thing I just mentioned, and rightly so. I just used some fairly esoteric features of Python. First of all I used a function object (fun) as the place to store the lock variable (because everything in Python is an object, including functions). Second, what’s up with that * params, **kparams stuff? I’ll not explain it in detail but what it comes down to is that all the parameters passed to that function are caught in params and all named parameters in kparams. So if you would call the function caller like this: caller(’Zef’, 5, bla=’hello’) then params would contain ['Zef',5] and kparams would contain a hashtable: {’bla’:'hello’}. The only reason to use this is to kind-of forward all parameters given to caller to the actual function-to-be-called fun. And third, this synchronized function returns a function, which is possible in Python because functions are objects in Python. Confused yet? Don’t worry, it took a while before I grokked it as well. Lots of experimenting help.

Anyway, the thing I wanted to show is what you can do with it and how you can use it.
In Python 2.4 using decorators:

@synchronized
def mySyncedMethod():
  # manipulate data

That’s it. Quite close to Java’s syntax eh? Because Python 2.3 doesn’t support decorators yet, it looks less elegant:

def mySyncedMethod():
  # manipulate data
mySyncedMethod = synchronized(mySyncedMethod)

Note that the @synchronized decorator is just a fancy way of writing the mySencedMethod = synchronized(mySyncedMethod) line in Python 2.3.

And now for something more simple to understand: some simple functional programming. I think people with some functional programming experience will appreciate this the most. One-line functions in Python can be defined on the fly using the lambda construct, like so:

plusTwo = lambda x: x + 2
print plusTwo(8) # 10

You can nicely apply this feature with the map and filter functions. map applies a function to every item of a list and returns the result. filter tests every element in a list using a function (returning a boolean) and returns all elements for which the function returned True.

numbers = range(10) # 0, 1, 2, ..., 9
print map(lambda x: x*x, numbers) # 0, 1, 4, ..., 81
print filter(lambda x: x % 2 == 0, numbers) # 0, 2, 4, 6, 8

There’s a shorter, more convenient language construct that does the same thing as map:

numbers = range(10) # 0, 1, 2, ..., 9
print [x*x for x in numbers] # 0, 1, 4, ..., 81

Why don’t you do like, for example, “Sam Gentile”:http://samgentile.com/blog/archive/2004/12/27/12406.aspx (a C# MVP) and make learning Python a new year’s resolution? You won’t regret it.

Rss Commenti

15 Commenti

  1. Keep spreading the word ^_^

    #1 Dagur

  2. Keep in mind that lambdas are supposed to be phased out in what is jokingly called Python 3000. Search for that and Python 3.0 to find out why (the basic argument is that it’s pointless to have both lambda and def in the language).

    #2 Manuzhai

  3. lambda’s are used all the time when creating GUI’s (for callbacks). I haven’t heard this about python 3000 but I knew that they would at least change the syntax.

    #3 Dagur

  4. yup, lambdas are going; map, filter and reduce are going. But don’t worry, we’re getting *optional* static typing….

    #4 john

  5. Wow, optional static typing, *sigh*.

    #5 Bob

  6. I don’t think you have to worry about lambdas dissapearing anytime soon. From everything I’ve read, Python 3000 is no longer considered a potential reality, but more of a thought excersize. The best ideas from which will be borrowed to extend the existing Python language.

    That said, I wouldn’t mind seeing lambdas go. Sooo unreadable.

    #6 Brendan

  7. FYI, your decorator is seriously broken; it does not do anything remotely like Java’s synchronized methods. For one thing, it will deadlock if a method is re-entered, even if it’s being called on a different object. For another, it does not protect against other synchronized methods being called on the same object. Compare with the following, which handles re-entrance, parallel access to different objects, and prevents calls to other synchronized methods on the same object:

    def synchronized(func):

    def wrapper(self,*__args,**__kw):
    try:
    rlock = self._sync_lock
    except AttributeError:
    from threading import RLock
    rlock = self.__dict__.setdefault(’_sync_lock’,RLock())

    rlock.acquire()
    try:
    return func(self,*__args,**__kw)
    finally:
    rlock.release()

    wrapper.__name__ = func.__name__
    wrapper.__dict__ = func.__dict__
    wrapper.__doc__ = func.__doc__
    return wrapper

    #7 Phillip J. Eby

  8. Argh. It looks like this thing uses code-unfriendly markup. :(

    #8 Phillip J. Eby

  9. I like your example, decorators need more & more examples as they are not trivial to understand and implement. Getting back to the example, what would be the cleanest way to call the deallocator:
    thread.deallocate_lock(fun.lock)
    upon program exit.
    We can use RAI here (create a small class with ctor & dtor for allocation & deallocation).

    What do you think?

    #9 Mike K.

  10. I’m not much of a theading expert, so it could be that I made some mistakes (and apparantly I did).

    #10 Zef Hemel

  11. s/isoteric/esoteric/

    #11 flippetygibbet

  12. s/flippetygibbet/flibbertigibbet/ ;)

    “For one thing, it will deadlock if a method is re-entered”

    Do you mean it would break if called recursively, as it would be waiting for itself to return to release the lock?

    “For another, it does not protect against other synchronized methods being called on the same object.”
    I thought the point of this was that only one thread at a time could call a synchronised method.

    If the intention is instead that only one access to an object’s methods is allowed at a time, wouldn’t that be better handled by the object in question?

    #12 flibble

  13. By default, java uses recursive mutexes (which you probably already know). Since they are recusive, it allows for one accessor to call another accessor, within the same instantiated object, without worry of deadlock, while ensuring reentrant protection.

    Now then, having said that, you are using RLock, which which is a recursive mutex; which seems to invalidate much of his point. My gut feeling on his comments is that he’s out in left field here.

    #13 oracle

  14. “”" Do you mean it would break if called recursively, as it would be waiting for itself to return to release the lock?”"”

    Yes, that’s why it needs an RLock.

    “”"I thought the point of this was that only one thread at a time could call a synchronised method.”"”

    No, it’s that only one thread at a time can call *any* synchronized method on the same object. The blog post’s version of a @synchronized simply disallows any invocation of that particular function on any object, which is vastly different from Java’s “synchronized” concept; it is not even remotely “close enough”.

    “”"If the intention is instead that only one access to an object’s methods is allowed at a time, wouldn’t that be better handled by the object in question?”"”

    Yep, which is why my version stores the lock on the object, rather than using a single global lock.

    #14 Phillip J. Eby

Sorry, the comment form is closed at this time.