Yesterday I ran into a silly error which wasn’t very obvious at first, so I thought I’d share. It happened after renaming some module imports, here is the code and the traceback:

[gist]https://gist.github.com/1028793[/gist]

As you can see, there is a module called hostand also a local variable called the same used in a for loop. Then why does Python think the first use of the host variable is done before its assignment? The answer can be found here in the Python reference guide. Let me highlight an important paragraph:

If a name binding operation occurs anywhere within a code block, all uses of the name within the block are treated as references to the current block. This can lead to errors when a name is used within a block before it is bound. This rule is subtle. Python lacks declarations and allows name binding operations to occur anywhere within a code block. The local variables of a code block can be determined by scanning the entire text of the block for name binding operations.

This means that because the host variable is assigned in the function, no matter where in the function, its scope is that function, and the global scope is not taken into account. One could think that adding global host at the top of the function would solve it. But that is plain wrong. If you do that, the host module will be overwritten by the last value of the host variable after iterating the for loop.

Then how do we solve it? There is no way to do it in Python 2.X, just change the name of either the imported module or the local variable. In Python 3 the nonlocal keyword can be used, as pointed out here.

:wq

This is a quite common mistake in Python: using mutable types as default function arguments. Lets see how this happens.

First, here is an example of default function arguments with non-mutable types:

[gist]https://gist.github.com/1008875[/gist]

As you can see, the bar variable has a different id on each instance, they are in fact, different objects. Now lets see what happens if the default argument is a mutable type, like a list.

[gist]https://gist.github.com/1008874[/gist]

Oh, the bar variable is modified on both instances! It has the same id, son they are effectively references to the same object.

Why is this? In Python, default argument values are evaluated only once, when the file is parsed. So, a list object is created, and its reference is used as the default argument to every instance of the class we create.

In order to avoid this we should pass None as the default value and then set bar to the empty list if the passed argument is None:

[gist]https://gist.github.com/1008879[/gist]

:wq

It’s very annoying for a program to crash, but it happens. When it happens if the system was configured to dump the core you’ll get a core file with hopefully enough information to debug the crash.

But we don’t know if the system will be configured to dump the core, so we may want to do this from our Python program. We can use the resource module in the standard library. The following example shows how to enable core dumps if —dump-core option is specified:

[gist]https://gist.github.com/772101[/gist]

PS: This worked just fine for the root user but didn’t seem to do the job for a normal user.

Registry is a quite common design pattern which I recently needed to implement and I found Python class decorators very useful so I thought I’d write about it. :–)

In the registry pattern we have a global object we call the registry or registrar which contains references to shared objects and it’s the only entity which can access those.

In my case, I had a plugin style architecture in which different plugins might be dynamically added without any configuration. The first implementation I made used a metaclass to add the plugin class to the registry:

[gist]https://gist.github.com/771884[/gist]

That worked, at least for the beginning. But later I found a limitation: plugins needed the Plugin metaclass, so I couldn’t use another metaclass on them if I needed to. And at some point I did feel the need to do that. I wanted the plugins to use the Singleton metaclass (to implement the Singleton pattern) but then I couldn’t use the Plugin metaclass because only one metaclass can be used (and I didn’t want to create a SingletonPlugin metaclass). And class decorators saved the day:

[gist]https://gist.github.com/771888[/gist]

Class decorators are supported only in Python >= 2.6, but that wasn’t a problem. :–)

:wq

In Python we have a way for creating anonymous functions at runtime: lambda. With lambda we can create a function that will not be bond to any name. That is, we don’t need to def a function, and lambda can be used instead.

[gist]https://gist.github.com/707898[/gist]

Lambda can also be used to call another function by fixing certain argument:

[gist]https://gist.github.com/707899[/gist]

However, lambda functions use late binding for the arguents they get. That is, when you create a lambda function passing a variable as an argument, it’s not immediately copied, a reference to the scope is kept and the value is resolved when the function is called. Thus, if the value of that argument changes within that scope the lambda function will be called with the changed value. Lets see it in action:

[gist]https://gist.github.com/707901[/gist]

Unexpected? It depends, but it was definitely unexpected when I run into it. The solution for this is to bind early, by copying the value at the time of the creation of the function. We can do this 2 ways:

Using a fixed parameter in the lambda function:

[gist]https://gist.github.com/707903[/gist]

Using functools.partial:

[gist]https://gist.github.com/707904[/gist]

I personally like the second approach, using partial, since its more explicit and I really see option one as a workaround to how lambda functions work.