Killing Multithreaded Python Programs with Ctrl-C

If you have ever done multithreaded programming in Python you have probably found it frustrating that you can't simply hit Ctrl-C in the terminal and have it exit like a normal Python process. Instead you have to put the process in the background (Ctrl-D) and then either "kill %%" or kill the PID. The good news is that it doesn't have to be this way. After experimenting a bit I finally figured out why it doesn't work normally and what you have to do to make it work.

Normally when I write a threaded program in Python it looks something like this...

The problem is with this program is that if you hit Ctrl-C it doesn't do anything. The reason is that join() is a blocking operation. As a result the process will only receive the signal for Ctrl-C when join() becomes unblocked, which in this case will never happen.

In order to handle Ctrl-C with multiple threads you can use the following code:

This code does a few things differently in order to make it handle Ctrl-C as we would like. First, instead of using join() we use join(timeout) which tries to join a thread but will timeout if it does not occur after the timeout elapses. This allows the main thread of execution to continue doing other things, in particular waiting for a KeyboardInterrupt to be thrown which is what Ctrl-C raises. Since join will return upon timeout, we need to keep any threads which aren't None and respond to isAlive().

The next thing is that if child threads never return or take a really long time you need a way to notify the child that it should die. This is accomplished by the kill_recieved flag in the Worker class. When that flag is set by the parent process the child knows that it should finish up what it is doing and return.

The last thing is something that caught me off guard a bit. Initially, in the main() while loop I was trying to catch all exceptions that came up by using "try...except Exception:". As it turns out Exception does not include KeyboardInterrupt, meaning that Ctrl-C's that are raised in that block will not be caught. If you instead use "try...except KeyboardInterrupt:" or just "try..except:" it will work as you expect it to.

So there you have it...how to exit multithreaded Python programs using Ctrl-C.

Using Memcached as a Distributed Locking Service

One of the beauties of memcached is that while its interface is incredibly simple, it is so robust and flexible that it can do nearly anything.

As an example, I am currently working on a project where we need to do distributed locking. The reason for this is that we are running into what is generally know as the lost update problem. Basically, we need to read an object, update it, and write it back in a serialized fashion in order to ensure that no updates are lost. The easiest solution is to have a lock which must be acquired in order to do the read/update/write operations. Locks are great with a set of threads but once you break out of the context of a single machine, you need something distributed. While there are certainly other library or complex pieces of code that do this, I find this to be a pretty elegant solution for a set of nodes which have access to the same memcached servers.

Provided below is a class called MemcacheMutex which provides the standard acquire/release mutex interface but with the twist in that it is backed by memcached.

There is one big caveat. Since memcached is not persistent it is possible that the lock could get evicted from the cache. This could result in a case where P1 acquires the lock, the key gets evicted, then P2 is able to acquire the lock even though it has not been released by P1. If your memcached is doing tons of operations or you are holding onto the lock for really long periods of time then this could become an issue. In that case, you should use persistent version of memcached, memcachedb.