Link Box

March 26, 2008 in Programming
Tags: cache, consistent hashing, distributed, libevent, memcached

Ok, so here is the first of what I hope to be a new weekly habit – my ‘Link Box’. Since I spare far too much time reading other people’s blogs than posting to my own, I thought I might share some of the more interesting posts.

The first thing to catch my eye this week was a post from Spiteful about Consistent Hashing. I’d never really thought about how load balancing works, thinking modulus hashing was a commonly used solution. What I like about the idea of consistent hashing is how you can have resources with different capabilities, and it seems to handle a resource dropping out pretty gracefully.

Tom also mentioned memcached – specifclly how Last.fm have modified it to support consistent hashing for their caching requirements. I think memcached is a brilliant piece of software, and I’m itching to apply it in my next project (when I think one up!).

I had a look at some more distributed hash tables, and stumbled across Cacheman, a distributed hashtable written for Windows – in .NET I believe (or the client library is at least). Being a bit of a .NET (s/.NET/M$/) skeptic, I was pretty impressed at how Cacheman appears to be. After talking to a fellow student at my university who is doing his final year project in .NET (and hearing about how versatile it is), I’m going to have to take some time to learn C# or Visual C++ 2008. I’m hoping Sriram will release the source soon so that I can take a peek.

After reading up a bit more on memcached and how it works, I saw that it uses libevent – which I don’t think I’ve ever encountered before. Let me put it like this: libevent is crazy cool. It is basiclly an event-based API for file descriptors. From it’s website:

“The libevent API provides a mechanism to execute a callback function when a specific event occurs on a file descriptor or after a timeout has been reached. Furthermore, libevent also support callbacks due to signals or regular timeouts. “

I’m going to have a play with this tonight and see if I can make some crazy event-driven server. Maybe something like Third Rail’s example of how to write a simple web server using libevent in only 40 lines of code!

So there you have it. A weeks worth of interesting links – one for everyday I’ve gone to work and pretended to be busy. I’ll try and prepare one for next week, if you are interested.