I have added MemcacheRing to hash_ring 1.1. This class encapsulates python-memcache's Client class so keys are distributed using consistent hashing.
You can read more about consistent hashing in two other posts:
Usage of MemcacheRing
from hash_ring import MemcacheRing mc=MemcacheRing(['192.168.0.23:11212','192.168.0.66:11212']) mc.set('hello','world') printmc.get('hello')
Implementation:
import memcache import types from hash_ring import HashRing class MemcacheRing(memcache.Client): """Extends python-memcache so it uses consistent hashing to distribute the keys. """ def __init__(self, servers, *k, **kw): self.hash_ring = HashRing(servers) memcache.Client.__init__(self, servers, *k, **kw) self.server_mapping = {} for server_uri, server_obj in zip(servers, self.servers): self.server_mapping[server_uri] = server_obj def _get_server(self, key): if type(key) == types.TupleType: return memcache.Client._get_server(key) for i in range(self._SERVER_RETRIES): iterator = self.hash_ring.iterate_nodes(key) for server_uri in iterator: server_obj = self.server_mapping[server_uri] if server_obj.connect(): return server_obj, key return None, None
Other than this, I am also using the idea of consistent hashing to implement a distributed hash map. This may also become open-source some time in the future.