July 21st, 2010 by Tyler Hicks-Wright
Cheeky Python: A Redis CLI
Recently, at work, we started using MiniRedis as a lightweight store for some job queuing on Kiln's backend. We had originally planned on using the full Redis, but because we have to deploy licensed Kiln on Windows, we had to come up with our own solution.
So far, it's been working very well for us. The Redis command set is pretty small and very straightforward, which makes it easy to clone. The only annoyance we've run into is that the stable command line interface (CLI) for Redis only speaks the 1.x protocol, while MiniRedis only speaks the 2.0 version. The redis CLI also does not run on Windows. This is a bit of a problem, since we're still working on our queuing system and we need to do testing.
To get around not having a client to use, Ben would telnet in to the MiniRedis port and type out the commands manually. It ended up looking a bit like this:
I, on the other hand, would fire up Python and, using the redis-py library (which is a very nice client library), issue commands directly from there. Neither option was very convenient.
So one day, tired of having to do all the imports and set up the connection, I decided to put together a CLI using Python.
import cmd class RedisCli(cmd.Cmd, object): pass if __name__ == '__main__': RedisCli().cmdloop()
The basic CLI is very simple. Python's
cmd module takes care of all the hard parts for you. If you run this script, you'll get a prompt
(Cmd) that supports one command:
help. Unfortunately, that's all you have. You can't even exit gracefully. So that's the first thing I added:
class RedisCli(cmd.Cmd, object): def do_exit(self, line): return True do_EOF = do_exit
cmd module lets you define commands by writing a
do_foo() method which takes the line the user typed in. The above code gives you the
exit command, and also makes
EOF (Unix: Ctrl-D, Windows: Ctrl-Z) exit for you. That's helpful, but it doesn't really add much in terms of actual functionality. For that, we import the Redis client libraries
from redis import Redis
initialize the connection
class RedisCli(cmd.Cmd, object): def __init__(self, host, port): self.redis = Redis(host=host, port=port)
and add some of the Redis commands
def do_get(self, line): print self.redis.get(line) def do_set(self, line): key, value = line.split() print self.redis.set(key, value)
This is nice, because
help will now list
set. But Redis has many more commands available. One option would be to continue adding all the Redis commands until you had the full set specified and properly parsing the command line. That's pretty time consuming, brittle, and just plain boring. Personally, I don't have the patience.
cmd module has a
default() method which is called for any unrecognized functions. That means we can get rid of
do_set and replace them with:
def default(self, line): parts = line.split() print getattr(self.redis, parts)(*parts[1:])
Huh? Let me explain:
getattr() takes an object and an attribute name, and returns that attribute if it exists. To illustrate, calling
getattr(obj, 'foo') is the same as calling
obj.foo. In this case, we're assuming the user typed in one of the functions defined in the
Redis object. We use
getattr to get that function, and then we pass the rest of the arguments to it using Python's
*args syntax. This sidesteps the problem of not knowing how many arguments the commands take.
Unfortunately, this approach is prone to errors. For example, the user can type in
__init__, which will call the
Redis object's constructor again, overwriting our connection. Someone could also try to call
foobarbazbat, which does not exist in the
Redis object and will throw an error. Lastly, we've also lost our list of commands when you type
To fix this, we're going to have to do some spelunking in the
Redis object. Fortunately, Python's
dir() function returns all of the Redis object's attributes. We can then iterate over them, filter out any that start with an underscore (Python's convention for private attributes) and make sure they're callable. We then use
setattr to create a function in our own class that calls into the
Redis object and prints the result.
def __init__(self, host, port): super(RedisCli, self).__init__() self.redis = Redis(host=host, port=port) for name in dir(self.redis): if not name.startswith('_'): attr = getattr(self.redis, name) if callable(attr): setattr(self.__class__, 'do_' + name, self._make_cmd(name)) @staticmethod def _make_cmd(name): def handler(self, line): parts = line.split() print getattr(self.redis, name)(*parts) return handler
Notice here that
_make_cmd is creating a new function inside of it, and returning that function so we can set the
do_foo of our own class to the function that calls
self.redis.foo(). Likewise for any callable function in the
Now if we type
help on our command line, we'll get a list of all functions in the
Redis object. Also, if we try to access anything private, like
__init__, we'll be told that syntax is unknown. This also means that our
default() method is no longer necessary, since we've already enumerated everything that we could possibly call on the
You'll notice, however, that help lists all of the functions as "Undocumented". It would be really nice if we could also get documentation for each of these commands. Now that we can easily list all of the available commands, we could write the documentation ourselves by specifying a
help_foo() function for each command. However, this is boring and like I said before, I don't have that kind of patience. It also turns out that writing our own documentation would be redundant, as the authors of our Redis client library have done a good job documenting each function in the form of docstrings:
def get(self, name): """ Return the value at key ``name``, or None of the key doesn't exist """
Python takes these docstrings pretty seriously. In fact, they become an attribute of the function itself, called
__doc__. This is great for us, because it means we can pull those docstrings into our CLI and make them documentation for our commands. We use the same method as before to dynamically add
help_foo() methods to our own class for every function that has a docstring:
doc = (getattr(func, '__doc__', '') or '').strip() if doc: # Not everything has a docstring setattr(self.__class__, 'help_' + name, self._make_help(doc)) @staticmethod def _make_help(self, doc): def help(self): print doc return help
So now if we type
help, we'll get a list of "Documented" and "Undocumented" commands. If we type
help get, it'll tell us "Return the value at key “name“, or None of the key doesn't exist." Awesome!
This is getting to be a useful little CLI. In addition to having a documented list of all commands available, we also get tab completion for our commands, so if we type
l<TAB>, we get the list of all list commands. (On Windows, you'll need the pyreadline module installed for this to work.) But what if we could also autocomplete our keys? Some of our keys get pretty long, and typing them out is a pain. We could define a
complete_foo() method for each of our functions, but all we're ever going to be completing are the keys, so we can just use the
completedefault, which is a catchall completion, to grab our keys for us.
def completedefault(self, text, line, start, end): return self.redis.keys(text + '*').split()
Once we've added this, we can type
get bar<TAB> and we'll get all of the keys that start with
We're in the home stretch now. Just to make things a little nicer, let's modify the prompt and the intro message so the user knows they're in the Redis CLI:
def __init__(self, host, port): ... self.prompt = '(Redis) ' self.intro = '\nConnected to Redis on %s:%d' % (host, port)
And finally, because this is a tool we want to be able to use with different servers, let's add the ability to specify a host (
--host) and port (
import getopt if __name__ == '__main__': opts = dict(getopt.getopt(sys.argv[1:], 'h:p:', ['host=', 'port='])) host = opts.get('-h', None) or opts.get('--host', 'localhost') port = int(opts.get('-p', None) or opts.get('--port', 12345)) RedisCli(host=host, port=port).cmdloop()
To finish, we just add some error checking so we don't get bailed out with an exception if we happen to make a typo. Here's the final script:
import cmd import getopt import sys from redis import Redis from redis.exceptions import ConnectionError, ResponseError class RedisCli(cmd.Cmd, object): def __init__(self, host, port): super(RedisCli, self).__init__() self.redis = Redis(host=host, port=port) self.prompt = '(Redis) ' self.intro = '\nConnected to Redis on %s:%d' % (host, port) for name in dir(self.redis): if not name.startswith('_'): attr = getattr(self.redis, name) if callable(attr): setattr(self.__class__, 'do_' + name, self._make_cmd(name)) doc = (getattr(attr, '__doc__', '') or '').strip() if doc: doc = (' ' * 8) + doc # Fix up the indentation setattr(self.__class__, 'help_' + name, self._make_help(doc)) try: # Test the connection. It doesn't matter if 'a' exists or not. self.redis.get('a') except ConnectionError, e: print e sys.exit(1) @staticmethod def _make_cmd(name): def handler(self, line): parts = line.split() try: print getattr(self.redis, name)(*parts) except Exception, e: print 'Error:', e return handler @staticmethod def _make_help(doc): def help(self): print doc return help def completedefault(self, text, line, start, end): return self.redis.keys(text + '*').split() def do_exit(self, line): return True do_EOF = do_exit def emptyline(self): pass # By default, cmd repeats the command. We don't want to do that. if __name__ == '__main__': opts = dict(getopt.getopt(sys.argv[1:], 'h:p:', ['host=', 'port='])) host = opts.get('-h', None) or opts.get('--host', 'localhost') port = int(opts.get('-p', None) or opts.get('--port', 12345)) RedisCli(host=host, port=port).cmdloop()
And there we have it. A full-featured, robust, well-documented Redis CLI in about 60 lines of code. For comparison, the C version is over 500 lines of code, and has no help documentation or code completion.
The best part, though, is that this doesn't really know anything about Redis at all. The only parts that are aware of Redis are
__init__(), which sets up the
Redis object, and
completedefault(), which gets our keys. That means that you could easily adapt this script to be a CLI on top of any client library you have.
Cross posted from http://hicks-wright.net/blog/cheeky-python-a-redis-cli/.