When we initially added Web Hooks to Kiln, we were excited: they made integrating Kiln with the outside world trivial. And we’ve seen a lot of uses of Web Hooks: we’ve used them internally to power our continuous integration system, Mortar, and to integrate pushes into HipChat to keep our teams aware of what’s going on in our repositories. And we’ve heard of other customers getting their own CI systems integrated, and getting integrated with their own chat systems. And some of our customers who had continuous integration systems or corporate chat environments used Web Hooks to integrate Kiln into…
…er, wait. Something’s not right here. I seem to be repeating myself and then saying the same thing again.
While Web Hooks’ flexibility is great, and we’ve used them for some novel things in the past (such as covering a dev’s entire screen with Growl-powered push notifications), most Kiln users just want a quick way to tie Kiln into their existing infrastructure; the flexibility of Web Hooks is overkill for that.
The first two are very obvious: Kiln can post updates on what’s going on with your source code right into your chat environment, making it really easy to keep your support staff up-to-date on what you’re up to.
Continuous Integration
Next up, a lot of our users wanted integration with a continuous integration system, and by “a continuous integration system”, most of our customers meant “TeamCity”. So now, Kiln can talk directly to TeamCity: just give it your TeamCity URL, and Kiln will automatically let you choose which build configuration to notify and for which repositories. Easy as pie.
Pivotal Tracker
Finally, one of the common questions we get is, “Can I use Kiln without FogBugz?” The answer has always been “yes,” but a lot of people are really asking, “Can I integrate Kiln with other bug trackers?” As of now, the answer is yes: Kiln now directly supports Pivotal Tracker.
Just tell Kiln your Pivotal API token, then use your normal Pivotal commit syntax, and Kiln will add links to your changesets to the relevant stories.
These are just the first couple baked-in web hooks we’re providing. We’d love to hear about any others you’d like us to add. Simply let us know at our StackExchange.
Note: This post has been temporarily removed because we discovered some errors in the survey methodology which don't reflect our usual standards. -Joel Spolsky
If you've reached this page, you're interested in the topic. We'd love you to take our survey on the state of version control so we can check our data against another (admittedly self-selecting) sample. Please start here: https://www.survs.com/survey/JEOWZO8N5W
One of the things that I've been arguing is a key advantage of Mercurial, and distributed version control systems in general, is that they let you iterate much more quickly than traditional systems. This stems from many areas—Mercurial's operations are generally faster, its superior branching ability means we can easily keep developing new features even while we’re in the release candidate phase, and its vastly better code auditing makes it a lot easier for us to find problems when they arise—but that’s all for naught if you don’t have anything to show for that faster cycle.
We do. Beginning this week, the Kiln team will be aiming to put out a new version of Kiln On Demand every two weeks. Some of these releases will be bigger than others, obviously—on any given release, we may only have bug fixes that are actually ready for deployment—but it means that the time between you reporting a bug and you getting a fix for it, or between a feature being completed and it showing up on your On Demand account, will be drastically reduced.
So how’s the new release cycle working for us so far? Well, if you logged into Kiln on Thursday, you’d find quite a few things that didn’t exist on Monday. Let me point to just five:
Managing lots of repositories just got a lot easier
One thing that we’ve noticed is that distributed version control systems tend to encourage you to create lots of repositories and forks and branches. That’s all well and good, but it can make trying to keep tabs on everything going on extremely complicated. Kiln 2.3 tried to solve this problem with a new “Related” tab for your repositories. While we felt that was a step in the right direction, we thought it wasn’t quite there. So I’m really happy to announce the new and improved “Related” tab:
Our goal with this interface is to make it easy for you to see exactly what the differences are between any two repositories. We’ve taken inspiration from the GitHub network graph and our Incoming and Outgoing tabs and married it with Kiln’s Electric DAG to bring you something that we believe is more powerful than either. Our hope is that this interface will make it very easy to see exactly what's merged in any given set of branches, letting you quickly get tabs on what all of your team’s working on, and what the status of any given release branch is.
One of the little-known features that launched with Kiln 2.2 were public repositories. A lot of companies, including Fog Creek, have a few components that they want to release to the public. Kiln 2.3 introduced the concept of making a project public, allowing you to share select code with the world without exposing the rest of your Kiln installation. This worked great, but discovering what was available in a Kiln install was a lot more difficult than it should’ve been, due to the fact that unauthenticated users were simply sent to the Kiln activity feed.
Beginning with Kiln 2.4, when an unauthenticated user hits a Kiln site with public repositories, they’ll be given a list of all the public repositories on that system. We think this is a lot more discoverable, and provides new users with a much better starting point for delving into what public sources you’ve made available. You can try out the new system at http://mirrors.kilnhg.com/, our dedicated repository mirrors site, where you can follow, browse and search the code for Mercurial, Python, Vim, Rails, Redis, and Django, all from the comfort of Kiln’s interface.
Code reviews got a heck of a lot better
We know that one of the major features a lot of you love about Kiln is its code review UI. While we generally really like the review system, we felt that there were a few key weaknesses we wanted to address:
Having a review with multiple users is hard. Kiln 2.3 and earlier suggested you create separate reviews for each person you wanted to involve. This made having a single coherent conversation with multiple developers unintuitive. While multideveloper reviews are rare at Fog Creek, we’ve found that they’re extremely common for a lot of our customers.
Users who had a lot of code reviews frequently could get lost in what was going—especially if they had multiple code reviews on the same changeset. Unless users thought to give the reviews unique titles at creation time, they’d end up with a pile of reviews with nearly identical titles.
Frequently, a review would spark a need for making a case in FogBugz. For example, a given line of code might not be wrong, but might make a developer realize that syadmin action was required. To make a case, you’d have to leave Kiln, go to FogBugz, make a case, tie it to the review, and then go back to Kiln. This was annoying.
Customers had trouble figuring out how to use the UI. Buttons that controlled the review were at the top, but the controls for adding comments were all at the bottom. This split made it easy for new users to get confused by the system.
We tried to address all these issues with a brand-new UI:
First, all of the actions you can use to modify a review are all located right at the bottom. We’ve tried to make it clear how to resolve and comment on a review, and made it clearer that you can resolve a review with a message.
Second, we’ve made it easy to make a case right from any comment in a review. If you see something that doesn’t really need to get fixed as part of a review, but that you realized was a bug that needed to get addressed elsewhere, or inspired a great feature idea, just click the “Make Case” button on the relevant comment. We’ll make a case in FogBugz and immediately tie it to the review, making it easy for you to keep everything tightly linked together.
Third, we made it easy to retitle a review at any point: just click the title. If you have a pile of nearly identical reviews, just give them better names to help disambiguate them.
Finally, we’ve made it much easier to have lots of developers contribute to a review by making it clearer that a review can be assigned to anyone, and by autosubscribing anyone who has seen a review. We’ve found this system makes it feasible for lots of developers to actually stay on-top of a review, while leaving the old one-on-one style in tact for those who like the current way things work, and making sure that some developer ultimately has responsibility for resolving the code review.
While we’ve been enjoying this new behavior internally, it’s brand-new, and a change from how Kiln’s traditionally worked, so we’d love your feedback. If you hate it, we’ll kill it. If you have great suggestions, we’ll add them. Just let us know.
Diffs are way more powerful
At the center of every source control system are diffs. They’re the be-all end-all at the end of the day: if the system can’t tell you what’s changed, then it’s not really doing you any good.
For Kiln 2.4, we dramatically increased the power of our diff system. You can now change tab size, toggle whether to show whitespace, ignore whitespace changes completely, and finally, in answer to one of our most-requested features, display diffs side-by-side.
Everyone knows that their repository is a DAG, but what’s not always well exposed is that your file history is a DAG, too.
Beginning in Kiln 2.4, we expose that DAG in the file history view, making it much clearer what the actual history of your file was.
And a lot more
Those are five of the things we're most proud of, but check out our full release notes if you're curious to take a look at everything we just shipped. Some of the things I didn't get a chance to talk about are:
You can tell Kiln to ignore files
Kiln now supports Mercurial bookmarks
Massive bfiles improvements
Search improvements
Plus, in case you missed it, we’ve added these features to Kiln since the new year:
Changesets related to a case are shown inline in FogBugz
Kiln allows jumping to the electric DAG directly from a given changeset
Reviews are now repository-independent
Reviews can be assigned to virtual users
Review comments can be threaded
And more!
Final Thoughts
We’re very excited about where Kiln is going to go this year. We really believe that having such a rapid release cycle will make us much better able to respond to your bugs and feature requests, and ensure that you get a steady stream of improvements throughout the year.
If you have any questions or concerns about any of this, don’t hesitate to contact us. We’re always listening.
Kiln 2.2 is now out! The team has been sprinting since the 2.0 release, and we're proud to introduce four new features: public repositories, subrepository support, streamlined history navigation, and improved code reviews.
Kiln's growth has been incredible since its launch just over a year ago. We hope the new additions will make managing your team's code even smoother, helping you spend more time coding and less time administering all of that awesome code.
Public Repositories
Most noticeably, we've added support for public repositories. We hope these prove to be an amazing resource for open-source projects and for companies sharing development libraries, APIs, and other code. Benjamin describes more of our thinking behind public repos in his earlier blog post.
Subrepositories and common and shared files
You can now fully browse subrepositories from the file view of the repository that they are included in. You can also see common and shared files and subrepositories, which makes updates to libraries much easier to manage.
Named branch browsing and tagged navigation
Kiln's search capability is a lifesaver, but we've also added two improvements to history browsing to find certain changesets:
First, with named branches, you can filter to just changesets on one particular named branch.
Second, to find a particular spot within a version history and visualize what was included in the release and what came after, the new "tags" drop-down menu lets you highlight the included changesets and jump straight to the tagged update in the electrified DAG.
Code review improvements
We've done a ton of work to revamp code reviews. The two changes that you'll likely notice the most are threaded comments, which help you give feedback clearly to specific parts of a review, and code snippets, which let you add formatted code inside your comments.
Questions? Comments? Want to see what other tricks Kiln can now do? Find the full documentation and 2.2 release notes on the Kiln Knowledge Exchange.
Want to see what Kiln can do for you? Sign-up for our 45-day free trial!
A lot of Kiln features come about because we have an itch to scratch. In fact, the very idea of Kiln arose because I wanted to be able to stop asking sysadmins to make new repositories for us and Tyler wanted a code review tool that didn’t stink. Then, our JSON API came about because customers wanted to be able to build on top of the great foundation that Kiln provided. Then, code search happened because it had become so easy to put everything into source control that we were having trouble finding anything. Finally, our increasing movement to allow code reviews to float among related repositories has been driven by a mix of internal and external desire to make them ever more flexible.
But there’s one feature that few customers have asked for, but that we feel is really important: making it very easy to share your source code with others. Open-source projects are extremely common these days. Many companies, including Fog Creek, share a lot of code with the broader community. While Kiln provides a whole host of tools that would make doing open-source development a lot easier, most projects didn’t want to use Kiln for one key reason: you needed to log in to Kiln to view source code. While that’s what you want 99.9% of the time, it can be a real bear when you’re just trying to share open-source code with your customers or the community at large.
We were initially nervous about how to implement this feature. We wanted to make it easy to share code with others, but we also wanted it to be impossible to accidentally make private code public. After playing with a few different designs, we settled on one that we believe makes a lot of sense, keeping your private code fully secure while making it really easy to share what you want to be public.
So we’re thrilled to announce a brand-new feature in Kiln 2.2: Public Projects. All repositories located in a public project can be accessed by anyone with a web browser; no log-in required. Visitors to your site can see a customized version of Kiln’s activity feed that lists only activity that’s been going on in public projects. They can subscribe to the RSS feed to stay abreast of updates, browse through your code, use Kiln’s phenomenal repository search, and clone the code to their local machines. Public projects are prominently labeled, making them hard to miss. And because you can only make things public at a project level, you never have to worry about accidentally exposing your work to theworld as you reorganize a project or work in the permissions pane. We really believe it’s the best of both worlds.
There’s no better way to explain this feature than to show it off. So, as of today, we’d also like to announce the Kiln Mirrors Project, located at http://mirrors.kilnhg.com. The Mirrors Project hosts mirrors of a variety of open-source projects, such as the Python programming language and Mercurial itself. It’s not only a convenient place to grab copies of your favorite open-source software; it’s also a really easy way to see what Kiln offers without even signing up for a trial. If you like what you see, and you’re an open-source project, don’t forget that Kiln has a free edition for up to two users, and we’re generally happy to expand that for genuine all-volunteer open-source projects. It’s very easy to get started.
"Just as with the beta release, it’s helpful to roll out the final release to our hosted customers in stages. We call this “leaking the release,” which sounds kind of gross, but I like to think of it like sugar water leaking out of a bird feeder rather than, …well, you can come up with an appropriately negative metaphor yourself. The birds love it. As before, this gradual process helps us to catch issues before they have a chance of affecting everyone. Now that we’re dealing with 2-3 orders of magnitude more customers, it’s much more likely that we’ll run into performance problems with the new code. One serendipitous benefit of this type of release was that buzz about the new versions progressively increased over the course of the month leading up to the full rollout."
Earlier this summer we decided to experiment with some new ways of getting out information about FogBugz and Kiln, and since we had the talented Lerone Wilson already doing some video work for us, making product videos was an easy choice.
A lot of the product videos were–what shall we say?–a little on the boring side. We wanted to make something a little better than you usually see and avoid all the usual mistakes, like having your bored product manager mumble his lines through his nose.
We ran through our actor options here, but most of us sounded like Keanu Reeves, only dead.
Since we're in New York we put out a call for actors and got dozens of submissions from some really talented people, and found a guy we liked. Only later did we discover that he's a veteran of the Onion News Network. (NSFW – Swearing). He was a lot of fun to work with.
That's it really. A few days of filming and we had video covering all of the major parts of FogBugz and our new product Kiln. We had a half day of time left that we had paid for so we added some "Howto" videos describing common tasks in FogBugz.
We hope these are useful! We'd love to hear what you think, and if there are other topics you would like us to cover, let us know.
Recently, at work, we started using MiniRedis as a lightweight store for some job queuing on Kiln's backend. We had originally planned on using the full Redis, but because we have to deploy licensed Kiln on Windows, we had to come up with our own solution.
So far, it's been working very well for us. The Redis command set is pretty small and very straightforward, which makes it easy to clone. The only annoyance we've run into is that the stable command line interface (CLI) for Redis only speaks the 1.x protocol, while MiniRedis only speaks the 2.0 version. The redis CLI also does not run on Windows. This is a bit of a problem, since we're still working on our queuing system and we need to do testing.
To get around not having a client to use, Ben would telnet in to the MiniRedis port and type out the commands manually. It ended up looking a bit like this:
I, on the other hand, would fire up Python and, using the redis-py library (which is a very nice client library), issue commands directly from there. Neither option was very convenient.
So one day, tired of having to do all the imports and set up the connection, I decided to put together a CLI using Python.
import cmd
class RedisCli(cmd.Cmd, object):
pass
if __name__ == '__main__':
RedisCli().cmdloop()
The basic CLI is very simple. Python's cmd module takes care of all the hard parts for you. If you run this script, you'll get a prompt (Cmd) that supports one command: help. Unfortunately, that's all you have. You can't even exit gracefully. So that's the first thing I added:
The cmd module lets you define commands by writing a do_foo() method which takes the line the user typed in. The above code gives you the exit command, and also makes EOF (Unix: Ctrl-D, Windows: Ctrl-Z) exit for you. That's helpful, but it doesn't really add much in terms of actual functionality. For that, we import the Redis client libraries
This is nice, because help will now list get and set. But Redis has many more commands available. One option would be to continue adding all the Redis commands until you had the full set specified and properly parsing the command line. That's pretty time consuming, brittle, and just plain boring. Personally, I don't have the patience.
Fortunately, the cmd module has a default() method which is called for any unrecognized functions. That means we can get rid of do_get and do_set and replace them with:
def default(self, line):
parts = line.split()
print getattr(self.redis, parts[0])(*parts[1:])
Huh? Let me explain: getattr() takes an object and an attribute name, and returns that attribute if it exists. To illustrate, calling getattr(obj, 'foo') is the same as calling obj.foo. In this case, we're assuming the user typed in one of the functions defined in the Redis object. We use getattr to get that function, and then we pass the rest of the arguments to it using Python's *args syntax. This sidesteps the problem of not knowing how many arguments the commands take.
Unfortunately, this approach is prone to errors. For example, the user can type in __init__, which will call the Redis object's constructor again, overwriting our connection. Someone could also try to call foobarbazbat, which does not exist in the Redis object and will throw an error. Lastly, we've also lost our list of commands when you type help.
To fix this, we're going to have to do some spelunking in the Redis object. Fortunately, Python's dir() function returns all of the Redis object's attributes. We can then iterate over them, filter out any that start with an underscore (Python's convention for private attributes) and make sure they're callable. We then use setattr to create a function in our own class that calls into the Redis object and prints the result.
def __init__(self, host, port):
super(RedisCli, self).__init__()
self.redis = Redis(host=host, port=port)
for name in dir(self.redis):
if not name.startswith('_'):
attr = getattr(self.redis, name)
if callable(attr):
setattr(self.__class__, 'do_' + name, self._make_cmd(name))
@staticmethod
def _make_cmd(name):
def handler(self, line):
parts = line.split()
print getattr(self.redis, name)(*parts)
return handler
Notice here that _make_cmd is creating a new function inside of it, and returning that function so we can set the do_foo of our own class to the function that calls self.redis.foo(). Likewise for any callable function in the Redis class.
Now if we type help on our command line, we'll get a list of all functions in the Redis object. Also, if we try to access anything private, like __init__, we'll be told that syntax is unknown. This also means that our default() method is no longer necessary, since we've already enumerated everything that we could possibly call on the Redis object.
You'll notice, however, that help lists all of the functions as "Undocumented". It would be really nice if we could also get documentation for each of these commands. Now that we can easily list all of the available commands, we could write the documentation ourselves by specifying a help_foo() function for each command. However, this is boring and like I said before, I don't have that kind of patience. It also turns out that writing our own documentation would be redundant, as the authors of our Redis client library have done a good job documenting each function in the form of docstrings:
def get(self, name):
"""
Return the value at key ``name``, or None of the key doesn't exist
"""
Python takes these docstrings pretty seriously. In fact, they become an attribute of the function itself, called __doc__. This is great for us, because it means we can pull those docstrings into our CLI and make them documentation for our commands. We use the same method as before to dynamically add help_foo() methods to our own class for every function that has a docstring:
doc = (getattr(func, '__doc__', '') or '').strip()
if doc: # Not everything has a docstring
setattr(self.__class__, 'help_' + name, self._make_help(doc))
@staticmethod
def _make_help(self, doc):
def help(self):
print doc
return help
So now if we type help, we'll get a list of "Documented" and "Undocumented" commands. If we type help get, it'll tell us "Return the value at key “name“, or None of the key doesn't exist." Awesome!
This is getting to be a useful little CLI. In addition to having a documented list of all commands available, we also get tab completion for our commands, so if we type l<TAB>, we get the list of all list commands. (On Windows, you'll need the pyreadline module installed for this to work.) But what if we could also autocomplete our keys? Some of our keys get pretty long, and typing them out is a pain. We could define a complete_foo() method for each of our functions, but all we're ever going to be completing are the keys, so we can just use the completedefault, which is a catchall completion, to grab our keys for us.
Once we've added this, we can type get bar<TAB> and we'll get all of the keys that start with bar.
We're in the home stretch now. Just to make things a little nicer, let's modify the prompt and the intro message so the user knows they're in the Redis CLI:
def __init__(self, host, port):
...
self.prompt = '(Redis) '
self.intro = '\nConnected to Redis on %s:%d' % (host, port)
And finally, because this is a tool we want to be able to use with different servers, let's add the ability to specify a host (-h or --host) and port (-p or --port).
import getopt
if __name__ == '__main__':
opts = dict(getopt.getopt(sys.argv[1:], 'h:p:', ['host=', 'port='])[0])
host = opts.get('-h', None) or opts.get('--host', 'localhost')
port = int(opts.get('-p', None) or opts.get('--port', 12345))
RedisCli(host=host, port=port).cmdloop()
To finish, we just add some error checking so we don't get bailed out with an exception if we happen to make a typo. Here's the final script:
import cmd
import getopt
import sys
from redis import Redis
from redis.exceptions import ConnectionError, ResponseError
class RedisCli(cmd.Cmd, object):
def __init__(self, host, port):
super(RedisCli, self).__init__()
self.redis = Redis(host=host, port=port)
self.prompt = '(Redis) '
self.intro = '\nConnected to Redis on %s:%d' % (host, port)
for name in dir(self.redis):
if not name.startswith('_'):
attr = getattr(self.redis, name)
if callable(attr):
setattr(self.__class__, 'do_' + name, self._make_cmd(name))
doc = (getattr(attr, '__doc__', '') or '').strip()
if doc:
doc = (' ' * + doc # Fix up the indentation
setattr(self.__class__, 'help_' + name, self._make_help(doc))
try:
# Test the connection. It doesn't matter if 'a' exists or not.
self.redis.get('a')
except ConnectionError, e:
print e
sys.exit(1)
@staticmethod
def _make_cmd(name):
def handler(self, line):
parts = line.split()
try:
print getattr(self.redis, name)(*parts)
except Exception, e:
print 'Error:', e
return handler
@staticmethod
def _make_help(doc):
def help(self):
print doc
return help
def completedefault(self, text, line, start, end):
return self.redis.keys(text + '*').split()
def do_exit(self, line):
return True
do_EOF = do_exit
def emptyline(self):
pass # By default, cmd repeats the command. We don't want to do that.
if __name__ == '__main__':
opts = dict(getopt.getopt(sys.argv[1:], 'h:p:', ['host=', 'port='])[0])
host = opts.get('-h', None) or opts.get('--host', 'localhost')
port = int(opts.get('-p', None) or opts.get('--port', 12345))
RedisCli(host=host, port=port).cmdloop()
And there we have it. A full-featured, robust, well-documented Redis CLI in about 60 lines of code. For comparison, the C version is over 500 lines of code, and has no help documentation or code completion.
The best part, though, is that this doesn't really know anything about Redis at all. The only parts that are aware of Redis are __init__(), which sets up the Redis object, and completedefault(), which gets our keys. That means that you could easily adapt this script to be a CLI on top of any client library you have.
This is a call for beta testers interested in trying out our new version control product: Kiln (http://www.kilnhg.com/). Mercurial is the version control system under the hood, so if you've got FogBugz and wanted to try out a DVCS, this is a great opportunity. If you just want to experience the awesomeness of version control, code review, and bug tracking all working together seamlessly, this is a great opportunity.
I have to mention that this is a beta. There will be some bugs during installation, and possibly after. The good news is that Kiln has been running for several months and hosting production code for a bunch of people in the On Demand environment. We think the risk is pretty low, but as software developers you know there is no way for us to guarantee that there won't be bugs.
We are specifically looking for licensed (install it on your server) testers, although you can use the same form if you're a FogBugz on Demand customer (the more the merrier!):
Licensed Kiln will only run on Windows 2003/2008 Server with SQL Server 2005/2008/2008 Express. Folks running other operating systems or database servers are out of luck for the time being.
ONE MORE THING: This release also introduces a new version of FogBugz. There have been a few UI changes to support having FogBugz integrate with other applications. If you don't want to test Kiln, but would be willing to test the new version of FogBugz, send me an email to jason (at) fogcreek (dot) com.
+ doc # Fix up the indentation
setattr(self.__class__, 'help_' + name, self._make_help(doc))
try:
# Test the connection. It doesn't matter if 'a' exists or not.
self.redis.get('a')
except ConnectionError, e:
print e
sys.exit(1)
@staticmethod
def _make_cmd(name):
def handler(self, line):
parts = line.split()
try:
print getattr(self.redis, name)(*parts)
except Exception, e:
print 'Error:', e
return handler
@staticmethod
def _make_help(doc):
def help(self):
print doc
return help
def completedefault(self, text, line, start, end):
return self.redis.keys(text + '*').split()
def do_exit(self, line):
return True
do_EOF = do_exit
def emptyline(self):
pass # By default, cmd repeats the command. We don't want to do that.
if __name__ == '__main__':
opts = dict(getopt.getopt(sys.argv[1:], 'h:p:', ['host=', 'port='])[0])
host = opts.get('-h', None) or opts.get('--host', 'localhost')
port = int(opts.get('-p', None) or opts.get('--port', 12345))
RedisCli(host=host, port=port).cmdloop()
