Archive for the ‘Kiln’ Category

The State of the Kiln

January 9th, 2012 by Benjamin Pollack

As the new year opens, I thought it was high time to look back at the last year of Kiln and see what we’ve accomplished, then turn forward and discuss what we’re working on for the next twelve months.

Hold onto your hats. There’s a lot to like.

The Year in Review

The Kiln Dodo enjoying a partyWe had a tough act to follow last year. In 2010, we launched Kiln On Demand and Kiln for Your Server, shipped piles of new features, and found ourselves home to tens of thousands of users’ source code. But Kiln was new at the time; most of those of you who came to us were already familiar with distributed source control in one way or another. Kiln’s mission is to bring distributed version control to as many people as possible, and that means appealing to more than just early adopters.

To make that happen, we made a giant list of what was keeping people from jumping to Kiln, and we spent nine months fixing every issue one by one. We rewrote the review system to be more user-friendly (by overhauling the UI and making it easier to make cases and turning reviews into multiparty discussions instead of one-on-one chats). We helped Kiln integrate tightly with lots of other services so you could keep your existing infrastructure. We allowed configurable diffs, we taught Kiln how to display images in your repositories, we beefed up the API, we contributed largefiles (based on kbfiles, our extension that makes it trivial to have large binary files in Mercurial) back to the main Mercurial project, and we added a full-blown groups permission system so you can have large Kiln installs without getting lost in a quagmire of user management.

The result? Kiln 2.7, which we think gets even those who don’t have a lot of experience with DVCS easily using distributed version control, rather than getting lost in a muddle of tool arcana.

What’s to Come

“First nine months,” I hear you say. “Fascinating. That might be an impressive list. But the year is twelve months long, so what have you done for us lately?”

We have a strong rule against publishing road-maps for products, for a simple reason: we don’t like to disappoint you. We hate promising you’ll have some awesome new feature in Kiln 45.7, like telekinetic abilities, and then smashing your hopes most excellently by announcing we’ve had to delay them until 45.8. Instead, we prefer to surprise you each release with piles of new goodies. A kind of monthly Christmas for Kiln, if you will.

Of course, the flip-side is that we know you can feel abandoned when we go dark for awhile, even if we’ve got a really good reason for doing that. And I think we’ve been too quiet lately, so some of you are wondering if we’ve been run over by a bus.

Good news: we’re still very much here! And we’re still working hard. Unfortunately, our current projects are harder than our old ones, so we haven’t had much to show for it for a few months.

To fix that, I want to welcome you behind the curtain, and introduce you to three features coming your way in the next few months: the Kiln Client for Mac OS X; SSH support; and vastly improved search and general performance across the entire app.

Kiln Client for OS X

I admit it: Kiln may run on Windows, and I may love writing code for .NET, but at home, everything I own runs OS X. So it frustrates me that the Kiln Client is only available for Windows.

That’ll be changing shortly. Coming soon to a Mac near you: Kiln Client for OS X.

A screenshot of TortoiseHg for OS X

While it looks and works similarly to the Windows client, since both are based on the excellent TortoiseHg, we’ve taken the time to make it work like a real Mac app. You install it by drag-and-drop, and it takes care of the rest. It’ll automatically put Mercurial on your path, it’ll automatically update itself when there are changes, and otherwise behave just like a good Mac citizen.

SSH Support

While we love Mercurial over HTTP, the simple truth is that it’s not perfect for everyone. HTTP-based pushes put you into HTTP timeout hell, where you have to make sure your timeouts are correct at every level of your network (load balancer, web server, web site in IIS, and so on). For Kiln On Demand, that’s okay, because we control the full infrastructure. For our licensed customers, this can be a serious problem.

Good news: SSH suffers none of these issues, and Kiln is adding SSH support. And not just for On Demand; it’ll be available for licensed customers as well, as a single, easy-to-use, turnkey solution.

Improved General Search

Search in Kiln is a big deal, and we love it and we know you love it, but we want to be even faster, and we want it to be better with non-Windows-compatible file names.

So we’re rebuilding Kiln’s search story around Elastic Search, an excellent NoSQL search solution. We’re currently still heavily in the development stage, but already, our search times are down from a second or two for very large Kiln installations to just a couple of milliseconds. We think this will be a big deal for our licensed customers, and a huge deal for our On Demand customers.

We’ll have this work on Kiln On Demand in a couple of months, and for Kiln licensed customers about a month later.

So we believe there’s a lot to love in the coming months for Kiln. And we hope that this roadmap of what we’re doing, and the rough timeline we plan to do it in, will get you as excited as we are.

In the meantime, happy Kilning.

FogBugz and Kiln Build and Release Report Card

November 3rd, 2011 by Rock Hymas

My wife and I are a couple months into our first year of homeschooling, and we’ve been discussing how the kids are doing with it, and the challenges we’ve faced as their teachers and parents. Those discussions have led us to make some important changes, in the same way that a school report card can get parents and children to make changes that lead to more learning and happier kids. In that spirit, I offer here a report card on the build and release management work we’ve been doing on FogBugz and Kiln [1] over the last 12-18 months.

About a year ago, I was in the middle of writing a series of posts about the practices of build and release management. Many of those practices came from looking at what we were doing with FogBugz and Kiln and identifying what worked, and what didn’t. Others came from what I saw and experienced while working within the Microsoft Office organization, and still more came from discussions within our team at the Creek. Though in some areas we were doing things right, most of the posts were a description of what our existing build and release (non-)system was doing utterly wrong and what I hoped we could change to fix it. So the series was often written in a spirit of hope that we would one day live up to the ideal.

In fact, you could take many of the posts I wrote last fall and read them as the opposite of how things existed at Fog Creek in the summer of 2010. So, in the spirit of Jason Cohen’s recent BoS talk about honesty, I’m going to be brutally honest about how we were doing and how we are now doing.

First, how we were doing. Here’s the report card for June of 2010, divided into subjects, of course.

Report Card for June 2010
Subject Grade
Fast builds/deploys

Deploying upgrades to www.fogcreek.com takes over an hour

Other builds and deployments often take much longer than they need to

Upgrades to FogBugz or Kiln often involve downtime of a few seconds to half an hour or more

D
No hard dependencies

Checked out source code is tied to absolute paths in some cases

The build and deployment scripts are full of hard coded paths and other hard dependencies

F
Single commands (automate, automate, automate)

We can’t run most builds with a single command

Many of the build and deployment scripts are combined

Deploying a new version takes multiple steps

Different scripts are used for dev builds vs official builds

Different configurations of the build use different build scripts

There is no tooling to make it easy to setup your dev system

There are no scripts to configure a new server for our hosted environment, or a new dev box for development

D
Staging Environment

We have no staging environment for our hosted FogBugz and Kiln products

F
Continuous Integration

There is no continuous integration builds or tests running

F
Scripts @ production quality

The scripts are old, unmaintained, with lots of cruft

Many built binaries are stored directly in the VCS

D
Onboarding new developers

There is no easy way for devs to build installers

There are no build scripts that do incremental builds

D
Insight into builds and releases

Old builds are not saved for most official configurations

We aren’t tracking build statistics like failures, build quality, etc.

Logs of builds are not systematically kept and stored anywhere

Build failures do not automatically notify anyone, nor do successful builds

There are three different types of version numbers for our three different deployment targets

F

All in all, things were a mess. ”Good grief!”, I hear you cry. “So what have you done about these issues? Anything? What are you planning to do?” I’m glad you asked.

My initial goal in the role of build and release manager was to eliminate the position. I wanted to get things running smoothly, and spread the knowledge and ownership of the problems through the team, so that I could go back to doing product development. As such, a decent amount of the progress described here was done by other great developers here at the Creek. Back in January (when I joined FogBugz as the team lead), I thought I’d gotten about halfway through the work required for that to happen. But, of course, new ideas come up constantly, so maybe it never ends.

Since January, I haven’t been able to do as much. In addition to my small efforts, other great Creekers have picked up some of the slack. Here’s our report card for October 2011.

Report card for October 2011
Subject Grade
Fast builds/deploys

A new, fast build machine

Reduced web site deployments from taking ~80 minutes to taking ~8 minutes

FogBugz and Kiln deployments now faster, because they don’t also rebuild the product

Upgrades to FogBugz or Kiln still involve downtime of a few seconds to half an hour or more

C+
No hard dependencies

Removed all the absolute paths

Removed many other hard dependencies

Removed some remaining hard dependencies that occasionally caused build failures

B
Single commands (automate, automate, automate)

Mortar, an internal website for kicking off builds and deployments

Lots of help on this one from Benjamin at bitquabit

Separated build and deploy scripts for hosted builds

Some of the build and deployment scripts are still combined

Deploying a new version takes multiple steps

Different scripts are still used for dev builds vs official builds

Different configurations of the build still use different build scripts, but some combining work has been done

There is still no tooling to make it easy to setup your dev system

There are still no scripts to configure a new server for our hosted environment, or a new dev box for development

C+
Staging Environment

Still no progress on the staging environment

F
Continuous Integration

Continuous integration builds exist, mostly just building the products

Basic tests have been added to some continuous integration builds

B
Scripts @ production quality

Removed many checked in binaries, building them instead

Many scripts converted from FinalBuilder to Python, but a few still remain

B-
Onboarding new developers

Branch builds, for any branch a dev wants to setup

bf, a better tool for building FogBugz incrementally on dev machines

Aaron kick-started this and continues to be the driving force

There is still no easy way for devs to build installers

D
Insight into builds and releases

Saving old hosted builds

Moved to a single type of version number no matter the config

Build logs for all official builds

Cases filed on build failure

Notifying build owners when builds complete, successfully or not

We aren’t tracking build statistics like failures, build quality, etc.

B-

As you can see, there is still plenty of work to do. The glaring failure in both report cards is our lack of a staging environment. We’re now working on making that a reality, with high hopes that it will help us start catching a new class of bugs before they make it into our customers hands. As part of that work, we’ll also be able to work on:

  • Scripts for configuring our servers, by type
  • No-downtime upgrades
  • Move our internal dogfooding to the staging environment, for better consolidation and testing of deployment scripts

Other possibilities for further work should be pretty clear from the report card, but the staging environment is the key focus when it comes to improving our builds and releases.

One note about the culture for both FogBugz and Kiln. We’ve learned a ton in the last couple years by seeing at close range what can be done in greenfield development on products like Trello, or teams that actively maintain and improve their processes for builds and releases, like StackOverflow. Obviously, when working with a legacy product like FogBugz, the costs of implementing certain practices may vary a lot, and we have to take that into consideration, but the inspiration that comes from seeing what is possible keeps us working to make things better.

When I look at this report card, I feel like my oldest son a little bit. He’s our perfectionist, always expecting to get everything right the first time through, and pretty disappointed when that doesn’t happen. We’re hoping that some of the changes we make in our homeschooling will help him relax a bit so he can learn more freely. And yes, it’s somewhat humbling to look at our build and release report card and admit how far we have to go. But I have high hopes that in another six months we’ll have improved our grades again.

 

[1] What about Trello?

Trello has the awesome advantage of being greenfield development. As such, they’ve really started from the ground up by doing the right things when it comes to builds and releases. They frequently release midday with no downtime, they have a nice staging environment, their releases take almost no time at all. They have run into some interesting load challenges with updating their clients after an upgrade, but I’ll let the Trello team talk about that in their own sweet time.

The End of an Internship

August 24th, 2011 by Benjamin Pollack

One thing that really made a huge difference to me when I was an intern is that I knew that what I was working on was actually going to ship to actual customers.  Not at some distant point in the future. Now.  Copilot started going live piecemeal about eight weeks after we started, which meant that we actually got to see customers react to what we’d made while we were still in our internship. I’ve mentioned before that this is one of the reasons that I got completely addicted to Fog Creek. But for the past two summers, we haven’t really done that with our interns. Yes, they wrote major code, and yes, it shipped to customers…but only months after the interns had already gone home.

Interns party at the beach on their last day

This year, on the Kiln team, we changed that: three times this summer—as part of Kiln 2.5.119 and Kiln 2.5.166 for Kiln On Demand, and in Kiln 2.5.177 for customers with on-site Kiln installations—we released intern code. Every last intern on the Kiln team has gotten to see their code actually running in the production Kiln environment before they’ve gone home. Some interns even shipped new code with all three releases, getting feedback from real, live, breathing customers each time.

But these are interns, I hear you say. So what? We’re talking dinky features here, right?

You tell me. They made Kiln show you images in diffs and when browsing repositories. They got Kiln running the newest version of Mercurial from the top of the stack to the bottom, including landing piles of patches in TortoiseHg and a few in Mercurial itself. They enhanced Kiln’s code search to allow filtering on files by globs and regexes, allowed greater flexibility in subscribing to Kiln’s activity feed, and dramatically improved Kiln’s ability to work with continuous integration systems. Not content with that, they added lots of UI refinement in changeset/bug integration, and gave back to the Mercurial community by taking our proprietary kbfiles extension, which makes it trivial to work with large files in a DVCS, and turning it into the generic largefiles extension for Mercurial that will hopefully be included in an upcoming Mercurial release.

These are huge changes. Changes that make a daily impact in our customers’ lives.

This is one of the things that I absolutely love about Fog Creek’s internship program. We treat interns like full-fledged employees, and our interns continually reward us by producing amazing things.

This obviously works well for us here at Fog Creek, but I also think it works out really well for our interns. One of our interns, Andrew Pritchard, noted:

Working at Fog Creek has already expanded my horizons more than any of my previous experiences—and not only because I am working for the first time on code destined to land in paying customers’ eager hands!… As it turns out, complete immersion, combined with the intrinsic and extrinsic motivation (thanks Joel!) that come with a paying job, can make learning an entirely new set of technologies surprisingly easy. And complete immersion is exactly what it has been—from the very first day, I was adding ASP.NET MVC actions from scratch, writing jQuery selectors, refactoring CSS into idiomatic Less, and creating markup for stylized buttons for the oft-requested case-changeset dissociation feature, which made it into the first Hosted release of the summer.

When I finished working on Copilot for my summer, I had learned a lot about writing real software. Not stuff for class; stuff to ship. And it has served me amazingly well, not just at Fog Creek, but on every semester of school I’ve had since then, and on every hobby project I’ve taken on.

I’m sad to see this summer’s internship come to a close, but I’m very happy at what we were able to get done this summer—and I think all of our Kiln users will be, too.

And don’t forget: if you’re a student, and you want to be part of an internship or co-op like this, apply for our next cycle. I’d love to have you on my team.

Slaughterhouse 1.9: How I Learned to Love Contributing to Mercurial

July 14th, 2011 by Kevin Gessner

For most of my programming career, I’ve treated open-source software much like a slaughterhouse: I’m more than happy to use all of the wonderful things that open-source devs produce (Linux, jQuery, and WebKit, to name a few), but I don’t want to know what goes on behind the scenes. I’m sure it’s bloody and not nearly as pretty as what gets tagged and bundled each release.

Working on Kiln the past few years, I’ve become more and more familiar with one open-source project in particular. Kiln is built on top of Mercurial, using both its high-level and low-level interfaces. We’re proud to run one of the largest installations of Mercurial in the world with our Kiln On Demand service. From our users and from our own experience, we’ve come to know Mercurial’s strengths and its weaknesses, and because Mercurial is open source, we can also submit patches to help fix them. Over the past two years, we’ve watched Mercurial release several new versions, each with a patch or two from our team.

Of course, with our engineering resources, we can do more than just submit a patch every now and then. Each year, the Mercurial core team assembles for a code sprint. This past spring, Benjamin asked me if I would like to attend this year’s sprint in Copenhagen, to get my hands dirty on an awesome open-source project. Count me in! Of course, there’s no such thing as a free lunch, even with a generous employer like Fog Creek: Benjamin sent me to actually get stuff done, not just to enjoy the weather.

But what could a lowly coder like me do? I started by getting some patches into Mercurial. The project’s backlog of bugs and small feature requests is seemingly bottomless, giving lots of opportunities to get rid of little problems and paper-cuts. I was able to commit patches to add Git-style parent revision references to Mercurial’s revsets, and to clean up some other gnarly parts of the code base. I also worked on improving Mercurial’s handling of the vagaries of different OSes, especially the differences between Windows and the various unixes. Every OS has its share of reserved filenames, from OS X’s .DS_Store to Windows’s CON and AUX. If you add a file to a repository that’s reserved on a different OS, you won’t be able to check out the repository on that OS—and that’s no good. Additionally, two filenames that differ only in case (like foo and FOO) are allowed on some filesystems, but not others—even within a single OS. Mercurial now helps protect you from yourself, warning you when your changes would cause problems on other computers.

You mean, Etherpad exists in the real world, too?

Now, if you take a bunch of geeks who are used to working alone at their screens, and put them all in a big room to work together, how do you think we’ll collaborate? Alone and at our screens, using Titanpad and email, of course! Traffic on the dev mailing list went way up as patches, feedback, and bug reports bounced around. But the sprint really shined when we started working together to plan and design new features that will make Mercurial more powerful and flexible.

The Mercurial project isn’t as big as some open-source projects, but it has more than six years of history and hundreds of committers. Dozens of them are actively and passionately involved in the project, and most of those were at the sprint, so when it came time to plan the direction of the project as a whole, there were plenty of well-argued opinions and ideas.

The first big feature planning was for filesets—a query language for finding and specifying files in your repository, much like how revsets work for finding changesets. We all collaborated on creating a set of predicates that would make the language useful. Being able to quickly go back and forth about ideas in person made the brainstorming process much smoother than if we were scattered around the globe.

The second big brainstorming question was developing the interface and implementation for a brand-new feature, called “dead changesets”. In a large project, you can end up with changesets, or even whole branches, that are dead ends, ideas that didn’t pan out or features that were cancelled. While having these around for posterity can be helpful, they can also just be clutter in your day-to-day work.

As with any new software feature, there were questions ranging from UI to implementation details to backwards compatibility that needed to be answered. Over a couple hours of aggressive whiteboarding, we worked out a plan that seems to fit the bill: when you mark a set of changesets as “dead”, they’ll still be around (in case you ever need to reference that pie-in-the-sky refactoring), but they won’t be displayed or transferred unless you explicitly request or resurrect them. The feature didn’t make the cut for 1.9, but even so, the team now has a head start on getting this done for another release.

Finally, we discussed bringing one of Kiln’s major features—kbfiles—into Mercurial as an official feature. kbfiles enables you to track the history of large files (like images, libraries, and executables) in your Mercurial repository, without keeping a copy of every version of every file on your computer. We ship kbfiles as part of Kiln, but we believe that it is valuable for every Mercurial user. Over the next few months, we’re going to tie up some loose ends, so that kbfiles works just as well with vanilla Mercurial as it does with Kiln, and make it available as an official extension. You can even follow along in our public Kiln installation as we work on that.

Perhaps most importantly, I had the chance to meet a large part of the Mercurial dev team, including the project lead, Matt Mackall. For a couple months, I had known the team only via IRC and email, so putting names to faces, working side by side, and even simply sharing meals humanized the whole process. Working together is faster, easier, and just plain more fun when you’re all in the same room (even when you’re still interacting virtually), as opposed the usual not-being-on-the-same-continent situation. Thanks to the whole team for being helpful and friendly to this n00b.

Mercurial 1.9 was released earlier this month, with my patches, mpm’s implementation of filesets, and a host of new features, bug fixes, and general awesomeness from the whole team. Mercurial is now faster, easier to use, and more powerful than the last version—an amazing accomplishment for any software product, but especially for one developed by a team as large and diverse as Mercurial’s.

This code sprint was the first time I’d been deeply involved in the inner workings of an open-source project. I’m happy to say that the process wasn’t nearly as bloody as I’d feared. I’m proud to be a part of the project, even if it’s in a small way.

Caching for Kiln performance improvement

June 10th, 2011 by Dan Ostlund

“Sluggish” might be a good word for a lazy summer weekend. It might be an apt description for a snail who really, really wants to get to that next blade of grass. In a web app it’s the kiss of death.

I have to apologize to my StackExchange friends because Google fed me an Answers.com link to my question about how long people are willing to wait for a web page to respond. It authoritatively said people won’t wait “more than 30-40 seconds” which concretely proves that this answer hasn’t been updated since 1996. More recent studies say something along the lines of “2-3 seconds” before people start to wander off looking for a reliable technology like a papyrus page listing the price of wheat in Mesopotamia. If it’s a business app, they don’t even have that option; they have to wait as their blood pressure slowly increases and your credibility takes a swan dive off a cliff.

Kiln’s not sluggish, but as it grows we face interesting problems and are constantly seeking ways to keep performance snappy. More people means more load, which means more time to process requests, and throwing hardware at the problem only works some of the time. The Kiln team recently instituted an entirely new caching scheme for keeping performance fast and blood pressures low. And so far it’s been working like a champ.

Distributed Version Control University

April 28th, 2011 by Benjamin Pollack

One of the major tasks we wanted to accomplish on the Fog Creek World Tour was to teach people about distributed version control systems, such as Mercurial. A lot of people have been clamoring for a video of the talk we gave, so I’m happy to announce that we’ve finally got it live:

One of the things we struggled with when trying to design the talk was how to bring people up-to-speed on the basics of distributed version control, and get them to understand why distributed version control was so amazing. Our solution was to break the talk into two pieces: the first part covers the very basics of Mercurial, the how do I use this?. The second half discusses some of the workflows that are hard in traditional systems, but really easy in distributed ones, the why do I want to use this?. Finally, there’s a long question and answer session at the end that’s probably interesting regardless of your skill level.

Saw the video, and liked what you saw? Take Kiln for a test drive! You can even follow along with the talk, learning Mercurial and playing with the very same workflows we discuss in the video.

So you wanna version your database…

April 13th, 2011 by Benjamin Pollack

When we launched Kiln, I knew we were solving a bunch of major problems that users actively had. I knew that because every time that I talked to someone, and walked them through what Kiln was capable of, they got really excited and asked piles of questions of how they could integrate Kiln into their current workflow.

But there was one question that just kept coming up time and time again:

How do you version databases?

I really empathize with this problem, because it impacts every single web application I write. It’s easy to version your code, to version your documentation, to version your images, to version your deployment scripts. But versioning the database schema gets hard. For really big systems, you end up needing piles of migration tools, but for most stuff, that’s overkill. You just need a simple way of tracking and applying changes to your schema. Using version control to store your migration scripts ends up being this weird version controlling your version control.

The great folks at Red Gate actually already came up with a good solution to this problem, called SQL Source Control, which allows you to version your schema, and even optionally your data, right inside your existing source control system. The only problem is that, up to now, it only worked with traditional SCMs.

The key words being: up to now.

Committing database changes in SQL Souce Control

Red Gate and Fog Creek are both happy to announce that SQL Source Control 2.1 now includes full support for Mercurial, meaning you can now trivially version your database schema in Kiln. And because SQL Source Control ultimately just works with the SQL statements that generate your database, it’s easy to make sense of the diffs from within Kiln. You’ll finally be able to bring the power of DVCS and Kiln to your database.

If you’d like to try it out, there’s a 28-day free trial available that will let you play with SQL Source Control and see how it integrates into your workflow.

Introducing Kiln Glazes

April 12th, 2011 by Tyler Hicks-Wright

One of the first FogBugz plugins available after the FogBugz Plugin API was released was the very handy BugMonkey plugin. I’ve had some fun tweaking FogBugz, like removing the time from date displays, reversing the order of bug events, and using BugMonkey to add syntax highlighting to BugMonkey. BugMonkey has helped a lot of people implement custom views or workflows that didn’t make sense as FogBugz features.

Meanwhile, Kiln has been growing as a product, and has reached the point where a level of customization makes sense. Of course, we had the Kiln API and Web Hooks, but those only addressed external tools. If you wanted to change the interface, you were limited to using Chrome or GreaseMonkey user scripts, like Kiln Hash by John Isaacks.

To alleviate that dependency, we are introducing Kiln Glaze, a port of BugMonkey to Kiln. To access Kiln Glazes, click the “My Settings” dropdown and select “Kiln Glazes”. Glaze is initially disabled, so a site administrator will have to click “Enable Glaze”. Once it is enabled, you will see a interface that is very similar to BugMonkey:

The syntax is also very similar to BugMonkey. There are single-line fields for name:, description:, author:, and version:. Following that are two multi-line fields, js: for any Javascript code and css: for any CSS styles. For example, John’s “Kiln Hash” Chrome extension is very easily converted into a Kiln Glaze:

name:        Changeset Hash
description: Shows the changeset ID for each changeset in a list.
author:      John Isaacks and Tyler Hicks-Wright
version:     1.0.0.0

js:
$('[id^=changesetList] tr').each(function () {
  var tr = $(this);
  var sid = tr.attr('sid');
  if(sid)
  {
    var first = $('<span>')
                    .addClass('csetHash')
                    .text(sid.substring(0,10));
    var rest = $('<span>')
                    .addClass('hashRest')
                    .text(sid.substring(10))
                    .hide();
    rest.appendTo(first);
    tr.find('td span.changesetDescription').append(first);
  }
});
$('.csetHash').hover(
  function(){
    $(this).find('.hashRest').css('display','');
  },
  function(){
    $(this).find('.hashRest').css('display','none');
  }
);

css:
.csetHash
{
  color: #777;
}

Which results in the changeset IDs being visible on each line: 

One note about Kiln Glazes: we do not consider our DOM or JavaScript to be part of a supported API, so it may be possible for element IDs or classes to change without warning. From our experience with BugMonkey, this is usually not much of an issue, since existing DOM and JavaScript does not change that often, but it is something to be aware of.

That said, have fun with Glaze, and if you come up with any that you think other people might be interested in, please post them to the Kiln StackExchange!

Rethinking Reviews

April 11th, 2011 by Benjamin Pollack

Few people know this, but Kiln started life as nothing but a review system.

Tyler and I both wanted to have a code review system to help us at Fog Creek, and disliked all of the available ones, so we wrote one up that used Mercurial’s branching and merging tools to have a saner workflow for fun as part of a competition. To make that work, we had to give Kiln basic repository hosting abilities. After we brought Kiln in-house, we drastically built up the project hosting capabilities and redefined the product as a complete distributed source control system, but the review system, in more-or-less its original form, stayed with the product as a major feature.

Unfortunately, the review system we came up with was based on FogBugz, and while that system worked great for Fog Creek, it simply didn’t fit a lot of teams very well. Kiln reviews, like FogBugz cases, could be assigned to only one person at a time. This worked great for making sure that a single person was always responsible for a review, but ended up getting in the way of the inherently collaborative nature of improving source code: each developer’s thoughts on the code would end up in its own code review. While they were linked, assembling a coherent picture of all comments on a given piece of code was much harder than it should’ve been.

So we went back to the drawing board, and for Kiln 2.4, came out with an entirely new way to do reviews.

An intermitent state of a Kiln multiuser review

Beginning today, Kiln reviews can have many reviewers, and reviewers can be added or removed at any time. Each reviewer can independently indicate that they approve or reject the changes and comment on the code.

A code review that has reached consensus

If all reviewers reach a consensus, the review is automatically marked “approved”, and the review opener is notified that the changes have been collectively approved or rejected. If the reviewers cannot reach a consensus, the requester can leave the review open until a consensus can be reached, or can acknowledge the stalemate and close the review with a brand-new status of Abandoned.

Viewing Kiln code reviews in FogBugz

This posed a slight issue for us: one of the best things about Kiln is that it integrates extremely tightly with FogBugz, meaning you can see reviews you’re responsible for right alongside your bugs and feature requests. We didn’t want to give up that ability with the new system. So what we did was add a new filter option to FogBugz, “My Code Reviews”, and a new search axis, reviewedby:me, so that you can easily find the reviews that are waiting for your feedback.

We feel that this new system is the best of both worlds:

  1. Reviews now have lots of reviewers, so conversation all takes place in a single case–not a collection of related cases.
  2. At the same time, everyone is still fully responsible for actually weighing in. While you can abstain from reviewing, that gets logged, and Kiln resists closing reviews before assignees have made a decision and recorded it.
  3. And we’ve done this without losing our tight integration with FogBugz.

The new system ships out today to licensed customers, and has been available on Kiln On Demand since late last week. If you haven’t yet had a chance to give Kiln a spin, now’s a great time to sign up for a free Kiln trial so you can play with the new review system first-hand. We think you’ll like what you see.

Supporting the Giants

April 5th, 2011 by Benjamin Pollack

One of the things that I mentioned when I discussed the importance of having a mission is that the Kiln team’s real purpose in life is to bring the awesomeness of distributed source control to as many people as possible. While we do a lot of that by making Kiln into the best source code management system available, we’re hardly doing it alone: Kiln builds off the amazing base provided by the Mercurial distributed version control system. Mercurial helps make Kiln what it is. For Kiln to really achieve what we want it to achieve, we need to help Mercurial itself shine, too.

This year, at Fog Creek, we’re doing that in two different ways:

  1. First, we’re helping sponsor the Mercurial 1.9 code sprint. The code sprint is where the Mercurial developers have a chance to talk face-to-face to solve outstanding issues, agree on how to implement major new features, fix piles of bugs, and so on.
  2. Second, we’ll be sending one of our own Kiln developers so that we can contribute some actual manpower to the bug fixing and feature development that goes on at the sprint.

This kind of paying back to open-source software is becoming increasingly common. Even when StackOverflow was brand-new, Jeff and Joel were already donating to support the projects that made StackOverflow possible, and they’ve kept up that tradition as StackOverflow has grown. I think this is a great way to pay back the open-source community, and to help further the creation of more great open-source tools.

If you’re interesting in helping the Mercurial project, the easiest way to help them out is to donate to the project or the sprint.