EBS 2.0

July 23rd, 2009 by Brett Kiefer

Death Marches

Figuring out when a software project will ship is a famously difficult problem, and nearly everyone who writes software for a living has to grapple with it. I remember asking an experienced developer at my first job how I should calculate my project ship date. He said (and please imagine him cheerily chirping in a Scottish accent): “Just go with your gut. Then double it, add twenty percent, and pray.”

For a long time that was the best advice I got, and it appeared that no one else had a better idea. So I got to see lots of smart, dedicated, otherwise rational software developers work brutal hours and take unwise shortcuts on miserable death march projects because, dammit, they had committed to a date and they were going to DELIVER.

Iterating on EBS

In our last major release of FogBugz, we launched a new approach to the problem called Evidence-Based Scheduling (EBS). EBS takes your team’s project milestones and tasks and uses a Monte Carlo method to generate a probability distribution of when you’ll ship your product.

Since it’s part of your project management system, you can get all of this without a lot of sweat: just estimate your tasks and tell FogBugz more or less when you’re working on something; it does the rest. The best thing is that (like a genetically modified shark!) EBS gets smarter, because it constantly feeds your historical data back into the prediction algorithm. If this is the first you’ve heard of it, check out Joel’s introduction here.

We’ve loved hearing from the customers who have found EBS useful. Thanks, y’all! At the same time, many users have come to us with great ideas for features we should add. And it’s their thoughtful suggestions that made us confident that time spent adding to EBS would be well-spent. Today I’m happy to announce EBS 2.0, launching as part of FogBugz 7. EBS 2.0 introduces two new features that make it possible to model real-world projects: inter-project timelines and strict dependency modeling.

Inter-Project Timelines

In EBS 1.0, each project’s timeline was considered individually, so it was difficult to model a dependency on a shared resource; for instance, if a system administrator had cases in multiple projects, FogBugz assumed that he would begin work on all of them full-time in parallel.

Instead, EBS 2.0 assumes that you will work on all of your milestones in order, but it allows you to take control of parallelization by assigning percent times to projects. So if you always spend 10% of your time recruiting, 20% of your time working on your Amazing FogBugz Plugin, 10% staring out the window, and the rest of your time working on your development tasks in all other projects, that’s now easy to model.

This allows you to divide your FogBugz install into granular projects (per-client, for instance), and still get meaningful answers from EBS about when users will start and finish work. While EBS 1.0 could model the work of the traditional, project-monogamous software developer pretty well, EBS 2.0 allows, say, a visual designer or consultant – anyone who divides time between multiple projects – to express her schedule accurately.

Strict Dependency Modeling

EBS 1.0 didn’t have strict dependencies. Each user was assumed to finish his tasks in the first milestone, then start working on the second immediately. And that’s the way it was, and we liked it. Usually. FogBugz is optimized for tracking software projects, and in most kinds of software development, a strict dependency is the exception, not the rule. Even if I’m waiting for someone else to finish a component, I can look at its spec and write my software accordingly.

But! Sometimes you really can’t start work on a milestone until something else happens first. For example, our sysadmins are stellar, but even they have a hell of a time racking hardware before it arrives, and we in turn have a hard time deploying and testing our software on servers with no operating system installed. In EBS 1.0, you just couldn’t express this kind of strict dependency.

EBS 2.0 gives you the vocabulary of strict dependencies and start dates. You can now say “No one can travel back in time until the Flux Capacitor is complete and the duped terror cell steals the plutonium.”

“. . . AND we can’t build the time machine until the DeLorean arrives on October 20th.”

Though the interface is simple, these dependencies can be used to build up a much more realistic model of your work. Given your dependencies, EBS will now build a directed graph of all of your milestones. It combinines the simple EBS 1.0 timeline with any of the strict dependencies and start dates you specify in order to calculate probable ship dates and risk for all of your projects.

Bonus: New Graphs!

As the model for project completion became more complex, we realized we’d need a more expressive graph. So we’ve added a very data-rich graph called the User Timeline. It shows all of the users on whom a milestone depends, along with a graphical representation of all of the work they have to do to complete the milestone. You can see their work on other projects, idle time caused by unsatisfied dependencies, and estimated completion probabilities. The graph is interactive, allowing you to drill down to get more information about any user’s remaining work, so that you can rebalance tasks and see what other projects are influencing your ship date.

Here we can see that Marty will be unable to travel back in time until Doc has finished the Flux Capacitor (the light blue) and gets the plutonium from the duped terror cell (orange). Until all of that is done, Marty is idle (the red area), because the only thing he’s supposed to do is blocked.

So if we need to get Marty time-travelling earlier than that, the graph tells us that we need to either break the dependency on the Flux Capacitor (not likely!) or speed up its development somehow. For instance, we can see that Marty is free right now, so if we can find work for him to do on the Flux Capacitor, we would speed up the whole project. If it sounds like EBS is helping us to find the probable critical path for the project, that’s because it is!

The Future

There is still a lot more that can be done with Evidence-Based Scheduling, and we’ll continue to invest in new features. Of course, with the release of the FogBugz Plugin Architecture in FogBugz 7, which provides hooks for interacting with EBS, we hope to see a lot of innovation coming from outside of Fog Creek. That’d be neat.

We’d be delighted to hear what you have to say about EBS 2.0, so create a free FogBugz account and try it out!

The FogBugz Developer Series

Next up, a look at the exciting FogBugz Plugin Architecture with FogBugz Plugins development lead David Fullerton.