Archive for the ‘FogBugz’ Category

Better BugzScout Error Reporting

January 30th, 2012 by Jude Allred

Prelude and a pain point.

One of the many ways to spawn cases in FogBugz is via a mechanism we call BugzScout. BugzScout is a simple interface for collecting crash reports from your software in the wild. For example, you might add an exception handler to your product which hooks into BugzScout so that when your customers encounter errors, you receive a case in FogBugz with details about the problem. Unlike the FogBugz XML API, which can also be used to serve this use case, BugzScout doesn’t require that you embed any type of FogBugz security token in your deployed application, and BugzScout will automatically collate all of the bug reports into cases based on titles of the reports.

We use BugzScout extensively throughout our own infrastructure — when FogBugz accounts misbehave, if an unparseable email gets stuck in an account’s mail queue, if an account’s full-text search index gets corrupted, etc., we receive cases in our FogBugz environment detailing the problem and often including key diagnostic information such as stack traces and assorted report-specific meta-data.

BugzScout has been around for a while, long enough to get a little crufty, and we’ve felt some BugzScout pain as we’ve scaled up. Pain can lead to cases:

We generally agreed with these suggestions, and we’ve definitely known the frustration of some small process misbehaving and sending literally thousands of BugzScout reports our way.

Better.

The first thing we came up with was what we all thought would be a completely amazing BugzScout-esque product which could solve the BugzScout use case perfectly. But we didn’t build that… its scope was slightly larger than the part-of-a-week we wanted to spend on it. Plus, building a new product wouldn’t have solved our immediate problems fast enough. Our driving force behind the improvements became the following insight:

Sometimes you get a human-readable number of BugzScout reports. In these cases, you absolutely want the full error text from each occurrence, and it’s reasonable to route the event through all of the standard notification hooks. If someone doesn’t want to hear more about this error, they can direct BugzScout to stop reporting via editing the case.

Other times, you get so many reports that no human will read them. The case is spammed with stack traces and error notifications. After some point, additional stack traces are of no use — you’re not going to read through them anyway, they just make the case cumbersome. At this scale, you need BugzScout to do a few things:

  1. Shut up and let me fix it. You’ve already gotten a bunch of emails about the problem- one additional email per occurrence is of no use to anyone, but can legitimately hinder your efforts to actually deal with the problem.
  2. Help me know I’ve fixed it. It’s always been possible to determine if more reports are coming in by either watching the occurrence count of the BugzScout case or by monitoring when the case was last modified by BugzScout. Unfortunately, if you wanted to accomplish #1 by setting BugzScout to ‘Stop Reporting’, you’d also stop getting updated timestamps from when new reports came in.

We decided that reconciling #1 and #2 would be a great improvement for BugzScout and would be well within the scope of a few days of development. We felt confident that this type of change would solve our pain point, but before we went any further we wanted to make sure we weren’t operating in a no-fact zone. We reached out to our SaaS platform and gathered some anonymous data on how people were using BugzScout. The data confirmed our guess: Our use cases are valid and a significant amount of people are using BugzScout-based error reporting.

We settled on:

  • BugzScout will now automatically place itself into “Stop Reporting” mode after 50 occurrences of a report.
  • Last Occurrence is a new case field which is stored alongside all BugzScout cases and is fully searchable, filterable, etc. Last Occurrence is the timestamp of the most recent BugzScout report, regardless of whether that report was recorded as a case event.

Resulting in:

  • Human-readable amounts of BugzScout reports should be unaffected.
  • Human-unreadable amounts BugzScout reports will automatically cut off after 50. This means that the bloody tide of email will be curbed; BugzScout cases are limited to having a workable number of events in them.
  • If it comes time to collect more reports, you can do this by editing the case and instructing Scout to ‘Continue Reporting’.
  • Last Occurrence is a new and useful filterable value.

Shipped.

We’ve been dogfooding the new BugzScout in our FogBugz environment for the last month and have been quite happy with the improvements. We’ve found that the automatic shutoff of case event generation after 50 reports has allowed us to function much better when dealing with issues at scale.

Here’s an example of how our team likes to monitor the incoming BugzScout reports using the new Last Occurrence column:

This isn’t the only way we triage our BugzScout reports, but this simple filter lets us quickly spot any issues that are happening right now and know if they’ve happened with enough frequency to warrant immediate attention.

These changes went live to FogBugz On Demand two days ago, so we hope that you’ve found them useful. Licensed FogBugz customers will get these changes in our next licensed release.

Let Them Have Cake … And Ice Cream Too!

November 17th, 2011 by Rock Hymas

Consider a cake shop owner in a small town in Siberia. He sells the best cake around. Whenever someone wants a treat after dinner they stroll down to the cake shop and get a nice big slice of cake. But along comes a new treat: ice cream. Our cake shop owner ridicules his neighbor for starting an ice cream stand, saying no one will buy it because it’s cold. This is Siberia, after all. But rather than running an oven all the time, she can just keep her ice cream outside in the freezing cold, so it sells for less. Suddenly a whole bunch of people who could never afford cake are stopping at the ice cream stand after both lunch and dinner. It’s cheap, it tastes good, and the teenagers are having contests to see how much they can eat before passing out from brain freeze. It seems the whole town is there, and some of the cake shop customers start getting ice cream just because they want to hang out with their friends at the ice cream stand. All of a sudden the neighbor is rolling in the dough and she wants to buy out the cake shop since they have a great location.

Now suppose the owner of the cake shop recognizes that lots of his customers will like the ice cream (even in Siberia) because it’s so much cheaper. So he sets up an ice cream stand and hires his neighbor to start selling it as fast as possible. He notices that some of his regular cake shop customers are going to the ice cream stand instead, because it’s cheaper. So cake sales are down a little. Rather than shutting down the ice cream stand though, he brings it into the cake shop, adds a special “cake” flavored ice cream, and becomes the town hero. Not only has he saved his business, but he’s also been able to meet the dessert needs of more of his fellow townspeople. Cake is selling even better than before, ice cream is selling even better than that, and the cake-fanatics and ice cream groupies all get to hang out together.

Clayton Christensen’s talk at this year’s Business of Software was all about how companies disrupt, and are disrupted by, other companies. In building a product, you (or those who came before you) made decisions that are really hard to change after the fact. That’s fine; those “stakes in the ground” are what made the product successful. But it also limits the viable lifetime of the product. At some point disruption happens.

Disruption =

Larger Market + Lower Price + Different Measure

 

Professor Christensen pointed out that disruption occurs when a company solves a given problem for a larger audience at a lower price point with a different measuring stick for comparing value. Companies that successfully avoid being disrupted are usually able to do so only by disrupting themselves. Few companies are able to pull that trick off, and it typically involves having a different team or business unit set up in order to avoid all the baggage that the last-generation products carry with them.

What does this have to do with Fog Creek? Well, I’ll admit that I was a little skeptical when Fog Creek launched Trello. My worries pretty much lined up with those outlined in this post from a FogBugz customer. Though it’s taken some time, I’m starting to see how important it is for Fog Creek to prepare for the disruption of its flagship product, FogBugz.  By doing so, we can make sure FogBugz will keep solving our customers problems and keep making us money.

Trello fits the model for disrupting Fogbugz. It solves the problem of planning the work on a project for a larger audience than just software companies, at a lower price point than a FogBugz or a Jira, with a different measuring stick: putting Trello and FogBugz next to each other on a feature comparison chart doesn’t make any sense.

Trello’s target market is much larger than that of FogBugz. Where FogBugz targeted software development teams with features like bug tracking, automated crash reporting, and evidence-based scheduling, Trello can provide value to any group of two or more people working together on something that can be broken down into steps. Even though any group of people could use FogBugz to do the same thing, using FogBugz doesn’t really make sense when you’re in HR doing hiring, or in law working through cases, or in a studio vetting country bands. Using Trello does.

Additionally, Trello has a lower price point: free. Everything currently offered by Trello is free, and will remain so going forward. Yes, there will probably be value-added features and services that Fog Creek will charge for at some point. Compared to free, though, FogBugz is expensive at $25 per user per month. When you just want something simple to plan your wedding, that doesn’t make any sense. But if you’re managing software projects for reasonably complex products, then FogBugz easily adds that much value, and our customers make that clear by coming back again and again.

Trello also has a different measuring stick. And that’s the real reason it will eventually overtake FogBugz, even for software projects. Or rather, it will take over some of the roles that FogBugz fills in a software development company. We already use it here at Fog Creek to manage our work at a coarse-grained level. It provides a potentially public view into product development, which is cool to see.

The real key, though, is that Trello, like FogBugz, is opinionated–but it has very different opinions. Rather than seeing work on a project as a large set of small items that need to be tracked individually, it sees project work as a small set of somewhat larger tasks that fit into a bigger whole, a workflow defined by the team. If FogBugz tried to create some kind of dashboard view of your bugs to compete with Trello, you’d be so overwhelmed by minutia that you’d give up and walk away. No one wants that kind of view when they’re dealing with hundreds or thousands of individuals cases. But Trello redefines the way we see our project work and that fundamentally changes the game.

Wait a sec! I’m on the FogBugz team. What am I saying?! Am I just making a Steve Yegge “TMI” mistake by posting this publicly? Am I talking myself out of a job?

 

TMI?

 

I don’t think so. And here’s why.

The same legacy that prevents it from winning in the larger Trello market gives it a competitive advantage in the market for software development teams. FogBugz will continue to make money, and it is still growing at a nice pace. It needs investment, but not the kind of investment that would attempt to turn it into Trello. Rather, the kind of investment that will take advantage of disrupting products like Trello while preserving it’s usefulness in the niche of software development teams.

And our customers, for the most part, aren’t going to jump ship for Trello anytime soon. Currently, Trello can replace only a very small part of what FogBugz provides. One customer pointed out that it cannot handle bulk editing, screenshot captures are painful, and categorization and search aren’t designed for the situation where you have lots of items. Additionally, FogBugz supports incoming and outgoing email, automated crash reports, and deep hierarchies of work. You can install it within your network and use it completely internally. It’s also got awesome source control integration with Kiln. These are all things that software development teams care about, often passionately. Trello, and other products like it, may eventually meet some of these needs, or integrate with other tools that do, but that will take time. Time that will allow FogBugz to further differentiate itself.

Besides, learning new ways of working is hard. Anecdote time! We spent some time over the last few months rethinking the UI for FogBugz. As the team lead, I made the call to focus on that, and did what I could to protect the team through the process. Unfortunately, when it was done, Joel pointed out that it wouldn’t fly with our customers. Why? Because we “moved the cheese” in too many ways. It would have required our customers to learn new ways of working, and human nature doesn’t like to do that. Though that does limit what we can do with FogBugz going forward, it also strengthens the ties that our current customers have with the product. They know where to click, which keyboard shortcuts to use, and they know what to expect. That helps them work faster, get in the flow, and keep the boss happy.

Finally, and related to the last point, FogBugz has an ecosystem of supporting technology around it. When our customers start using FogBugz and Kiln, they often integrate with a set of tools and technology. This is one reason it’s harder to innovate on those products, because our customers don’t just rely on not having to learn new ways of working, they also rely on their custom plugins not breaking, on their random Python scripts still working, and on third-party tools continuing to integrate well with FogBugz. Each of those is a barrier to entry – they make it harder for a competing product to win our customers away from us (yep, even if the competitor is made by us as well) because they also add value for the customer.

In short, for almost all of FogBugz existing customers, and most of the larger market of customers using a competing bug tracker, using Trello instead is not the right business decision. And within that market FogBugz can make sure that business decision doesn’t change by doing the right kinds of investment. Investment that will keep FogBugz relevant and growing into the future is core-competency investment, not more features or chasing faster competitors in a race decided by a radically different “finish line”.

And there are tons of things we can do in the core of FogBugz: reduce the complexity of the product both for our users and for our developers; make it faster; phase out old features that aren’t being used by anyone; fix the backlog of bugs; improve our up-time; integrate with Trello and other apps. That last point is probably the most salient. As Trello grows and becomes a better fit for certain kinds of project management work, FogBugz can increasingly integrate with and offload those areas to Trello. At the same time, it can target specific problems that software development teams face, and provide value by solving them well.

In the end our customers will be able to have their cake … and their ice cream too.

FogBugz and Kiln Build and Release Report Card

November 3rd, 2011 by Rock Hymas

My wife and I are a couple months into our first year of homeschooling, and we’ve been discussing how the kids are doing with it, and the challenges we’ve faced as their teachers and parents. Those discussions have led us to make some important changes, in the same way that a school report card can get parents and children to make changes that lead to more learning and happier kids. In that spirit, I offer here a report card on the build and release management work we’ve been doing on FogBugz and Kiln [1] over the last 12-18 months.

About a year ago, I was in the middle of writing a series of posts about the practices of build and release management. Many of those practices came from looking at what we were doing with FogBugz and Kiln and identifying what worked, and what didn’t. Others came from what I saw and experienced while working within the Microsoft Office organization, and still more came from discussions within our team at the Creek. Though in some areas we were doing things right, most of the posts were a description of what our existing build and release (non-)system was doing utterly wrong and what I hoped we could change to fix it. So the series was often written in a spirit of hope that we would one day live up to the ideal.

In fact, you could take many of the posts I wrote last fall and read them as the opposite of how things existed at Fog Creek in the summer of 2010. So, in the spirit of Jason Cohen’s recent BoS talk about honesty, I’m going to be brutally honest about how we were doing and how we are now doing.

First, how we were doing. Here’s the report card for June of 2010, divided into subjects, of course.

Report Card for June 2010
Subject Grade
Fast builds/deploys

Deploying upgrades to www.fogcreek.com takes over an hour

Other builds and deployments often take much longer than they need to

Upgrades to FogBugz or Kiln often involve downtime of a few seconds to half an hour or more

D
No hard dependencies

Checked out source code is tied to absolute paths in some cases

The build and deployment scripts are full of hard coded paths and other hard dependencies

F
Single commands (automate, automate, automate)

We can’t run most builds with a single command

Many of the build and deployment scripts are combined

Deploying a new version takes multiple steps

Different scripts are used for dev builds vs official builds

Different configurations of the build use different build scripts

There is no tooling to make it easy to setup your dev system

There are no scripts to configure a new server for our hosted environment, or a new dev box for development

D
Staging Environment

We have no staging environment for our hosted FogBugz and Kiln products

F
Continuous Integration

There is no continuous integration builds or tests running

F
Scripts @ production quality

The scripts are old, unmaintained, with lots of cruft

Many built binaries are stored directly in the VCS

D
Onboarding new developers

There is no easy way for devs to build installers

There are no build scripts that do incremental builds

D
Insight into builds and releases

Old builds are not saved for most official configurations

We aren’t tracking build statistics like failures, build quality, etc.

Logs of builds are not systematically kept and stored anywhere

Build failures do not automatically notify anyone, nor do successful builds

There are three different types of version numbers for our three different deployment targets

F

All in all, things were a mess. ”Good grief!”, I hear you cry. “So what have you done about these issues? Anything? What are you planning to do?” I’m glad you asked.

My initial goal in the role of build and release manager was to eliminate the position. I wanted to get things running smoothly, and spread the knowledge and ownership of the problems through the team, so that I could go back to doing product development. As such, a decent amount of the progress described here was done by other great developers here at the Creek. Back in January (when I joined FogBugz as the team lead), I thought I’d gotten about halfway through the work required for that to happen. But, of course, new ideas come up constantly, so maybe it never ends.

Since January, I haven’t been able to do as much. In addition to my small efforts, other great Creekers have picked up some of the slack. Here’s our report card for October 2011.

Report card for October 2011
Subject Grade
Fast builds/deploys

A new, fast build machine

Reduced web site deployments from taking ~80 minutes to taking ~8 minutes

FogBugz and Kiln deployments now faster, because they don’t also rebuild the product

Upgrades to FogBugz or Kiln still involve downtime of a few seconds to half an hour or more

C+
No hard dependencies

Removed all the absolute paths

Removed many other hard dependencies

Removed some remaining hard dependencies that occasionally caused build failures

B
Single commands (automate, automate, automate)

Mortar, an internal website for kicking off builds and deployments

Lots of help on this one from Benjamin at bitquabit

Separated build and deploy scripts for hosted builds

Some of the build and deployment scripts are still combined

Deploying a new version takes multiple steps

Different scripts are still used for dev builds vs official builds

Different configurations of the build still use different build scripts, but some combining work has been done

There is still no tooling to make it easy to setup your dev system

There are still no scripts to configure a new server for our hosted environment, or a new dev box for development

C+
Staging Environment

Still no progress on the staging environment

F
Continuous Integration

Continuous integration builds exist, mostly just building the products

Basic tests have been added to some continuous integration builds

B
Scripts @ production quality

Removed many checked in binaries, building them instead

Many scripts converted from FinalBuilder to Python, but a few still remain

B-
Onboarding new developers

Branch builds, for any branch a dev wants to setup

bf, a better tool for building FogBugz incrementally on dev machines

Aaron kick-started this and continues to be the driving force

There is still no easy way for devs to build installers

D
Insight into builds and releases

Saving old hosted builds

Moved to a single type of version number no matter the config

Build logs for all official builds

Cases filed on build failure

Notifying build owners when builds complete, successfully or not

We aren’t tracking build statistics like failures, build quality, etc.

B-

As you can see, there is still plenty of work to do. The glaring failure in both report cards is our lack of a staging environment. We’re now working on making that a reality, with high hopes that it will help us start catching a new class of bugs before they make it into our customers hands. As part of that work, we’ll also be able to work on:

  • Scripts for configuring our servers, by type
  • No-downtime upgrades
  • Move our internal dogfooding to the staging environment, for better consolidation and testing of deployment scripts

Other possibilities for further work should be pretty clear from the report card, but the staging environment is the key focus when it comes to improving our builds and releases.

One note about the culture for both FogBugz and Kiln. We’ve learned a ton in the last couple years by seeing at close range what can be done in greenfield development on products like Trello, or teams that actively maintain and improve their processes for builds and releases, like StackOverflow. Obviously, when working with a legacy product like FogBugz, the costs of implementing certain practices may vary a lot, and we have to take that into consideration, but the inspiration that comes from seeing what is possible keeps us working to make things better.

When I look at this report card, I feel like my oldest son a little bit. He’s our perfectionist, always expecting to get everything right the first time through, and pretty disappointed when that doesn’t happen. We’re hoping that some of the changes we make in our homeschooling will help him relax a bit so he can learn more freely. And yes, it’s somewhat humbling to look at our build and release report card and admit how far we have to go. But I have high hopes that in another six months we’ll have improved our grades again.

 

[1] What about Trello?

Trello has the awesome advantage of being greenfield development. As such, they’ve really started from the ground up by doing the right things when it comes to builds and releases. They frequently release midday with no downtime, they have a nice staging environment, their releases take almost no time at all. They have run into some interesting load challenges with updating their clients after an upgrade, but I’ll let the Trello team talk about that in their own sweet time.

Faces with Your Cases: Profile Pictures in FogBugz

August 2nd, 2011 by Steven Banks

Picture this.  You’re a FogBugz user and you were just assigned a case by Joe Shmoe from the other side of the floor.  You’ve heard his name a couple of times, and could probably pick him out if you had six guesses.  Anyway, after reading through the case,  you realize you know neither (a) why this case was assigned to you, nor (b) what at all you’re being asked to do.

At this point you have a few options.  Normally we would suggest that you assign the case back to Joe, asking for more clarification.  Or you could edit the case with your cry for help and notify Joe.

But this case has been set to FIX-ME-NOW-OR-WE-ALL-EXPLODE priority, and bouncing cases back and forth can take a while.  Why don’t we just find Joe and have a short chat with him?

But recall: you don’t know who Joe is!  Maybe your company has a directory.  And maybe it’s even up-to-date!  If so, that’s great!  Use it!  Find Joe!  But maybe there’s no directory.  And walking around looking for someone that you maybe should already know anyway doesn’t sound thrilling.

We Creekers aren’t too keen on awkward social embarrassment.  To spare you some, or maybe just to give you a chance to show off your favorite lolcats, we’re happy to introduce enhanced profile info in FogBugz!

We’ve been really busy working on the newest version of FogBugz and some of its features are getting an early release as we get them done.  This is one of them.

If you mouse over Joe’s name you can see his info — including a profile picture —  and you realize you would have picked him fourth if you had just guessed.

popup

You can upload a picture, or use Gravatar, or use the super-cute default-kiwi.  Licensed customers just need to upgrade to FogBugz version 8.5.122.  The feature is already live in FogBugz On Demand.

Default-Kiwi

By default, your Gravatar is used (or the kiwi, if you don’t have one).  If you don’t like the kiwi, or your Gravatar isn’t — ahem — work-appropriate, or you just have that picture you’ve been dying to show off that you uploaded as your Facebook profile picture but then everyone moved to Google+ and didn’t invite you so now you’re bitter and don’t want to join anyway, you should totally check out your User Options page, by clicking on My Settings (in the top right) –> Options.  There you’ll find your current picture, with options to upload a new one, use the kiwi, or use Gravatar.   After you make a choice, you get a sneak peek before having to commit.  Maybe that picture doesn’t look so great after all…

My Settings --> OptionsUser Options for Joe Shmoe

We’re also excited to announce that profile pictures have already jumped into Kiln.  If you’ve ever tried to follow a code review in which lots of people commented, you might remember how hard it is to tell who said what.  Those days are behind us!  Profile pictures in code reviews make such complicated reviews much easier to follow.  Even if you don’t recognize the pictures, you can see immediately how many commenters there are and which ones said what.

Code Review

And while Kiln code reviews are the first area to be graced with your attractive face, my bet is that they won’t be the last.  There’s a simple API in place for developers of FogBugz, Kiln, or any of their plugins to inject profile pictures wherever they see fit.   Just use the function GetProfilePictureUrl(pxSize) on the CPerson class, where pxSize is the sidelength in pixels of the square image you want (available as of API version 3.46 and FB version 8.5.122).  Then use that url as an image src and you’re all set.  And while profile pictures might seem a little gimmicky, they have the potential to add a touch of personality to an otherwise let’s-get-down-to-business product, while also greatly improving clarity and usability in places like code reviews or — imagine! — case histories.

 

Joe Shmoe’s profile picture is a CC image courtesy of Rodrigo Sá, and is originally entitled Série Nerd.

Project Groups – Reviving a Lost “Feature”

June 21st, 2011 by Adam Wishneusky

For FogBugz 8.0, we completely revamped the way permissions work to make them match what users are used to seeing in other applications and in operating systems.

People don’t think of adding projects to groups; they think of adding users to groups. It turns out, however, that adding projects to groups was something people found useful. (The evidence points to them finding it more useful than the permissions feature itself.) You could add all your projects your consulting client, Kitten Industries, Inc. to a group, then filter by that group and see all the cases in those projects. The group didn’t even have to have people in it!

To solve this problem, we enlisted the help of the FogBugz ecosystem. Super-user John Fuex used the FogBugz plugin architecture to implement Project Groups to revive the ability to collect various projects together as one could back in FogBugz 7.

Project Groups are easy to configure as an administrator from the Plugin page. Groups may contain many projects and projects may belong to many groups.

Project Group List

Specify a Project Group to show when editing a filter or search for a group with the projectgroup: search axis.

Searching for Project Groups in FogBugz

Project Groups is downloadable from the Plugin Gallery for FogBugz For-Your-Server, or if you use FogBugz On Demand, it is installed by default (available tomorrow).

Case Dependencies

June 17th, 2011 by Rich Armstrong

For years, Fog Creek’s approach to developing products has been guided by a simple principle elucidated by a fella named Joel Spolsky:

Design, for my purposes, is about making tradeoffs.

The big trade-off we’re always making is between power and complexity. If you want to see the effect of blindly adding features, just open iTunes. (Don’t worry, we’ll wait.)

Many people who first open FogBugz and start using it very quickly come to two questions:

Question 1: How do I delete a case?

It’s not like in nine years of developing FogBugz, we didn’t have time to add a delete button. We just found that closing a case was good enough for 99.9% of eventualities. And solving that last 0.01% would add significant complexity. We can add deletion to satisfy some customers, but not without losing others. A trade-off.

People who need to actually excise cases from their database can do so. Even so, we have thousands and thousands of customers happily using FogBugz On Demand with no ability to excise data, and they never complain about it. We get a request to delete an inadvertently emailed credit card number from a case maybe once a month.

Lack of deletion might confuse you for the first 5 minutes, like the 5 minutes you spent looking for the Off switch on your first iPod. Steve Jobs didn’t include a note with every iPod:

BTW: there’s no Off switch. — luv steve

He knew you’d figure it out.

And those people, the ones who absolutely need the Off switch on their music player or else they can’t use it? Well, they found themselves on the wrong side of a design trade-off.

Question 2: How do I indicate that A needs to be done before B?

We have several good ways of indicating this in FogBugz. Milestones are the primary one. Parent-child relationships can do a pretty good job. The Project Backlog plugin is probably the most granular way we offer.

The venerable open source bug tracker, Bugzilla, though, offers blocking. Blocking seems handy. You can reflect in the system that

“I can’t do this until Jim delivers his logging library.”

It seems very reasonable (like having an Off switch on a music player). It’s the second thing many people think when they open FogBugz for the first time (including your correspondent, years ago).

In the software world, though, these kinds of true dependencies come up very rarely. Even in manufacturing, deciding that something is a dependency too early can be really inefficient. For instance, if you’re making espresso machines, you’re going to need a heating element. Is that a dependency? Yes. Is it blocking? Up to a point. There’s a lot of the espresso machine you can make while you wait for your heating elements to clear customs. When they arrive, you’re already well into the process.

In software development, it’s the same, but moreso. Programmers have mock objects and stub functions. These define an interface but just dumbly return a single (or random) value when you interact with them. These are easy to write, and are the classic way of making progress while you’re waiting on another engineer (or company) to deliver. If you could put a fake heating element into your espresso machine that would magically transform into the real heating element (when they clear customs) without having to take the whole dang thing apart, that would considerably reduce the number of true dependencies in your manufacturing process. (And, you’d probably crush your competitors in the espresso machine business.)

Software engineering can be a pretty parochial endeavor. We left blocking out of FogBugz on purpose. We hoped it would encourage efficiency and cooperation. Stub in a function and stop obsessing over Jim. He’s probably not the one stealing your lunch from the fridge anyway. (It’s actually Margaret.)

But we still kept getting the question, and our answer was always a version of the above. To be honest, we’ve never been thrilled about this trade-off. It’s not as clear-cut as the deletion or required field trade-off.

And then we released a plug-in architecture, and our answer became the above plus, “But if you really, really wanted it, you could write a plug-in.” Even if we didn’t think of this as a bluff, the very talented developers at Quarantainenet called it anyway.

The team at Quarantainenet wrote a Case Dependencies plug-in for their own usage. Version 1 came out last summer, and they’ve continued to improve it. They’ve generously made it available to the world for free. We really think they’ve done an excellent job with the interface work. In filter/grid view, you can filter blocked cases so they’re hidden. You can sort cases by number of cases they rely on, or number of cases they’re blocking.

Filter by various blocking options

One big feature that argues for using the plug-in, even if you’re not going to use it for Bugzilla-style blocking, is the ability to have cases automatically reactivate when all their dependencies are closed. This fills a long-unmet need for customer communication: “Tell me when my fix is released.”

Reactivate resolved cases when dependencies close

On top of that, Quarantainenet has put us to shame by offering comprehensive PDF documentation of their plug-in, something that’s not currently available for FogBugz itself. We’re lucky to have them as customers.

This really highlights the best part of the offering plug-ins. It allows us (and our users) to offer additional functionality without complicating the basic user experience. Anyone who adds the plug-in knows exactly what they’re adding, and is expecting the additional complexity that comes along with a voluntary addition of features.

It also highlights to us the awesomeness of having developers as customers. They have the ability to change things, make things better, if you give them the opportunity.

Marnix and the team at Quarantainenet say Case Dependencies really works great with their workflow and has renewed their love of FogBugz.

You can download the plug-in from our plug-in gallery.

 

Get More Data From Your Screenshot Tool

June 16th, 2011 by Ben McCormack

Many of our customers are already using one of several great screen capture tools to easily get screenshots into FogBugz. However, if you’re doing QA work for web sites, you likely need to add additional information to the case once it is submitted. BugDigger takes care of some of this extra work for you by providing detailed data about the user’s environment when the screenshot is captured.

For example, let’s say I’m reviewing the FogBugz home page. BugDigger provides a Firefox extension that lets me create a bug report for this page in one click.

clip_image002

From there, it’s very straightforward to enter some basic details, edit your screenshot, and submit it to FogBugz:

6-14-2011 11-31-47 AM

This creates the case within FogBugz with some information about the user’s environment:

6-13-2011 3-59-23 PM

Clicking the BugDigger link within the case takes us to even more detailed information, including the user’s previous actions at the site before submitting the screenshot:

6-13-2011 4-30-22 PM

BugDigger has a fully functional free trial to get started with their service. In case you don’t already have a FogBugz account to link to, you can give us a try as well.

FogBugz GitHub Integration

March 15th, 2011 by Philipp Tsipman

GitHub LogoHas your version control gone distributed?  Do you now have GitHub hosting your code?  GitHub integrates directly with FogBugz, and it takes just a few minutes to link the two up!

Here are the steps.

(Many thanks to David Kennedy for kindly putting these together.) To get FogBugz 8 On Demand talking to a private GitHub repository:

A. Configure FogBugz

  1. Go to ‘Admin’ -> ‘Source Control’, and add a generic repository.
  2. Give it a name like ‘GitHub – [repo name]‘.
  3. Note the ‘ixRepository’ number on the ‘Redownload Scripts’ URI.
  4. Set the ‘Diff URL’ to:https://github.com/[company]/[repo]/commit/^R2
  5. Set the ‘Log URL’ to:https://github.com/[company]/[repo]/commits/^FILE

To be clear, if your repo displays as ‘/fooinc/foo’ in GitHub then [company] should be ‘fooinc’ and [repo] ‘foo’ in the examples above.

B. Configure GitHub

  1. Click on your repository.
  2. Go to ‘Admin’ -> ‘Service Hooks’ -> ‘FogBugz’
  3. Set the ‘Cvssubmit Url’ to (note the https):https://[company].fogbugz.com/cvsSubmit.asp
  4. Set the ‘Fb Version’ to ’7.0′
  5. Set the ‘Fb Repoid’ to the value of the ‘ixRepository’ from step 1.3
  6. Tick the ‘Active’ checkbox.

During a git commit, you can provide a comment of the form: “See bugzid:123. Brief summary of the fix. Longer explanation.”

Enjoy it!  Happy coding.

AppDomains, AppDomains: Can’t live with ‘em…but can live with fewer of them

March 13th, 2011 by Tim Stewart

Recently we noticed that FogBugz was slowwiiiiinng down.  Some customers still praised it compared to the competition. But, it was definitely no longer as instant as we were craving it to be. We decided to investigate.
We discovered the culprit quickly enough:  AppDomains.  Yes, those fuzzy wonderful things that keep FogBugz humming for thousands of customers each with their own user Plugins, Extras, BugMonkey scripts, API calls, and whatnot.

The problem started rather innocently.

We want to allow third-party plug-ins to run on FogBugz On Demand, so our developer community can continue to expand and improve FogBugz’s functionality. But to protect your FogBugz instance from being affected negatively by code we don’t directly control, we segregate each plugin’s operations into its own AppDomain, a discrete area in which the plugins can operate without affecting one another. We created one AppDomain for each account running third-party plugins.
It turns out, IIS/.NET don’t handle hundreds or thousands of AppDomains very well. The marginal performance hit for each additional AppDomain grew with each AppDomain that was added.  The graph below gives an idea of this extra-linear growth. With 33% more AppDomains, we saw a 400% increase in response times.
Performance degraded extra-linearly for each new AppDomain, and we were adding AppDomains linearly per new customer (one new AppDomain per customer per plug-in).
We’ve contacted Microsoft to learn why AppDomains scaling is not linear, but it’s rather clear that it’s not.

Solution

Once we determined what the problem was, we figured this: theoretically, any given plug-in only needs one AppDomain to run across all FogBugz On Demand accounts. We could load each plugin into its own AppDomain, since there are far fewer plugins than there are accounts.
The only potential issue was that a value stored in a static variable in the AppDomain would be visible across multiple accounts. This caused quite a bit of concern until one of our engineers pointed out that we already vet On Demand plug-ins for security, with a rigorous review for XSS vulnerabilities. If we’re willing to trust our review process to guard against nasty XSS attacks, why aren’t we willing to review for static variables to prevent data bleedover?
We reviewed all plug-ins available to FogBugz On Demand and found zero need for changes outside one plug-in developed in-house. (It shall remain nameless… but if you wanted to download the source code for old versions of all the plug-ins and unzip and pore over the source code, you could probably figure it out).

How did we do?

Just to give a sense of the improvement, our web servers each had on the order of 1,000 AppDomains before we deployed the fix. Once we deployed the fix, they each had less than 50. Response times, accordingly, got much much better.  Since deploying, we’ve also noticed that our web servers and our backend servers are using much less CPU and memory.
Our next performance attack is client-side.  We’re also improving performance profiling of installed FogBugz so we can be alerted early in testing as to whether a new feature degrades response times.
Please keep an eye on your FogBugz performance.  We hope you’ll be seeing it get faster and faster!

A 7-Year-Old Bug Closed: FogBugz 8.3 with Rich Case Editing

February 9th, 2011 by Philipp Tsipman

We finally closed a bug from 7 years ago. Starting with version 8.3, FogBugz supports:

Rich text cases

And rich text email!

Currently, we're only supporting outgoing rich text email, but we've already got plans to add support for incoming rich text email as well.

Sometimes technology moves at break-neck speed. Sometimes, it takes years. For Apple, it took 22 years to go from the idea of a touch-based "Knowledge Navigator" to its successful implementation as the iPad. The clincher was: inexpensive glass coated in a grid of indium tin oxide. For us, the clincher was adding CKEditor to our wiki this summer. It took just a few more months to finish the rest. We hope the wait was worth it.