Archive for the ‘General’ Category

Are You Dogfooding in to a False Sense of Security?

October 30th, 2014 by Gareth Wilson

Dogfooding
Like many of you, we’re keen Dogfooders here at Fog Creek. Since the phrase took hold at Microsoft in the late eighties, the practice of using your own products to help test them before wider release has become widespread throughout the tech sector and beyond.

And with good reason. Forcing yourself to use your own products means you end up fixing bugs before creating new features. It means you spot and resolve pain points before your product is out of the door. It also means spreading product knowledge throughout your organization.

Hear some more about Dogfooding here at Fog Creek in the following video:
 

But are you just Dogfooding yourself in to a false sense of security?

Consider the following questions:
 

  • Just how typical a user of your product are you?
    Few are lucky enough to develop products for people exactly like themselves. Here are Fog Creek for example, treating our developers well is a top priority. So typically that means we’re trying our software on a new Mac with a couple of high-res 30″ screens, probably using Chrome. That’s not to brag, but some of our customers might not be so lucky. For them the norm might be an aging PC, with a small screen and of course, forced use of Internet Explorer. If we just relied on Dogfooding, we’d be missing a bunch of browser, layout and design issues.
  •  

    • Also, when was the last time you:

    • Created a new account in your software?
    • Started from scratch rather than upgrading from a previous version?
    • Tried migrating from a third-party solution?

     
    These are all common scenarios for customers, especially new ones. Yet these are all missed in typical Dogfooding.

  •  

  • Just how many customers only use your own products?
    I’d wager most use some hybrid mixture of yours along with some from other vendors. After all, it wouldn’t be unwise for an experienced IT administrator to preach use of a range of products so as to avoid vendor lock-in.

    What’s more, if you’re only using your own tools then it can lead to a form of myopia. You become so tied to your own solutions that you fail to see better alternatives. Take Uber for example. They’re disrupting the Taxi industry not because of their decades of experience in the sector, but almost because of a lack of it. They approached the space with a fresh perspective, which meant they could see another way of doing things.

  •  

  • Is anyone not reporting issues, assuming someone else will? And what about that workaround you’ve become used to, knowing the devs are almost definitely going to resolve it any day now?

 

It’s worth questioning yourself as it’s easy to become over-reliant on Dogfooding. It’s no panacea, and no excuse for poor test coverage. Nor a reason to not have a thorough QA process. With it, beta testers are still going to throw up those big issues that leave you wondering just how you managed to use it and miss it!

So, yes, you should use your own products. But what you really want to do is use your own products like a customer. Better yet, use them like a new customer would.

How to Get Started –and Stick with– Usability Testing

October 27th, 2014 by Gareth Wilson

As you’re working on a piece of software, you get to know it well. Perhaps too well. How to complete common actions become so obvious to you that you don’t need to think about them any more. This isn’t the case though when someone new comes to use your product. So getting your product in to the hands of users to test is a key step in its development. But of course, you already know this. Almost everybody does. Yet few people get around to actually doing it, and those that do, often stop after an initial release or phase.

Check out the video below featuring usability expert and author of “Don’t Make Me Think”, Steve Krug. Then read on for tips to help you get started, and stick with, usability testing.
 

Usability Testing doesn’t have to be complicated

Keep it simple
Usability testing doesn’t have to be complicated. Upon hearing the term ‘usability testing’, you might begin to imagine a testing lab, with cameras setup to film eyes, face, clicks and the screen. But it doesn’t need to be like this. All you really need is to find someone, provide something to test and a place to do it.

Take anyone you can get
Test participants don’t need to be familiar with your product. They don’t even have to be from your target audience. So grab a colleague (in our video we roped-in our Office Manager), a friend or anyone else who is around.

You don’t need hordes of people either. In fact, there’s little benefit in testing with more than 5 people.

Test early
You don’t need to have built something to begin user testing. Some sketches, linked screenshots or a simple prototype will do. The earlier you start the better, so you can avoid building unnecessary stuff later. A paper prototype can be enough to begin and start getting useful feedback.

Don’t get caught up with design preferences and feature requests
Your testers come with pre-conceived ideas of what’s expected of them. Most common is the idea that you’re after design feedback, in particular colors. Layout and design issues are important, but things like color are rarely your major problem. Unless people have a strong reaction (using words like “puke”), it’s often safe to ignore such feedback. Same goes with feature requests. You will already have loads of your own. So unless their idea immediately makes sense to all, you can skip these too.

Start with a few theories
Generate a few theories about why something important is happening. Such as why users aren’t converting, not using a particular feature or how they might complete a particular task. Think of a scenario to test this, and then run that test. After each test, debrief – discuss amongst you what you observed and why that might be.

Test once a month
Once you have carried out a test, make some changes, create more theories and set about testing them too. It doesn’t need to be too often – just one morning, once a month. This will provide a constant flow of feedback that will keep you on top of your major issues.

Once you’re in to it, use tools to help
There’s a bunch of useful tools which can help you with usability testing. Use Five Second Test to get quick feedback on UI, web pages and sketches. InVision for creating prototypes. LookBack (mobile) and SilverbackApp (desktop) to get in-app user-feedback.

If you try out these tips then there’s really no barrier to you starting and sticking with usability testing. Don’t forget to check out the video above to hear more on usability testing from Steve Krug and some of the Creekers.

7 Ways to Get More Out of Beta Testing

October 23rd, 2014 by Gareth Wilson

The weird and wonderful bugs that get thrown up when real users first start using your code never ceases to amaze. There’s always some odd edge case that had been overlooked, despite you having thought about little else for several weeks. We’ve been through this many times whilst developing FogBugz, Kiln and Trello. Beta testing is an important stage in the testing process. To learn a little about them check out the following video, then read on for our key tips:

 

 

Here’s 7 things you can do to get the most out of our your beta tests.

1. Ask for a commitment to provide feedback
Response rates will be higher if you ask your beta testers upfront to commit to providing feedback. This doesn’t have to be formal, just part of an application form. But having agreed to it, people are more likely to follow through.

2. Do not release with known bugs
Most beta testers will only provide feedback once. So you don’t want to burn any tester to just hear about known issues.

3. Allow enough time
Use the following as a rough guide. For a major development effort, say about a year’s work, then you’ll want 10-12 weeks for beta testing. Decrease as necessary – so if it took a month to develop, then around a week will suffice.

4. Be feature complete
Only beta test when you’re feature complete. Adding in things as you go sets you back to the start. Otherwise, it just means the new code and its impact on existing functionality isn’t as well tested as the rest. Something you’ll regret later.

5. Make it easy to get in touch
You want to make it as easy as possible for your beta testers to provide feedback, so give them direct emails and offer to jump on a Hangout/Skype if they’d prefer.

6. Remind, but don’t annoy
Whilst your product might be front and center for you, it’s not going to be that way for your beta testers. You’ll want to remind them along the way. But don’t over do it, they’re helping you out so you don’t want to annoy them with too many emails.

7. Don’t forget to provide feedback
Make sure during and after the test that you send them updates about how you are putting their feedback to use. People like to hear that their time wasn’t wasted. And don’t be tight with the swag – a free t-shirt can do wonders!

To learn more about beta testing be sure to check out the video above.

Fog Creek and Heartbleed

April 10th, 2014 by Mendy Berkowitz

Along with the rest of the internet, Fog Creek has been reacting to the Heartbleed vulnerability which was discovered this Monday.

TL;DR: FogBugz and Kiln On Demand, Copilot and the Fog Creek website were not vulnerable. Trello was vulnerable and has been remediated.

Fog Creek handles SSL connections at the load balancers in front of the application servers. The load balancers for FogBugz and Kiln On Demand, Copilot and the fogcreek.com website are using the 0.9.8 branch of OpenSSL. This version does not have the newer heartbeat extension and is therefore not vulnerable to Heartbleed. None of your data on fogbugz.com, kilnhg.com, copilot.com or fogcreek.com has been exposed to the Heartbleed vulnerability.

The load balancers for Trello were using the 1.0.1 branch of OpenSSL and were vulnerable. We have upgraded OpenSSL and replaced the Trello certificates. More details and important recommended steps for all users are available in a post on the Trello blog.

We have also reviewed our vendors (for things like this blog) to ensure that our other SSL certificates were not potentially compromised. Fortunately we have not had to replace any other certificates.

If you have configured web hooks in your Kiln On Demand account, we recommend that you verify that the target servers have not been affected. If you use one of the pre-configured types, or your custom web hook uses HTTPS, the data sent to that external service may be at risk. We do not however, send any of your Kiln login credentials, so they are safe.

We take the security of your data seriously. We are committed to protecting it and to maintaining clear and open communication with you. If you have any questions, comments or concerns please contact us.

Fog Creek’s Ad Hoc Remote Work Policy, or, Working From Grandma’s House

September 12th, 2013 by Rich Armstrong

Not grandma.
Not grandma.

source

Once every few months someone walks into my office and says, “Hey, I’ve got this (reunion|wedding|grandparent). I’d like to figure out how to spend some more time with my family. Can I work remotely from my grandma’s house for a few days?”

I always said yes, and got pretty uneven results. When someone wasn’t productive, it was usually due to unforeseen circumstances that seemed obvious after the fact. So we started a list of things you’ll need for a short stint of working remotely. This list is a “policy” inasmuch as it represents all the things you need to do in order to be “covered” while working remotely for a short stint. If something foreseeable comes up that’s not on this list, then it’s our problem, not yours.

Two important things that this is list is NOT applicable to:

  • Working from home waiting for the cable guy or a mattress delivery. That all happens under your usual flexible work arrangement.
  • Going somewhere you’ve never gone before, like a cabin in Iceland or an apartment in Paris, where you don’t know what the infrastructure will be like. We’ll call this “working from Borneo”. We’re not against Borneo, mind you. It’s just not what we’re talking about here.

One thing this list is kinda applicable to:

  • Permanent remote work. After writing this all down, we realized that almost everything we’d recommend to the short stint people applies to people working remotely. We’re still learning from them, so might have more to say later.

Requirements

  • A computer to work on. Duh.
  • “Approval” from your team lead. We’re not big on bureaucracy here, but this is just common courtesy.  Most of what you’ll have to do is covered below in “known availability”, but you should start with talking to your team lead.
  • An internet connection capable of easily handling a Google Hangout. Since we’ve opened up to remote workers, Hangouts have taken the place of conversations in the hallway and standups in someone’s office. Downstream is usually not the issue. Most basic internet plans are at least 5 mbps down, which is no problem. Being able to participate in a video conference usually means more that 1 mbps up. A lot of cheap internet service only offers 1 mbps up. How do you know if Google Hangouts will work where you’re going? Do a test Google Hangout with a person at the place you are going to be! (If there’s no one on the other end to do this with, the place you’re going is either vacation or Borneo (see above).)
  • Access to the resources you need to work. Make sure you can:
      • log in to your laptop as a local administrator without connecting to the Fog Creek network. This is especially true for Mac users, as Macs do not cache your credentials
      • connect to the VPN from your new location. Not all Internet connections, or routers, are created equal and not all will allow you to get on the standard VPN
      • RDP into your HQ machine, if you don’t have everything you need on your laptop

    If you’re unsure about any of this, talk to the sysadmins before you discover you can’t work remotely.

  • A headset. The built-in speakers and microphone on your laptop/monitor suck. The added friction of talking to someone without a headset makes your team members not want to invite you to a Hangout, which means you miss communication that you would normally have received.
  • A dedicated room with a door that closes. At Fog Creek, devs have offices with doors that close. People who aren’t devs are surrounded by coworkers who’ve learned to keep from inadvertently distracting each other. Your aunt has not learned this, so it’s probably a good idea to sequester yourself as much as is feasible. You need a place that is as distraction-free as your normal work environment, and maybe a little more.
  • No child care responsibilities during working hours. You can’t take care of kids and work at the same time.
  • Known availability and overlap with your team. Set standard hours and make sure people know them. If you’re not going to be working the hours you normally work, overcommunicate this to your team by posting in chat (ex: “Going to the gym for an hour”). Nothing is more frustrating to team productivity than chat messages like “Has anyone seen X?” “When is X getting online?” “Can someone else take this bug? X isn’t responding.” Try not to pop in and out. Work sustained sessions with only a few significant interruptions (e.g., lunch, dog walking, gym, light-saber fight with your nephew).
  • A dedicated phone that fits into your position’s workflow, where applicable. If you are an employee who spends a reasonable amount of time on the phone, it should be relatively easy for people to get you on the phone and for you to call them. This might require you to contact the sysadmins for phone forwarding. We haven’t done a lot of this yet, so we’re not awesome at it. Maybe you can use your experiences here to make the next person awesome?

 

Suggestions for Short Stint Remote

When you’re used to working in the office and you leave it, you’re like a snail without a shell. Consider these suggestions a temporary exoskeleton. These suggestions are not because we don’t trust you. We trust you. To get a desired effect, we often accentuate parts of our behavior that don’t come naturally. We might be introverts at a networking event, forcing ourselves to start conversations with strangers. We might be on camera, forcing ourselves to oversmile like idiots. Most of these tips are to encourage unnatural but beneficial behaviors.

  • Don’t be coy. Awareness is largely covered in “known availability” above, so this bullet point covers your mental state. You might be tempted to downplay your absence from HQ. Puritan work ethic runs pretty deep with some.  Fact is you can ship like an animal from places like Hawaii or Cape Cod (where this document was started). But you might be embarrassed to talk about it, or you might be put on the defensive by someone saying “Great beach weather.” Ignore it and ship code. Clamming up or playing down helps no one. It’s the opposite of what we want; it’s the main reason we have a policy. If you’ve met the requirements above, you’re covered. You are not slacking. You are working. You should not be shy about letting people know where you are and what you’re doing because of what they might think.
  • Put a note on your monitor saying where you are and for how long, in case people wander by looking for you.
  • Have boundaries. The people you’re going to be around might not understand that, yes, you will actually be working. Don’t worry. Your in-laws/parents will be impressed by your work ethic!
  • Over-communicate and/or over-deliver. At HQ, you have a few ways of staying involved and available (face time, informal chats, lunch) that are not available to you remotely. You still have stuff like chat, commit messages, code reviews, cases, Trello boards, etc. If you’re a Creeker, chances are good you’re a low-key person who doesn’t crow about their achievements. You might want to flip that bit for this stint. That can be hard, but it’s important to keep involved. The best way to do this is….
  • Have a deliverable. This frees you from having to assert your productivity via chattiness.

That’s it! We put this stuff out there so that you can effectively take advantage of the flexibility offered by your work arrangement and continue to be productive as part of your team. If you learn something while setting up your environment, or while working remotely, that you wish you’d known earlier, please let me know. The list you see above is a work in progress.

Get Started with Your Own Solari Board

August 8th, 2013 by Will Thompson

Some time ago we introduced our Big Board as part of our “How Fog Creek does Customer Service” series. We explained how the Big Board is useful to us because of the high density of information, but we also left you with an open ended question: “Want your own big board?” The Solari board, one of the widgets on our Big Board, is so important to our workflow that we felt that everyone else should be able to use it with anything: so we open sourced it on GitHub.

IRL Solari boards are manufactured by the firm of Solari di Udine in Italy. You’ll find them all over the world in train stations. They’re not just a great example of mechanical displays. The sound they make is functional. When the board updates, the clattering of the flipping letters alerts everyone in the waiting room to the fact that new information is coming in. It’s a beautiful solution for public displays of information. To capture this feature, we have our Solari board play a clattering sound recorded from the real Solari board in Union Station in New Haven, Connecticut.

This javascript widget is a UI metaphor. When a new tech call comes in, the board makes the clattering sound, and everyone is gently alerted to the fact that a new call has been scheduled. This might sound bothersome, but it actually helps a lot. If you look up and see that the new call is assigned to someone else, but the email address is someone you’re working with, you can quickly swipe the case. We use the “Track” column to display the extension of the assigned rep on the tech call. It looks like a real train track number, and we all know each other’s extensions.

The source code comes with an example Python script that gathers a set of cases from a FogBugz installation via the FogBugzPy Python wrapper and the FogBugz XML API.

And should your webpage ever lose connection to the POST endpoint (where the Solari board retrieves its data), the fail whale is sure to let everyone know.

 

 

Now get out there and condense that data into a nice, aesthetically pleasing, train-inspired Solari board!

 

The Solari board was originally created by Rich Armstrong as a way to display tech calls, then rewritten and refined by Rob Sobers

Friday Linkparty

October 21st, 2011 by Dan Ostlund

The things we have been reading lately

Betrayed by an accelerometer

How to draw a circle

9 Metrics for making good decisions for your startup: From the Kissmetrics blog. This is one of the best blogs on marketing, startups, good web design, and more. It’s worth frequent visits.

A walking man, done in CSS3

Some “leaks” from the forthcoming Steve Jobs biography

How to sort file names: Raymond Chen proves that computer programmers are not normal human beings.

More, more, more: A nice riff on our constant push to do more and be more.

Friday Linkparty

October 14th, 2011 by Dan Ostlund

Our latest installment

First there was the Joel test, now there is the Rands test: Go read this now for the sake of your company!

An animated history of the iPhone’s evolution And a lot of other stuff too.

Dennis Ritchie

Paul Rand and Steve Jobs making the logo for NeXT: Look for the link to an interview where Jobs talks about Paul Rand.

A beautiful, interactive exploration of understanding and solving a mathematical problem

The Price of (Dev) Happiness: Part Three

October 11th, 2011 by Rich Armstrong

Buying lunch in lower Manhattan is not cheap. If you don’t brown-bag, you’re pretty much in for nine bucks, and more if you want to sit down. A meatball sub is $8.50. The Hanover Square Deli does a respectable dduk mandu guk for about $9.50. A meager-looking, but tasty, turkey and guacamole sandwich from British import Pret A Manger will set you back $6.75 (plus tax). Or, if you really want to eke out some savings, you can get a cup of soup and a hunk of bread for about $5.

Post-lunch, Pre-espresso machine

At Fog Creek, we have our lunch catered every day.  The cost of this to the company is $15.75 per person per day. Not counting drinks and snacks, this amounts to an extra $4,000 per year per employee. So why wouldn’t we just pay people $4,000 a year more? If they want to be frugal, they can buy that $5 soup and pocket the difference.

Well, first off, since lunch is a catered on-site meeting, the cost of that lunch is 100% deductible as a business expense. If you worked here, it’d cost the company $16 to give you $10 to go buy your own mediocre udon noodles.  (Don’t take any taxability or corporate expense stuff here too seriously; I’m not an accountant or lawyer.) Second, Fog Creek’s free lunch is much more because of how it fits with the rest of our workspace and culture.

Free lunch is nothing new. Google and other big tech companies have been giving their employees breakfast and lunch (and sometimes dinner). Free lunch has been around for more than a decade because it works for attracting and retaining top talent. (Or, rather, people think it works.)

Now, the food at Google, and the people who make it, are awesome. The food at Fog Creek is good, but nowhere approaching what Google does. We don’t have Sam, the sushi chef, doing hand-rolls to order. We don’t do miso black cod, lamb shanks, or osso buco. We don’t have a raw vegan station with selections so delicious they attract the most dedicated carnivores.

Here’s what we have at Fog Creek instead: no meetings.

For us, lunch is our only recurring meeting. The only standing interruption in the day/week of a developer here. Everything else is ad hoc or temporary. (One exception is our quarterly meeting to go over financials and to grill the founders with questions.) The default at Fog Creek is no recurring meetings. Once you’ve established that, recurring meetings become the exception, rather than the rule, and tend to wither naturally as their usefulness degrades over time. For example, the FogBugz team is currently doing a stand-up meeting for fifteen minutes every day right before lunch, but this won’t last. The Trello team was doing them before and shortly after launch, but they’ve subsided.

The Kiln team lead assures me that their weekly stand-up is a real recurring meeting, but at some point they’re going to lose interest and go back to coding. It’s what they do. Meetings have a network effect. They need other meetings to legitimize them. If you’re constantly scheduled with meetings, you don’t mind being interrupted one more time. Or, rather, you don’t have the energy to protest. When you proceed from the assumption of no meetings, you have to expend effort to keep a meeting going.

It’s not all a playground, of course. There are downsides. Things, sometimes important things, don’t get communicated to the right people at the right time. More often than not, the worst result is hurt feelings or slight confusion. Sometimes it’s more than that. But we wouldn’t give it up.

For someone who’s used to a standard work environment, it seems silly, cult-like, possibly even daunting, to be “forced” to break bread with your colleagues every day. It seemed odd to me before I came here, but by the end of the first week, nothing seemed more natural. When most of your “socialization” with your colleagues takes the form of mandatory, recurring meetings over a conference table, it’s natural to not want to see them again over the lunch table. People have been making decisions about your time all day; at lunch, you need some time to yourself.

But when most of your time is spent working in a private office, taking breaks according to your natural attention span, having short chats with one or two colleagues, it’s a pleasant prospect to surface for some pleasant conversation about StarCraft or football with nice, intelligent people. And maybe you’ll hash out that new feature, too.

One can make the argument for free lunch based strictly on developer productivity. A free lunch could give you a hundred hours per year from your best people, time saved in driving, waiting in line, etc. But, consider the depressing ease with which such a gain can be wiped out by a few recurring meetings.

So, for our final post in this series, we do have a number: $15.75 per day. $4,000 per employee per year. It’s a lot of money. But without the rest of the company culture, it would be sort of meaningless. A sense of entitlement grows rapidly around any perk you offer, and lunch is no different.

A lot of this series has been based on getting hard data out there so that developers, our main audience, have an easier time talking to management about some of the things that’ll make them more happy, healthy, and productive. For this post, it’s a bit more difficult. You might get your boss to spend $16 a day, but changing the culture of meetings in any workplace is nearly impossible. (During my tenure, Google tried no-meeting Thursdays and formal meeting-reduction task forces, reminiscent of Brazil’s Ministry of Debureaucratization, to no avail.) It would require rebuilding the company culture from bedrock.

Our bedrock is the idea that, once we’ve hired good people, it’s the effort we make to direct their intention, rather than their attention, that creates value. It’s not just our lunch benefit that springs from that, but nearly every other thing we do.

For Fog Creek, our founding principles, and the pains we take to stick by them, are the price of developer happiness, and that can’t be measured in dollars.

 

Friday Linkparty

October 7th, 2011 by Dan Ostlund

Here is our latest installment.

Why code reviews beat testing: We knew we put code reviews in Kiln for a reason.

Geoffrey Moore talks about core vs. context in your business: This is fantastic talk from the 2009 Business of Software conference. If nothing else, it will make you think hard about your business.

The king of code comments?

The recent Node.js controversy

The always fascinating Richard Feynman: Three videos that use Feynman’s voice to talk about beauty, prizes, and curiosity.

Algorithm is not a four-letter word: An awesome overview of maze generation algorithms.

How to generate music with one line of code


Looking for more?

Visit the Archives or subscribe via RSS.