Fog Creek

Saving Agile with BDD and Cucumber – Interview with Matt Wynne

Looking for audio only? Listen on and subscribe

Up Next
Secrets Behind Etsy’s Remote Working Success

In this interview with Matt Wynne, Co-founder of Cucumber Ltd. and author of The Cucumber Book, we discuss Behavior-Driven Development (BDD). We go into the typical problems Agile teams have, and how BDD can help. Including the importance of conversations, automated testing, refactoring and concrete examples. We also discuss common mistakes made by those new to BDD and its drawbacks.

Content and Timings

  • Introduction (0:00)
  • Common Problems Among Agile Development Teams (0:37)
  • How BDD Can Help (1:50)
  • The Importance of Conversations (2:51)
  • Why Refactoring is Often Forgotten (5:29)
  • Getting Started with BDD (11:33)
  • Common Mistakes with BDD (13:37)
  • Recommended Resources (16:04)



Matt Wynne is lead developer of Cucumber, the popular open source acceptance testing tool, and one of the founders of Cucumber Limited. He’s also the co-author of two books, The Cucumber Book and Cucumber Recipes. He regularly blogs and speaks at conferences about behavior driven development including the talk “Why BDD can save Agile”. Matt thank you so much for joining us today, it’s really a pleasure. Do you have anything more to share about yourself?

Actually the lead developer of Cucumber overall is my colleague and business co-founder Aslak Hellesøy, so I shouldn’t take all the credit!

Common Problems Among Agile Development Teams

From your experience in working with Agile teams what problems do you typically see them having that Agile needs saving from?

Honestly I can almost predict when I get a sales call the story they’re going to tell me. They’re going to say “We started using Scrum about 2/3 years ago, we’ve got a lot of small co-located team but we still end up finding a lot of defects. A lot of bugs are getting found by the testers after the sprint’s finished or at the end of the sprint. We’re struggling to finish all of our stories in a sprint because we’re getting a lot of bugs. The stories are turning out to be bigger than we expected. We very often don’t meet our sprint goals. Or we see that our velocity is declining over time. We just generally think that this is a problem with cooperation with communication.”

Yeah, I see these issues over and over and over again with teams that have adopted Scrum as a sort of project management practice but probably not adopted the rest of the practices that you actually need from extreme programming to make software development, to make Agile software development actually work.

“If we talked more about cleaning up our mess, rather than refactoring… it would make the conversation… about why it was valuable a lot easier”

How BDD Can Help

Okay, so why is BDD the answer?

What I love about my job is that I come into these companies where basically there’s people that care and they’ve got this new culture of co-location and self-organization and collaboration. They understand that that’s the way things ought to get done but they’ve missed some techniques for how to actually make that fly, especially the collaboration between the technical people who write the code or do the testing and the business people who want the software done. Even chopping things up into small user stories, you can still end up waiting two weeks until the end of a sprint to discover a misunderstanding. You still invested two whole weeks of people cranking code to find that they were cranking out the wrong code, that solved the wrong problem.

The Importance of Conversations

What we try and teach when we’re teaching teams how to use Cucumber, is not just how to write tests so that they can be automated using Cucumber the tool, it’s all about getting the business person, the tester and the developer to sit down together and decide and discuss what is it that we want the software to do. When we get to the end of the sprint what’s it going look like, how is it going to behave. That discussion throws up so much of the potential misunderstanding, assumptions that otherwise they would have only come across as they’ve gone through the delivery of the thing.

Really the magic of BDD is in playing the game of, if we have to explain to the computer how to test the behavior that we want, we have to have figured out the behavior we want. By collaboratively doing that, by sitting down together and doing that we have to thrash out between those three groups what is it that we want so the tester and the developer and the business person are all on the same page about what is it actually going to mean for this story to be done. By the time they get to writing the code it’s a much more straight forward process. A lot of those potential bugs have been ironed out.

Yeah, having these conversations is a really important part of the process.

Right. I think we have a combination of… We’re writing the tests, the acceptance tests in a way that they’re completely accessible to everybody, so everybody can get involved in the process. We have that conversation happening early on, but then we can automate those tests, so we’re also able to do test-driven development to drive out the behavior that we want because we’ve figured what the examples are that we want the software to do and now we can very easily turn those examples into automated tests. Actually doing the development test first has all of these benefits about making sure that you build just enough code to make the software testable, getting feedback from the test about whether the code is easy to test and therefore is going to be easy to modify in the future.

Also you’re able to safely do refactoring, and I think that’s the key benefit that most of these teams that have just adopted Scrum but not the other XP practices, have missed out on is that if you haven’t got good automated tests, it’s impossible to do refactoring. If you can’t do refactoring your code is just going to get gnarlier and gnarlier and more and more expensive to maintain over time.

“Day to day programmers go to work in this thing that is just… It’s a mess”

Why Refactoring is Often Forgotten

You say that refactoring often gets forgotten or put off. Why do you think that that’s the way it is?

I often think that part of the reason is because it’s invisible. Code quality, code cleanliness is invisible. If you imagine that you run a restaurant and the chefs were producing these meals that were unhygienic, the meals are coming out really slowly now and people are getting ill and complaining about the quality of the food, you would be able to walk into the kitchen and you could see this kitchen’s a mess. There’s shelves hanging off the walls, there’s a sink full of dirty dishes. Nobody can get anything done in this place.

Most code bases are in that kind of a state and day to day programmers go to work in this thing that is just … It’s a mess. We just kind of cope and get by and grab a pan and scrub it out a little bit and fry an egg in it and then give it out to the customers. We just cope all the time with the chaos and the mess. There’s a little bit of that that’s always got to be tolerated and go on but I think we don’t own up to it. I think, again, using the cooking analogy, it’s impossible to prepare a meal at home and not have to do a little bit of tidying up afterwards. Right?

But most software engineers will build a feature and go “There you go it’s done.” and they want to move on to the next thing. They don’t take that time to go and accept ‘I’m now going to have to spend a couple of hours here tidying some things up.’ I think that business people not seeing the mess that’s there is part of the thing. I think that business people not understanding the business benefit of doing it is part of the thing. I think the name refactoring is unfortunate because it sounds very technical, doesn’t it? I think if we talked more about cleaning up our mess, rather than refactoring, I think that it would make the conversation between developers and business people about why it was valuable a lot easier.

Talking about examples, is there an example of a ‘concrete example’?

I think the concrete example’s become, like really, really concrete, become most useful before you even get to test automation. It’s in the conversation. I very often do this thing in the training world I’ll just say to somebody, or I’ll say to the group “Somebody give me an example of the last time you used a piece of software.” The person will say to me “Okay, I checked my email on my phone.” Then we’ll say “Can you make that example more concrete? Can you tell me more about where you were, why you were checking the email?” Then they’ll say “Okay, I was on my commute to work this morning and I was getting a ride with my colleague. We were in a traffic jam and I pulled my phone out and I checked my email.”

Now we have a bit more context. The more concrete the example gets, now we know they’re on a phone, what kind of phone were they on? The more concrete the example is, the more vivid it is, the more it helps us to explore the problem domain and see these potential edge cases that we’re going to have to probably build if statements for.

So asking these questions, really digging in and not just making assumptions about what the process is?

The more you stay at that “check my email” level of abstraction, the more it becomes okay for everybody to go “Sounds fine.”, and you just miss the chance to dig in and discover those things that are going to end up meaning that you have to stay late on Thursday night to finish the story in time for demo day.

“The magic of BDD is in playing the game of, if we have to explain to the computer how to test the behavior that we want, we have to have figured out the behavior we want”

What are the main drawbacks or costs of BDD?

I think there’s a part of the problem solving space where you don’t understand the problem well enough and this is too rigorous a mechanism to apply. A very good example is that right now the Cucumber team are building a bunch of iterative prototypes for what the next version of Cucumber Pro, our collaboration platform, is going to look like and we’ve been testing those out with our users. Those prototypes haven’t got any tests. We’re just building them, we’re learning about React.js, we’re just knocking them together in the leanest, smallest thing that we can put together in order to put it in front of a user and say “What do you think? Would it be good if it worked like this?”

But we’re going to throw that code away once we’ve got that feedback from the user and then if it turns out to be stuff that we want to invest in, that’s when we’re going to apply BDD. I think that it can have a drawback if you try and apply it too early, when you still don’t understand the business problem well. I think the other obvious drawback for a team that are in that place where it would be good for them, is that it’s going to slow them down for a bit because they’ve got to learn new skills. It will feel slower anyway because your front loading your process with all of that uncertainty. Rather than putting your finger in your ears and ignoring the problems and misunderstandings that are going to come along later and diving in and starting to write some code and then waiting until the testers tell you everything is wrong with it in the next sprint, you’re trying to figure that stuff out now.

That figuring stuff out now takes a bit longer than it would have done to just pick up the keyboard and to start hacking and trying to write a test as you go takes a bit longer than just hacking together a solution. The payback comes in the longer term when you need to come back and change that code, it’s easier to change because there’s tests with it. But also the payback comes when you give it over for testing and the testers can’t find anything wrong with it because it does all of the stuff that you agreed they wanted you to make it do.

Getting Started with BDD

We’ve talk a lot about BDD here but how do I get it introduced or get it started with my team?

What we saw a lot of in the early days is that we’d have developers who had picked up Cucumber as a tool and they were trying to draw the business folks or the testers into the process of writing those acceptance tests. They were already developers who were bought into doing TDD and they wanted to have more involvement in writing their tests so they would pull people in. That can be one way. The other way that we often see is where you have a separate silo of testers. You have developers over here writing bugs and then testers over here finding them and the testers start to use Cucumber. I feel like this is a broken situation in the first place, I want them to just work together so that they don’t make the bugs in the first place, but it’s a start.

If there’s a start that this group now start using Cucumber, then they can start to maybe have the conversations about what should go into the Cucumber tests before the code gets written. That’s really where I think most teams will start to get a foothold in this process is. Simply by sitting down and doing, there’s this pattern a three amigos conversation where you have the business analyst or product owner, the tester and the developer, sitting down together. Just for 20 minutes before a story gets pulled into the sprint to discuss the acceptance criteria and try and give examples of all those acceptance criteria. Make sure we really understand and we’ve really seen the full scope of this story and think about what will those acceptance tests look like. Because if you can sit down and get an agreement before you write the code, you’re a lot more likely to build the if statements right the first time to actually cater for all of the different edge cases. I think that three amigos is the best place to start.

“If you can’t do refactoring your code is just going to get gnarlier and gnarlier”

Common Mistakes with BDD

One mistake I see them do a lot in that session is they’ll try and write an actual Gherkin feature file. They’ll get a laptop and a projector and they’ll try and type it out. It can be pretty boring for a product owner to sit there watching people typing in Gherkin. Definitely not the right way to do it. I think it’s much better to just have the conversation about the examples, go off and write your acceptance tests and then give them to the product owner to read, say “This is my understanding of the system you want me to build. Read this and tell me if I’ve understood it correctly.”

You’ve mentioned one mistake already in terms of people just starting to jump into a particular tool rather that starting with a more lightweight approach. Are there any other common mistakes you see people making with BDD?

In the early days the team are going to get sort of addicted to writing examples. They’re going to start to love it. They will try and describe every single thing they can think of that software should do in Gherkin. Then they’re probably going to try and use Cucumber to automate a full stack acceptance test that hits every layer in the system right from the GUI to the database. They’re going to try and use Cucumber to automate that example through all of those layers of the stack.

Those two things in combination are a real problem in the long run because that’s how you end up with 15 hours of tests. Because you try to document every single example that you want the system to do and you’ve tried to test it through the full stack. Two things can make this better. One thing is to think about, these are all examples of behavior that we want but how many of them do we really need to retain in a business readable format? How many of them could just be pushed down to being written in JUnit or RSpec tests? They could just be unit tests and also even of the ones that we want to retain as business readable, how many of them need to hit the entire stack?

Could some of these examples that really matter to us actually just pick up part of our domain model and test that bit of the domain model? Those are two ways of thinking about, we’ve defined examples that we want in the behavior but what are good long term ways of making the automated tests that validate that behavior, making them maintainable over the long run.

Recommended Resources

Can you recommend any resources for those wanting to learn more about Behavior Driven Development?

There are a few really good books to read. I think the best book I’ve seen about the overall process is John Smart’s book, “BDD in action”. It’s a really good view of how to make BDD work.

This is a really good book, it’s quite an oldie now but goodie, “Specification by example” by Gojko Adzic. What’s good about this book is that Gojko went around and interviewed tons of practitioners and discovered the details of what was working and not working for them. What you’ll find in here is lots of evidence about how to do it and how it’s worked in other places, which I think is really valuable if you’re trying to convince some skeptical people in your organization about “Hey, is this just a crazy new fad or something that really works?”

“Bridging the communication gap”, Gojko’s old ranty book as well, which is also very good. There’s our books, “The Cucumber book” I think is pretty good. From a practical kind of actually using Cucumber point of view, we’ve produced a video series. They get really deep and they’re talking about the stuff about the testing pyramid and the depth of tests. We work through an example. I think Cucumber School as well is a really good resource.

Matt, thank you so much for joining us today.

Thanks for having me.