Fog Creek

Embedded Testers In Development Teams – Interview with Trish Khoo


In this interview with Trish Khoo, Test Engineering Team Lead for Google My Business, we discuss the changing role of the tester in software development teams. She discusses the trend of embedding test specialists within development teams, benefits of the approach, common misconceptions developers have of testing and how to successfully introduce these changes into your organization.

You can read more about this and other testing-related topics on her blog.

Content and Timings

  • Introduction (0:00)
  • About Trish (0:30)
  • The Changing Role of the Software Tester (1:43)
  • Separation of Developers and Testers (6:30)
  • Maintaining Test Quality in Mixed Teams (10:48)
  • Getting Management Buy-In (12:54)
  • Ways of Incorporating Testing in Development Teams (14:27)
  • Recommended Testing Resources for Developers (15:48)

Transcript

Introduction

Derrick:
My name is Derrick, I’m joining from Fog Creek Software where for quite a while I held the role of support engineer. Now I’m helping with the sales team. So, Trish Khoo thank you very much for joining us today from Google out in sunny California where you’re the Tech Lead and Manager for the Test Engineering Team at Google.

Trish:
At Google My Business yeah.

About Trish

Derrick:
Great. So you were previously also in testing and development roles at Campaign Monitor and Microsoft, founded the Sydney Tester Meetups, and run the TestCast podcast, you write about testing on your blog at TrishKhoo.com and you’re a contributor to the Testing Planet and Testing Circus.

Trish:
Yeah, so that’s all true.

Derrick:
All true, yeah, you’re quite heavily involved. That’s fantastic.

Trish:
I’ve been moving countries a lot lately. I’ve just moved to San Francisco to join the Google headquarters. Prior to that I was working in London, also at Google, for about two years. Keep going back in time, I was in Sydney for about 7 years before that working at various places like you mentioned, Campaign Monitor, I also worked at a place called Salmat, Microsoft before that, I had a consultancy as well. So this is the journey that has lead me to be in Google working out of Mountain View. And I often go around speaking at a lot of conferences, mainly testing conferences, mainly speaking about software testing and software engineering and how the two can meet in the middle.

The Changing Role of the Software Tester

Derrick:
Over the last ten years have you seen the role of the tester change at Google and elsewhere?

Trish:
Yeah, so I guess for me personally I’ve seen software testing roles change in my own kind of role. Like, I’ve always worked in a siloed testing role and I think as I’ve gained more experience I feel like the roles I’ve had have been more overlapping roles in terms of, not just being specialised in a testing silo so much but being able to take on more programming skills, different tasks, take on things like release work, more software development work and just seeing the value of that. In terms of how Google has seen the software tester change, definitely Google over the last five years has gone through a huge change in terms of how the testing role was defined. Only as short as five years a go most of the software testing at Google was actually done manually. And there’s been a massive shift in order to take the software testing activity to being in software engineering teams and there was a new role created called Software Engineer in Test, which meant it was exactly what it sounds like, a Software Engineer who actually knows something about testing as well. Who was creating tools and increasing developer productivity from a testing perspective, so that means making software more testable, so that it’s easier to test and being able to get that fast feedback back to developers.

Derrick:
Why do you think these changes came about?

Trish:
I think there’s a few reasons. Like, first of all I think that if you have the separate roles of a manual tester and developer it becomes quite an inefficient process overall. Separating out the testing activity from the software engineering activity is, it creates this massive feedback loop. So say you finish implementing something in code, you send it to a tester to manually test. It takes them some time to notice it, get acquainted with what you’ve written, actually test it out and then give you that feedback. Whereas if you found a faster way to do that yourself, then that’s actually saved you and the project a lot of time. So, that’s one reason, just reducing that inefficiency in the process. I think another is that when software developers know more about testing it actually influences the way that they are actually creating software in the first place. They’re designing it for testability, and thinking in terms of what could go wrong and that has a huge impact on the robustness of the software that is created.

I think that software testing as a profession is a specialised field, especially when approached from a technology perspective, that sometimes is seen as a lesser role than the Software Engineering role. In terms of compensation, in terms of just status, and we really wanted to elevate that role to something that is better recognised and gets the recognition it deserves, in terms of being an area of expertise that actually takes a lot of time and study and practice to become proficient in. So that was another reason to just raise the bar in terms of bringing people in to that field and ensuring they’re recognized for what they do.

Derrick:
So you mentioned quite a few benefits to this role some of becoming more prominent and what these changes have introduced

Trish:
Yeah, I think that there, not so many benefits in terms of, like, I think that there are a lot of benefits in making the role a more technical role and a more well-rounded role in terms of skills, rather than siloing, you’re a manual tester now. There are a lot of benefits in terms of recognizing that both of these roles have overlapping goals. In terms of, saying not so much ‘you’re in charge of creating the software’, ‘you’re in charge of making it actually work properly’, but saying that both of you are in charge of making that software and making it work properly, and the goal is to create and ship product. No what skills do you both have in terms of reaching that goal now. You perhaps are a better architect in terms of creating the software, you perhaps are better at coming up with the test approach that’s going to help that get to a good place, but in terms of every day tasks and what you can do, there’s a lot more leeway in terms of the scope of what each person can contribute to that team and in terms of value.

Separation of Developers and Testers

Derrick:
There’s a traditional way of having QA, or testers. Devs just don’t have the right mentality to effectively test their own code – ‘devs build stuff, testers break stuff.’ Do you agree with that things should be separate?

Trish:
Yeah, I’ve actually heard that excuse quite a lot and I don’t really agree with it. I think that, I do hear quite a lot of excuses from developers who aren’t used to testing, in terms of why they don’t think that they need to take a more active role in it. And I think most of the time the first mistake that developers make is in thinking that testing is supposed to be easy. And then they think ‘well it doesn’t seem that easy, so it’s probably just something that I’m not supposed to do or something that is impossible to do right.’ But testing is in fact very, very hard, especially in terms of, if you haven’t done a lot of it before in your project and there’s not a lot of tests there, the code just isn’t written in a testable way. So there is actually a mountain of technical debt to be cleaned up there that needs to be recognized, before you can say ‘ok, lets start testing now.’ And another common excuse I hear is that if I start testing then it will actually take more time, so I just have to let my project manager or boss know that if I do testing then it’s going to slow down your project. But in fact, that’s also a fallacy, because yes ok, if there’s a mountain of technical debt to clean up then you are going to slow down in the short term, but in the long term, in terms of project time, the additional time you’re writing tests during code creation surprisingly makes it faster in project time, because by the end of the project you aren’t going to find as many bugs. So you’re not going to waste any time debugging, and fixing, and delaying the project right at the end, which is the most stressful time of the project.

Derrick:
It sounds like you’ve come across developers who haven’t wanted to test, how do you go about handling that?

Trish:
I think it’s difficult if you’re in a siloed role, as I have tended to be, just about the entirety of my career in testing. It’s really difficult to go to somebody in another siloed role and say ‘hey, this is how you should do your job,’ I think that it’s best to be able to find developers on your team who already embrace that and get them to be advocates for this new approach. Ideally in has to be top-down, it has to come from management in order to really convince people of the benefits. It’s not so much about getting the stick out and saying ‘you have to write tests or else,’ but just showing them why this is useful right. Like for some people it has a been an easy sell, because they just haven’t known how to do it and you can show them ‘you think this is difficult, and it is difficult, but here’s what it is going to achieve in the end.’ I had a great quote from a developer I work with who said that he thinks that ’tests are protection of his code from other developers,’ which I thought was a great way of putting it.

Derrick:
That’s an interesting way of putting it for sure, yeah.

Trish:
I think the other mistakes that developers often seem to make is when they think about testing, is that they think that testing is for finding bugs because that is one side effect of testing. But, testing is just as much about describing the behavior of the code and also just setting expectations, and most of all preventing bugs. So I often hear the excuse in terms of ‘I haven’t ever seen any bugs that would have been caught by the test that you are telling me to write,’ and you know, that’s true for the moment, but what’s to say that it won’t happen in the future, your projects going to go through changes. And I’ve had developers to come to a new code base and ‘well, how am I supposed to know what any of this does because there are no tests in it that describe what it is meant to do? I can figure out what the code does, but I don’t know what it’s purpose is.’ So I think that there is a big mentality shift there, and there’s a stark difference between developers who are used to testing and see the benefits and developer who aren’t used to testing don’t quite understand it.

Maintaining Test Quality in Mixed Teams

Derrick:
So having the roles sort of change now, where the developers also test, if you will call it a sort of ‘multi-skilling’, some people might say it’s a ‘Jack of all trades, master of none’ type scenario, where the depth and quality of testing is reduced over time. What’s your opinion on that?

Trish:
There’s two parts of testing here that really need to be defined. I’ve heard some people describe it as testing and checking. There is testing, which we can describe as exploratory testing, where you are searching for answers to very broad questions. And there is checking, which is answering questions we already know to ask, so very binary things like ‘can I log in or not’, ‘if I click this button does the page explode?’ that kind of thing. But exploratory testing questions are more like ‘now that I’ve implemented this feature, does this existing feature work right?’ or ‘how do these two things work together?’ or ’what happens if I do this?’ I’ve got no idea, right. So this is a manual testing kind of task. And I think that people who are used to it, if they’ve practised it, they become more experience with it, and people find testers who are very, very good at that. And I think that it’s good to have these specialists on the team and I think it’s ok to recognize that we have generalists and specialists, in terms of people who are very good at one thing and OK at a lot of other things. So it’s almost like an old Dungeons and Drgaons game, where you’ve got your skill set and you say ‘I’m really good at this skill, I’m OK at this skill and really bad at this one.’ But I think that the thing to recognize is that testing needs to be a core competency of software engineering. It’s kind of OK to be OK at it, and have someone else on the team be spectacular at it, but the idea is that you should be at a minimum of OK at it.

Getting Management Buy-In

Derrick:
Earlier we sort of talked about selling testing to different people and you mentioned the importance of testing coming from the top-down. How do you get that buy-in and why should they take it seriously?

Trish:
I think that it can be really difficult and of course it’s going to depend on the individuals that are within your organisation. I guess that goes without saying. But the thing that I usually focus on is time to launch and the quality of launches. So if you do post-mortems in your company, and well, even if you don’t. You certainly know the latest disasters that have happened in production. Just pointing to them, pointing to your last stressful launch and just say that this can help avoid all of that, right. It’s going to make your process more efficient and just focus on the fact that you can have more efficient launches as a result of this. And pointing to big companies like Google, like Microsoft, like other large companies as well that already follow these practices and seeing the results from them. And just pointing them as example to say ‘hey, these guys are seeing results in this way, may be we can start following them as well.’ One thing that’s really good to illustrate, are the feedback loops. And having a separate manual testing role and separate development role, and how much time that actually takes in order to get a bug fixed. As opposed to getting that feedback loop shortened, and earlier in the cycle, perhaps even getting it automated, and just show the difference there in terms of project time.

Ways of Incorporating Testing in Development Teams

Derrick:
What are some of the ways one can start to move towards this testing change in your organization? Could they just switch or is there more to it?

Trish:
I think it is more of a cultural change than it is a process change and that’s what makes it very hard. A wise person said to me that she actually thinks that you can only really achieve this change once you have parity in your team between developers who fully embrace and want this to be the way forward and developers who aren’t there yet. So once you’ve got a 50/50 and you can tip the balance slightly over, then you can start to see change with a lot of pairing and a lot of advocacy from the side that wants this to happen. That was one person’s opinion, and another person just said that you just have to start doing it. It’s important for management to be very transparent – why is this happening, what the benefits are, and just say just try it for one project. This was done at Microsoft in that way, and there wasn’t a lot of panicking, there was some that were a little skeptical, but no-one was freaking out and when they saw the benefits, then they saw that this is a good idea.

Recommended Testing Resources for Developers

Derrick:
Are there any additional resources you might recommend to developers, who see this and want to do it. How would they start to learn more about testing?

Trish:
About testing, I can recommend a couple of books actually for starters. There is one called ‘Driven by Tests,’ which is very much focussed on the unit testing side of things, but it’s a very good book, I’ve had that recommend to me. Then on the exploratory testing side of things, there’s a book called ‘Explore It’ by Elisabeth Hendrickson, which is aimed as an exploratory testing guide specifically written for developers. So I heartily recommend that book as well.

Derrick:
Trish Khoo thank you very much for joining us today.

Trish:
Great, thanks very much for your time.