23

Suppose you're working with bright programmer, but every time you test the code you find a serious, obvious bugs. Things the programmer could have noticed if they did their own testing before checking in.

Great testers are meant to do great work, and finding obvious bugs is both a waste of QA expertise and a slow way for the team to get things done.

How do you tell a programmer to test their own code before checking in, instead of just "chucking it over the wall"?

6 Answers 6

26

Disclaimer: I am approaching this as a programmer (who is here mostly to learn how to better interact with our testers.)

It's important to remember that the developer has not always been told how to test the code. Although we wish it wasn't true, the developer was given a set of requirements from someone who really didn't care about every aspect of the system. You can't expect every dev to know every part of the system.

So, the dev gets the spec, notes down the requirements (that were provided to him) and makes sure those requirements are met. Unfortunately, there are two or three other requirements not on the spec that were pre-existing, and now broken.

Whose fault is it? Is it the dev for breaking the existing requirements? Or was it the person who generated the spec and only told him what things to test? To be honest, in a lot of companies, developers aren't even told what they need to test.

Now, for me personally as a developer, if you tell me a way I can test a piece of code better, please do. If I can stop the bug from ever seeing you, I want to do it. Otherwise, I have to fix it in three days from now when the change filters its way up to you and then back down to me, I've forgotten everything about it, and it takes me an hour to make one simple fix that should've taken me 30 seconds when I ran that test.

So you can tell a dev they're not testing their code by giving them advice instead of orders. You can say "hey, I've noticed a few patches come up that have XYZ-attribute about them. Typically, I test them with __input and catch a lot of bugs that way." Most developers I know would love to avoid going back and fixing their code after it's gone out, and will be receptive to that.

I promise you if you're confrontational about it, you'll get a very angry developer who will throw a hissy fit. In that case, you'll be saying "You have buggy code" instead of saying "Hey, here's a way to be more efficient." The latter is much more appealing, and productive!

8
  • 6
    +1 for your overall answer and also for the first sentence. As a developer who started his professional career in testing I quite firmly believe that anyone who wants to be a developer should spend at least 6 months testing first. It's important to see what the other side of the fence looks like!
    – Rob
    Commented May 4, 2011 at 10:08
  • @Rob I can certainly appreciate the experience having a tester would bring. I think there are a lot of things that could bring that kind of experience (a diverse language background, different architecture backgrounds, different software-processes) to the table too, but they shouldn't necessarily be required. That's not to belittle the experience you'd get doing it though. I know in the development community, testing is kind of 'looked down on', like if you're a tester, it's because you're a bad developer; this couldn't be further from the truth.
    – corsiKa
    Commented May 4, 2011 at 23:10
  • 4
    Valuable insight. The only thing I wanted to say is that requirements are not a given. We rarely have the luxury of requirements on my team. Commented May 6, 2011 at 13:30
  • 1
    @Tristaan That's actually the same approach my company takes with developers! The analysts make the specs, and they really don't like it when we tell them how it could be improved, as if their 20 years working in manufacturing has enlightened them to database design any more than all my years of eating food makes me qualified to be a chef.
    – corsiKa
    Commented May 26, 2011 at 20:14
  • 1
    I hear ya, @glow. It comes down to everyone recognizing that we're all on the same team, we all have our areas of expertise, but that none of us has ALL the bits to be able to make a good product. We need all the different perspectives in order to make everything come out clean. Commented May 26, 2011 at 20:24
9

In the past I have approached this by firstly defining the "quality bar" as to what is expected and had this agreed by the whole team. This includes setting a defect threshold for the whole team (usually 0 Sev 1's and 10 bugs per developer). When a developer or the team gets above this threshold they must stop adding features and fix their bugs.

With this in place it sets the expectations on an agreed quality target for the application. The developer then knows what he is aiming for.

Then, I use metrics and focused testing to enforce this. If a developer, or area of the application was defect prone, we would focus our testing there, hit the threshold and development would stop until the quality was improved.

Developers hate it first, but they grow to crave working on a high quality code base because they can add new features confidently and quickly.

1
  • 2
    Doesn't this create horrible arguments about "this isn't a sev 1" "yes it is" "no it isn't", especially when you get near the tenth bug? It doesn't seem like an environment I'd like to work in as a tester - I hate playing pingpong, where one dev says "no, it's not my code that's at fault, his code is passing the wrong information", and the other dev says "no, it's not my code that's at fault, her code...."
    – testerab
    Commented May 4, 2011 at 21:19
7

If possible, sit down with them, explain the bugs. Explain how you found it and ask what kinds of tests they run before they say that it's ready. Start giving some suggestions. I've always found that starting the code quality discussion with some polite, constructive feedback has yielded the greatest results, letting them decide what what they think is sufficient. It's not that they don't want to deliver quality code, sometimes they just don't think of what to look for.

6

My recommendation is to write them up in the bug tracking system that you use. If you don't have one, you need to get one started. If the bugs are that obvious, then their project manager should have words with them. The stats from the tracking software should be able to support your argument that they are careless, or need better understanding of the business problems that the software is intended to solve.

How obvious are the "obvious" errors? My last company sold software in the tax/financial reporting arena. I had only been there for 5 years (and everyone else had been there longer than myself) when I left and many of the "obvious" domain knowledge rules and tests were things that I hadn't come across yet. One sample rule was that a Form 5500 filing may have a Schedule I or a Schedule H attachment, but cannot have both (neither is also acceptable). Everyone knew that one, and the only place it was "written" down was deep in the bowels of regulations from the IRS and PBGC; after I got called out (and embarassed) in meetings, I eventually learned it too.

At a previous employer, the closest to a bug report was shouting down the corridor "check your work better before checking it in!" which still didn't answer what was wrong. But then that company suffered from a lack of source control, so the last person to edit a file was responsible for everything that file had ever done wrong.

My point, and I do have one, is that what is obvious to you is probably not that obvious to someone else.

5

Sometimes I talk with them personally and ask to walk through certain issues, sometimes under the pretext of I maybe missed something and want their input. It's a diplomatic way of trying to get their side of things, and maybe see if you can pull out from them why they did not see it, or plant the seed for having them do it. Diplomacy is an important part of our work and the relations you need to have with other teams.

It might also be useful to find out why they do it this way, maybe they think they are checking completely but don't think about the area you are looking at; or maybe just looking at it wrong. Often, I have found developers code an application and test it the one way they expect it to work, while QA often tests it an entirely different way.

5

First, make sure you can back up your concern with some objective data, e.g. bug counts from your bug tracking system.

How you tell them depends on your relationship with the developer. If you trust and respect each other, you can try showing them your objective data. They may not be aware that they're delivering a lot of bugs.

If you don't have a strong rapport with the developer, it may be more productive to talk to your QA manager (if you have one), the development lead, or the dev manager.

Keep in mind that there may be many reasons for the developer's higher bug count; e.g.

  • developer is pressured by management to go too fast
  • developer is unfamiliar with the technology their using
  • developer has a lot of distractions (e.g. involvement in sales calls or customer support issues)
  • developer doesn't understand the requirements
  • developer doesn't believe it's important for this particular deliverable to be high-quality
  • tester is especially good at finding bugs
  • tester is more picky about bugs than other testers

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.