5

As a tester, I've been asked to provide some concrete input regarding unit testing - with regards to the overall product and process quality.

Our current situation is as follows:

  • Organisation has known explosive growth over a few years, there are two-week sprints with a production release a week later (after acceptance phase).
  • There are quite a few unit tests in place, but certainly not (sufficient) for every module or calculation.
  • Unit tests are being written occassionally at best, when the developer feels that its useful and that the test setup effort isn't too time-consuming.
  • Due to the complex nature of the application, said test setup is equally complex and has never been done for certain modules.
  • A new version of the application is being written (mostly front-end and middle layers), so a part of the current code is kind of legacy but will go a long way still.

Here are some of my thoughts so far, but I'm looking for any experiences or ideas that can help me construct a good proposal:

  • Immediately "forcing" developers to unit test to a given degree doesn't feel right, as I believe it should be a cultural mindset. But how to achieve that step by step?
  • When reading some Agile books, it's mostly about TDD but I fear that's too big a step.
  • Code coverage: could be a baseline percentage to strive for, but of course it's not indicative of having useful tests. Although it's easily made visible and an incentive to write tests nonetheless.
  • To decide whether we should invest a lot of time in test setups, could we look at how many defects could've been detected by unit tests (rather than testers)? In other words, would the ROI be positive? (Yes, I know early detection is always cheaper, but what is the cut-off to determine if very complex testing efforts produce equally useful tests in return?)

So my current proposal would be this - to slowly introduce the mindset:

  • If any defects found by a tester can be unit-tested, implement a test when fixing the defect
  • Same for production bugs
  • At each retrospective, go over the resolved stories that don't have unit tests and discuss if they are missed opportunities
  • Set a feasible code coverage standard for new code
  • During planning, have a tester and developer pair up to discuss possible test cases for unit tests.

Thanks

  • You've focused here on unit tests specifically, but is that all that's in scope? There are various other kinds of automated testing you could think about, that might be easier to introduce to a code base that hasn't been designed with testing in mind, that could help identify defects earlier. Do those already exist? Given the devs are already making tradeoffs on the ease of test setup, maybe focusing on improving that would enable them to write more unit tests? – jonrsharpe Apr 6 at 10:16
  • @jonrsharpe Yes, we have a minimal set of API tests in place, and regression UI tests are being worked on. However, I don't want to push API testing too much, because developers can just pass that on to the testers, and they still wouldn't feel a need for testing on their part. And with scalability in mind, I'm keeping the test pyramid in mind as well. – FDM Apr 6 at 11:37
  • Based on your descriptions of separate testing phases and an us-vs.-them mentality between dev and test, I think focusing specifically on unit testing is too narrow. Work on aligning the whole team around a shared goal of delivering quality software, and collaborate to identify the best ways to do that. – jonrsharpe Apr 6 at 12:05
  • Thanks for your input. It's not really an us-vs-them (sorry if I made it sound like that) but rather just a culture of writing code but (often) simply not thinking of unit tests. – FDM Apr 6 at 12:26
  • I find TDD much more simple than writing a bunch of ever more complex code and hoping I don't break it. Been there for years. Done that. The only significant challenge I see in TDD adoption is good technical organizational leadership. Writing tests first isn't hard per se, it's only hard when that's not what the org supports and nurtures. Sometimes I also find, when changing, that "100%" is the way to go. When you scope down to 90%, that's the door to continue previous practices. that is my experience from several companies but ymmv – Michael Durrant May 7 at 12:02
3

I think you got it mostly right, the best way to convince someone is by having data, proof and explanations. Look at it as a long and never ending journey and not a revolution, try to enjoy the way even if you don't yet see your end target.

A few comments though-

Before you implement a test for a bug you've found stop for a minute to think and evaluate. Some unit tests will be too expensive or risky to be added (for example require extensive modifications to legacy code), others are already well tested in integration or system tests and others could be too considered to rare to worth an effort.

Also remember that while unit tests are a great tool, having too many can lead to the same symptoms of having inflated code for example difficult maintenance and lower performance. So again use your best judgement and common sense. Measuring code coverage will not save you from this one, having multiple tests of the same area in the code will be measured as having the same coverage.

could we look at how many defects could've been detected by unit tests (rather than testers)?

Definitely, you can have a one time effort and go over old bugs to help you build an argument, and you can do it for a while for each new bug.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.