2

Details:

In a discussion about maintainability of automatic test cases in the area Cucumber + Junit / Unitest the topic "hardening of test cases" came up. You know this topic from your discussions about planning, effort, and how to harden test cases so that you don't have to make constant adjustments.

Further explanations:

  • I'm especially concerned that we always have changes in the area of API testing, of course, constant changes are a sticking point here.
  • So it is both a technical possibility to minimize the process of constant adjustments, but also to adapt the workflow in such a way that changes are included.
  • A larger area of the request is probably how I can minimize the effort for expired or changed data, and the changes by the tester? And if I can do this at all?

Questions:

How can I harden mock based api tests so that they don't have to be changed all the time? (Cucumber + Xray based)

How can I harden functional tests so that they do not have to be changed constantly? (Cucumber + Xray based)

How can I adapt the Definition of Ready / Done so that the area Test Case Adaptation and Hardening is planned accordingly?

| improve this question | |
  • 1
    Things need to be changed when application changes , if application is not stable then you have to change your test . Test cases are not not self adjusted. – PDHide Aug 10 at 9:54
  • 1
    Changes of what? You need to change the tests because the app changes? Or you need to change test data? Or both? What's the cause of the changes? – pavelsaman Aug 10 at 10:01
  • @pavelsaman General changes, yes, test data among other things change due to new developments. The cause of the changes would be, for example, a new development of certain areas. Changes in the GUI, changes to test data in fields – Mornon Aug 10 at 10:40
  • @PDHide The point is not that something has to be changed in the test cases from time to time. It's about hardening test cases in such a way that not every change means an adjustment. – Mornon Aug 10 at 10:43
  • @Mornon: Changes often require other changes, especially if the app changes. I'd try to make the architecture of the tests as clear as possible, so the changes could be implemented fast, but that's probably as much as one can do. – pavelsaman Aug 10 at 10:43
1
+100

Test automation code is code, it should follow the same princibles as maintaining regular code. One princible that comes to mind is:

The Single Responsibility Principle (SRP) states that there should never be more than one reason for a class to change.

When you need to change the test-code, challenge yourself does it follow the SRP. Is it a good reason to change? Restructure your code to prevent similar changes in the future. Now for each change make the code a bit better following the boy scout rule, e.g. leave the code cleaner than you found it.

Each time you change the code refactor it so it needs less change in the future. You will find an architecture that fits your context. Refactoring is not something you plan, but this is a constant process that you should practise continuously.

Sarah Mei has a rule to inline-everything in her talk about livable code. Meaning that you constantly make small improvements (and refactorings) as you work on a feature, also to your testsuite. This instead of separate planned hardening tasks. This might slow you down for a while. I do not think it should be part of the DoD. It is a how thing, not a criteria that defines if the product is done for the end-user. Just agree as a team that you spend time each feature to make code better, so that you can go faster in the future.

Good Agile teams go faster over time, not slower, because they lower the cost of change constantly. Read the works of Kent Beck. Recently he published the Programmer Test Princibles:

Summary — programmer tests should:

  • Minimize programmer waiting.
  • Run reliably.
  • Predict deployability.
  • Respond to behavior changes. Not respond to structure changes.
  • Be cheap to write. Be cheap to read.
  • Be cheap to change.

https://medium.com/@kentbeck_7670/programmer-test-principles-d01c064d7934

Developers should run these tests as they change code. They should keep the tests green. This is part of the development cycle. If the input and outputs of features change then the tests also need updating. There is no process that will make that go away, but if you need to restructure your test code every change than probably you are not using the SOLID princibles and other fundamental coding practises like using abstractions, e.g. page models for example.

| improve this answer | |
  • Thanks for the extensive information. They have helped me a lot, unfortunately this topic is much too little considered, but extremely important also in different planning scenarios. – Mornon Aug 19 at 6:03
  • But let's think about it further, in order to harden the test, one should also use tools like SonarQube for code checking on an Integration Stage, in order to make the tests more secure. – Mornon Aug 19 at 6:16
  • Another point which I would like to give at least for reconsideration. The language the testers should use should be kept as simple as possible. That's why I'm not a fan of Java based testing and tend to use Python. – Mornon Aug 19 at 6:32
  • 1
    I would not run SonarQube on the test code for security reasons, the testcode does not go to production, it needs a different type of quality. For the language I would keep it the same as the application, because it makes it easier for the developers to help extend and maintain the tests. – Niels van Reijmersdal Aug 19 at 8:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.