8

A team is developing software and testing it using a dummy back end. That team is not allowed to have an access to the real system for testing (politics), so my team is testing it with the real back end. Our team has nothing do with the team who developed the software.

We get the User Story which was implemented on last sprint and we get the software. We are external testers and not part of the Agile Team. Then we automate the testcases. Now the product owner wants us to automate the test case with Agile methods.

According to the Product owner, the specification team will gives us test cases (the product owner has no idea who writes the test cases) in the form of a User Story and we have to say how much story points for that user story. Has anyone worked like this before?

Let's say our sprint ended today and we automated the regression part, but tomorrow we got the new version of software. Our test cases may not work in new version if there were UI or any other changes. That means we have to update our testcases in coming sprint and wait till our next sprint ends and product owner reviews it and accepts it before we can run the test. It would already be late for the result.

How can we make this situation work? Any tips for someone who has experience on automating test cases with agile methodology but is not the part of the agile team who developed the software would be appreciated.

  • 9
    If that other team allows situation you describe, they are NOT agile, regardless how many times they repeat that incantation. Being "agile" is not a magic incantation, it means to follow the agile development principles, and your setup is far from that. I do agree with the answers, which are trying to make the best from the bad situation you are in. My feeling is that your team is being set up as convenient scapegoat for the future failure, so be prepared to abandon the ship when the inevitable disaster strikes. – Peter M. Nov 30 '17 at 14:58
  • 3
    So, so many points to the above comment. This is not agile, and this is very not good. This isn't even very good waterfall. – Kevin McKenzie Nov 30 '17 at 21:54
  • I agree. Not "siloing" is more important than any agile practices like user stories, or story points. "Individuals and interactions over processes and tools". Bring your testers into the agile team, and not having them external is how you have an agile team. One agile team, not two working groups using agile practices. – Lyndon White Dec 1 '17 at 4:29
10

There is no single "correct" answer here, but there are several things your team can do to deal with this situation. I'm going to assume that you have no problems with the estimation and sprint cycle aspect, and your main concern is that you don't want to be blocked when changes to the software you're testing break your existing tests.

  • Leave a maintenance buffer - This is probably the simplest way to handle things. Depending on how much time you think you're going to need for regression issues, you leave a buffer in your sprints for maintenance to update your automation and test cases to handle changes. If you don't know how much to leave, I'd suggest starting with a buffer of about 1/4 of your team capacity and adjusting up or down as you need.
  • Always have a maintenance user story - If you are required to document all your activity, the next step is to have a maintenance user story that you size to about 1/4 of the team capacity to cover any regression issues you need to account for.
  • Block the new items - This would be my preferred choice. If changes impact your existing automation, block any new items until the changes (which are technically regression issues) have been accounted for. If you have to justify your action, point out that when you have failing test cases, they could be masking something more serious so fixing these is a higher priority than adding new tests.
    • Make new items dependent on fixing failed tests - If possible, you should be treating the new tests as being dependent on the failing tests being fixed (and any errors they find being fixed by the development team).
    • Adjust sprint items - If the fixes/problems turn out to overflow the sprint, move new tests out so fixes can be made.

I'd recommend having a discussion with your lead or manager about which approach would work best with your team and the project before you decide which method you use. The key aspect of an agile process is that you can adapt to changes and adjust as necessary, including within a sprint.

One thing I would suggest is that you run your existing test automation against the new version of the software before you plan your sprint - that way you'll know if you need to include work to adapt to changes.

Above all, remember that the most important thing you are working on is automated regression of the software you're testing. Everything else can and should be secondary to keeping your regression test suite in good shape.

  • 2
    +1 Nice answer, but shouldn't "keeping your regression test suite in good shape" be the most important thing? Leading to: 'Everything should be secondary to keeping your regression test suite in good shape.' Your automated tests will need updating continuously, making sure this is easy and smooth to update is your primary job, second to making good new tests and test coverage. – Niels van Reijmersdal Dec 2 '17 at 16:21
  • @NielsvanReijmersdal makes a good point - maintenance of existing tests takes priority over adding new tests, otherwise you're losing test coverage with each broken test, and that can't be offset by adding test coverage. – Myles Mar 13 '18 at 10:08
  • 1
    @Myles - that is the reason for my final sentence. The most important thing is keeping the existing tests in good shape. – Kate Paulk Mar 13 '18 at 11:17
  • @KatePaulk - I just re-read your sentence and realised I missed it the first time. A great answer, as always. – Myles Mar 14 '18 at 12:49
5

The main thing to understand here is that there are no silver bullet. You have a bunch of different issues to address. I recommend you schedule a meeting and discuss the following points.

  • Our team has nothing do with the team who developed the software. Well that's a problem. Please explain a lot more about why not. why can't that change?

  • "We get the User Story". That's command and control language. Why are you not contributing to the story up front. a 10 minute conversation with a tester can save a week of development and similarly a 10 minute conversation with the development team can save a weeks worth of testing the wrong thing.

  • "the product owner has no idea who writes the test cases". Seriously? and you want advice? c'mon this you already know this is ridiculous. Please be braver in your conversations with them.

  • "Our test cases may not work in new version if there were UI or any other changes. " What? you are not in the loop when things change. Is this dysfunctional? well yeah

  • Who runs the tests? Get the developers the ability to run your tests and help maintain them when they break due to development changes. If the developer has to run passing tests themselves they can't pass on broken code.

It doesn't matter to me if another team writes the software part, I just need to focus on:

  • being part of the team that talks through the approach and high level intent up front and how to test the proposed solution BEFORE a single line of code is written.

  • scheduling meetings with the business and talking about how much you enjoy automating - but not being involved 'up-front' is a major issue they need to address to be effective... and keep employing you

  • Using agile practices in my testing so that even if the app state is fixed, I write quality, maintainable automation with peer reviews

Ultimately if the business says "this is just the ways things are, please accept it and write automation" then say that you want to create a quality solution based on good practices in the industry. Otherwise you'll work somewhere that respect you, your team and good principles. Lay it on the line for such a messed up sounding situation is my advice. The language needs some work but I'm not gonna wordsmith it for you. Not easy and will require a LOT of tact.

  • I want to add: the 'requirements' OP gets have not much to do with agile in my book. I agree with @michael: this needs a lot of tact, talk and patience. – Ray Oei Nov 30 '17 at 13:54
  • +1 Changed requirements to approach and high level intent @RayOei – Michael Durrant Dec 1 '17 at 12:43
1

How can we make this situation work?

Probably you can't, you will always lag behind, you will be confronted with impediments your team cannot solve, due to changes in new software releases.

Now the product owner wants us to automate the test case with Agile methods.

Get your team an Agile coach and let this person communicate with the coach of the Development team and figure out a way of working that will be beneficial to both team.

Just blindly copying Agile methods will just lead to Cargo Cult Agile and it sounds your teams are already knee deep in this.

I got some experience with:

automating test cases with agile methodology but not being part of the agile team who developed the software

Become part of the Agile development team! Making sure the team is delivering working and well tested software each iteration. Not adding testing as an after thought.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.