I am tasked with creating regression suite from scratch for this huge, confusing mess of a software.
General idea is like this:
- User presses "Create this thing", first step of thing creation is displayed, user fills many boxes, dates, checks many checkboxes, if everything is OK, user can proceed to step two, adds different information relevant to step two, refines existing from step one, uploads some data, moves to step three, etc.
- In total there are 12 steps, each step depends on previous one in what data to display, what options are available for user etc.
- Its not possible to open step 9, and check how that works. You need to either open existing thing that is left at step 9, or create new and move through all the steps.
- New features are added to existing steps. Like, in step 8, tool is added that lets user do some stuff, affecting some other steps. (main manual testing happens testing these features)
- This workflow is further split by sub-workflows accessible in specific steps (again, depending on general data entered in this thing)
- All these things after creation (starting from step one) are grouped into tons of reports, moved to different systems etc
It's all weird business logic that doesn't really make sense to anyone including users and new features adding further complexity just keep getting thrown on top of it.
My problem and question is this: How do you automate functionality of this?
So far, my approach, using webdriver/c#/nunit has not been very successful, due to following reasons:
[Test] methods are supposed to be simple- test one thing, assert something. I cant assert one thing. Because if that stuff I need to test is in step 5 and only accessible if I did something in step 3, my small simple test needs to "Create new thing" , and move correctly through all steps up to 5. Oh. And then create another and fill different data in step 3. And now my simple test is huge.
[Test]s are supposed to be independent. I cant do tests that test one thing, then pass some data to [Test]2 etc and move thorough steps in that way.
Management wants to automate User stories. "Button move to step 3 takes you to step 3" type. Useless, because you can always move to step 3 if you press the button, what matters is what happens during transition and what happens depends on various data entered in previous steps. Ahhh.
So to at least try something I try to organize tests like this:
- Describe elements in one place according to where they are found. Describe all common used methods in one place. Like PageObect model.
- Create huge "Create thing" function that takes many arguments, like what data to input in which step, at which step to stop and automates creation without really testing anything. And various others that automate some functionality depending on arguments passed but don't test anything themselves.
- When creating [Test]s I call functions that create and move thing through workflow first, with arguments that I need for that specific test. Then I manipulate some data and make assertions for that specific feature that I got to.
- So now my tests are kind of small and independent of each other BUT in reality they do a lot of crap and take a long time. And are horrible to debug.