Functional testing during or after development?

Question

I'm a newly hired junior QA engineer in a company who never had QA before.

I'm writing cucumber + selenium test for multiple projects and one of them is still in active development and is evolving really fast (a team of 5 developers is working on it full time).

While everything was fine at first, I now find myself on the position of having to change my tests again and again because they added stuff (added a page between two other ones), changed stuff (now you can't do that anymore), etc...

So, I'm wondering, am I doing something wrong here? Should I focus on the stable project and only test this one when the big part of development is over?

Doing these tests again and again seems like a waste of time. Especially when I found bugs that are "not bugs but just not developed yet". But maybe there is some benefit from doing tests during the development instead of when it's finished?

Answer 1

What you are describing is very common. There are things developers can do to partially insulate your tests from UI changes. For example, giving every element a unique ID can help. Of course, nothing will completely insulate your tests from UI changes.

I recommend talking to your manager about this. If you are the first QA engineer to the company, they may not have a clue about how to use a QA person. You may have to educate them about what kind of process works and what doesn't.

It may make more sense for you to focus on other kinds of automation. For example, you may want to focus on automating how your QA environment is configured, or how new releases are deployed into your QA environment. You might want to write scripts for populating test data in your database. All of those things would allow you to go faster. They may also be less susceptible to churn than writing UI tests.

It may also make sense to test the UI by hand until the UI stops changing so rapidly.

For what it's worth, in some companies, developers write their own UI tests. Throwing code over the wall to QA is an outdated methodology that much of the development community has left behind.

Answer 2

This depends on many aspects which are usually specific for a particular project/company. One one hand waiting for a stable feature makes you always running far behind development. Developing automated tests along with feature development will lead to constant rework. This is a normal case when you have no well-defined processes which guarantee some commitments on feature specification and release scope..

In your situation I would recommend to change architecture of your tests each time you encounter a piece of rework. This will let to minimize that kind of rework when you face it the next time. This will also develop your hard skills in test architecture.

Answer 3

Since you said you are using Selenium, I am assuming that you are doing UI automation.

UI Automation is very expensive and requires very high maintenance. So UI only automation is not recommended for unstable or highly evolving front end applications.

What you can do is, is to find out the UI functional flows which are more stable and less likely to get updated soon and automate those. Else better stick to doing manual testing for those tests.

You can also consider doing API level testing using Postman/RestAssured, if your application interacts using API services. API automation is much more easier to do and also less maintenance required.

Answer 4

I think i read a couple of anti-patterns:

• Developers not maintaining, owning, running and fixing tests as their code changes. It is great if QA engineers helps writing automated tests during development, but you cannot own them as they are tightly coupled to the code.
• Silos. Developers not writing automated functional tests. Pair test/program more with them.
• Too many automated tests on a functional level. I think one or two tests are enough per behaviour on a functional level, but most QA engineers try to cover all possible permutations on the wrong level. Less tests equals less maintenance.