Good day, beautiful and smart people!

We've been prototyping an application and it is more or less at the MVP stage and the customer, seeing the minimum desired functionality, is taking things seriously and wants further development.

At this point in time, we have a part-time tester with experience in test automation who is eager to apply his skills into practice and try a test automation framework he's been wanting to try for a long time.

At the same time, this is a very early time to invest in test automation as the application changes very frequently both on the backend and frontend sides.

For the moment, we are asking the QA engineer to go through customer business requirements and to validate the existing functionality against them as well as familiarize himself with the roadmap.

What other things could we ask him to do? What would be the best use of QA engineer's time while the project is in an early stage?

  • 1
    When you say "try a test automation framework he's been wanting to try for a long time" does it mean he never used this framework before? It could be a good idea to get familiarized with it early.
    – the_lotus
    Commented Dec 13, 2018 at 18:56
  • @the_lotus right, he went through tutorials and is familiar with the general concepts, but yeah, I can see that as one possible way to go. Thanks.
    – alecxe
    Commented Dec 13, 2018 at 18:57
  • Give them (inflatable, if legal insists) bats to beat proper coding practices into the developers. The earlier you realize your developers are being naughty the easier to correct it. Commented Dec 13, 2018 at 21:18
  • @TemporalWolf that is hilarious, I have a privilege to experience and understand that from both the sides :D
    – alecxe
    Commented Dec 13, 2018 at 21:21
  • 1
    I like to be involved in the design talks/process as well. That's where I start my testing. Even if it is only in theory. This person could sit down with the devs and draw some mockups of what the UI will look like. Come to some agreement on the elements that will be used and the locators for those elements. With that information, a QA Engineer can start building out page objects before the UI is even completed. Obviously it will need to be tweaked a bit once the feature is complete but this could give them a good jump start in testing and likely a quicker turn around time if a bug is found. Commented Mar 1, 2019 at 16:01

10 Answers 10


If the product is at the MVP stage and your QA is just getting started, there's a problem. As a BA I saw this time and time again with dev vs QA where the dev team only got the BA to do testing when they felt the product was "ready to deliver to QA" which is a notion all developers should aim to dispel.

Most developers went to university and took some programming classes. When they submit assignments, they are graded on how well they meet certain criteria, most of which is whether they pass the professors tests. This is an incredibly damaging practice because it teaches developers to be afraid of publicly failing a test.

Instead, QA should be testing long before these features are "ready." If you have a 10 working day sprint cycle, the things you work on on day 1 should be sent to QA on day 2. Don't send it to QA on day 6 with all your other things, have them give feedback on 6 different features, and have you spend day 7 and 8 fixing it all for a final test on day 9 and a release on day 10.

So if you're at the MVP stage, your QA person should already be intimately familiar with the product because they should have been doing testing on it every day since you could log in to the app. If you haven't, now is the time to get them started on it full time. If the person has tests that can be automated, do so. Don't be afraid to test something because it might change later. In fact, that's a good thing. A lot of times, if the test is structured well, it will still pass even after the change. And if it doesn't, that's fine. You are unlikely to scrap the whole test because of some changes. And if you do, well you need tests around the parts of your app the change the most because that's where the highest likelihood of bugs is going to be.

Long story short, this engineer should be having a healthy mix of manual and automated testing on a full time basis (especially if you have 3 or more developers working full time on the product.) They should focus on the things that are changing right now, as that is most likely to have bugs in it. They should be testing it before it is complete so early feedback can be sent.

  • 8
    "We think this will change later"...often doesn't change at all. Then you are stuck running trying to catch up on tests. And other people are making changes to surrounding things, so when you find something not working you don't know if it ever worked originally or if it was just broken as a side effect. Commented Dec 13, 2018 at 20:38

Are you aware that “70 to 85% of bugs in software are due to incomplete or inaccurate functional requirements?”

"There's no sense in being precise when you don't even know what you're talking about." -John von Neumann

What would be the best use of QA engineer's time while the project is in an early stage?

Test the requirements itself, not the product...

by asking the right questions to uncover & challenge implicit assumptions in the requirements.It will ensure that the requirements

are complete & consistent and are according to the customer’s needs.This could be the biggest payoff at this stage.

I also think what's precisely distinguishes a QA engineer from a test engineer is that QA engineer prevent bugs in the first place whereas a test engineer just merely finds them after the fact.

I have seen time after time , in projects even in later stages, fundamental assumptions are either not fully correct/ or at least not understood/challenged as a team.

Many who join the project late , either do not bother or have the confidence/courage to analyze & challenge the fundamental assumptions in requirements systematically.

To make testing requirements more effective, you can use the approach called heuristics testing, or testing with a strategy that relies on past data about probabilities. This targeted type of testing often allows for more intelligent investigation of where any bugs or problems may occur, even in requirements testing.

This strategy helps determine what types of errors are likely and how common errors occur in certain parts of the code. It also helps to check requirements against an accumulated base of problems. Be sure to cover all these areas in your app:

Structure (what the product is): Is it one program or many? What physical parts come with it? Can I test it module by module?

Function (what the product does): What are its functions? What kind of error handling does it do? What kind of user interface does it have? Does it do anything that is not visible to the user? How does it interface with the operating system?

Data (what it processes): What kinds of input does it process? What does its output look like? What kinds of modes or states can it be in? Does it come packaged with preset data? Is any of its input sensitive to timing or sequencing?

Platform (what it depends on): What operating systems does it run on? Does the environment have to be configured in any special way? Does it depend on third-party components?

Operations (how it will be used): Who will use it? Where and how will they use it? What will they use it for? Are there certain things that users are more likely to do? Is there user data that would make the tests more realistic?

You can invent your own heuristics and apply them to the whole application as well as to requirements analysis.

Good requirements should be clear and precise, with no uncertainty or ambiguity; should be measurable in terms of specific values; should be testable and complete; and shouldn’t contain any contradictions.

Further Reading: https://www.softwaretestinghelp.com/how-to-test-software-requirements-specification-srs/

  • 1
    Yeah, good thought about the API layer as logically this should generally be a more stable part of the stack as it, especially in the REST design, mimics the underlying data entities. Thanks!
    – alecxe
    Commented Dec 13, 2018 at 3:00

Although all observations are valid, from a project point of view, I am missing the motivational and learning point of view. Starting some automation now can also be part of the learning of that engineer, and the team. He will be motivated as he can do something he wants to do. Yes, the product will change. But he can also grow along with the development of the product. Starting the automation in the "future" might even be too late to catch up at that stage. Or very difficult because no one thought about the needs of the testing ("been there, done that"). Complexity grows so will the automation challenge. So from my perspective: start early. Together and alongside the development of the product. Learn and adapt. Refactoring is a given. For all types of coders.

Having said that: the quality assistance member can already help with reviewing unit tests. Help with tools that assist the team. Build mocks. Apart from what already is mentioned.

And returning to the person view: a coding tester is not very different from that other type of coder: they run if you burry them in paperwork... just my 2cnt.

  • I really like your thinking here. Start early and learn the lessons. Thank you for the "powerful" answer!
    – alecxe
    Commented Dec 13, 2018 at 18:02

I agree that it may be too early for much automation. But there is still a lot QA can bring to the project to help it be successful at this stage.

If it were me on the project, along with learning everything I could about the requirements, I would be looking at what the customer(s) really do & what their workflow is, what our competitors might offer and anything we've developed previously that might be of use (good or bad) on this project.

I would try to get invited to any/all meetings on design & development so I could understand not only the product requirements, but how the coders are developing & writing the product. Take note where there are heated discussions, confusion or areas of concern - those are areas you might need to give extra attention later.

Be curious & ask questions. Ask about tests you might run, or the labels for fields, or what the customer expects response time to be - ask about everything. This confirms everyone sees things the same way (easier to catch it now, than later). I've been amazed how many times a 'harmless' confirmation question has evolved into a full 20 minute conversation that fixed a major issue on paper instead of when it came to test.

I work in an agile shop - so I would begin to draft up some epics, and if possible some of the first stories. They may need flushed out later, but start getting the ideas down. And begin looking at your requirements - do you have a specific format required for test plans, regulatory documents you'll need to fill out with testing, etc. If you don't know what those are now would be a good time to gather that information.

If you see something new to you, now might be the time to give yourself a crash course in it. (I once did an introductory course on medical coding based on the project I was on... not needed, but very useful when we got further in.) While things are slow for QA is the time to squeeze that in.

Being your QA person is only half time on the project all this may not be possible, but even tackling a few of these things will be good for the overall project.


It sounds like you're on the right lines already but, in addition to going through the requirements and getting familiarised with the roadmap, it's definitely worth getting him to exploratory test the MVP and get a feel of the product as early as possible.

Also, static testing the requirement document would be beneficial as it allows for early feedback and your guy could shape the backend requirements to make his automation easier to implement -- I've worked in places where the automation team would get to name the page elements, rather than the developers or business analysts.

Failing that, creating some tests for the highest priority journeys (usually the 'happy paths') would showcase what the automation is capable of... and would likely impress the customer!

  • 2
    If they have even a semi-working product exploratory testing is a great idea!
    – CKlein
    Commented Dec 13, 2018 at 19:23

note: I assume you are CTO or similar role or the "boss" for QA stuff

In order to clarify my "idea" of QA during project/product development, I've write down following workflow as example:

  • requirements: formalize them, update them, take track on every release, as use case or user story.
  • define tests: define how user story are tested, with edge cases should be take in consideration
  • tools: find right tool/framework/system to build tests
  • tests: core QA work here, for every release run tests or create new tests to cover most of user stories. Test could be of the following types:
    • unit test: code must be tested on edge cases (null, worst scenario) always, everywhere
    • integration test: api integration test, using for example tool like postman
    • end user test: not automatable tests, should take care of core use cases and could be done by hand (or mouse)
  • document & retrospective: track releases, tests done / accepted / failure to bring insight on next steps where team can do better

To answer your question: "What could a QA engineer do when a project is in an early stage?", for me is:

  • help on requirements phase: understand them, help to track them, and be sure that every requirement should have an acceptance test, write in plain text. Find a way to specify them, track them, organize them and keep them updated in a "test wiki/document" is a huge work. I've found useful define not only user story/use case, but "user journey", to connect use cases together in a meaningful way). All team could benefit from this work and QA engineer can understand better and more quickly what system do and why and organize following phases

  • be in charge of define tests and tools phases: decide what to test, when and where is importand! Also QA engineer should be (with CTO approval, of course!) in charge here. Be critic on tool choice, be free to experiment with other tools is crucial for QA engineer to be accountable here

  • work on tests: core QA work here. Be responsabile and accountable that every user story is covered by a set of tests (unit/integration/end user tests) with a proportion of 70% / 20% / 10% for type of test. Critical path for every release of the software should be tested manually.

  • document & retrospective: don't forget this! Help project/product manager to identify areas of improvment, make proposal and give feedback on problems, so planning features/epics should be more easy. For example: we had a lot of problems in this area, should we invest more on redesign/re-think it ?

Background: developer&QA 10 years, product owner 1 year

  • 1
    This answer doesn't seem to be addressing the question that was asked. Instead it seems to be describing how to "build a proper QA stage" (whatever that means) which might be useful but again isn't addressing the question. Commented Jan 5, 2019 at 7:34
  • Thanks @ChrisKenst for sharing your toughts! You asked for "What other things could we ask him to do? What would be the best use of QA engineer's time while the project is in an early stage?" and I think review the process simply answer your question, in particular on define test/tools points, what do you think ?
    – Vokail
    Commented Jan 7, 2019 at 7:37
  • 1
    To be fair, I didn't ask for anything and am not the person who asked the original question. I was reading through the responses and found yours to be at odds / confusing. Asking the QA engineering to spend his time reviewing processes is fine, if they have them. Asking the QA engineer to define those processes wouldn't make any sense. Perhaps you could change your answer to better reflect this advice? Commented Jan 7, 2019 at 18:45
  • @ChrisKenst thanks for your comment, I've edited my answer, to clarify differente between workflow and actual answer
    – Vokail
    Commented Jan 8, 2019 at 13:30

I think it's a good time to ask questions about how you're defining quality for the app (ie, how quick it needs to be, how accessible it needs to be, supported browsers) I've also found that test automation can take a little time to get up and running. There's lots of drivers to be installed and/or docker/virtual machine type containers to get up and running, plus decisions on tools to use, security features, hooking it up to the build server, etc, and teaching anybody who wants to contribute or review results.

Have that ready to rock, even if it's just a login test, can be very helpful in the effort to 'automate with the sprint' that so many people are trying to achieve these days.


Need to discuss, the concern team members Business Analyst, Developer, Testing member about the project scope. One idea will be generated how to go into for testing.

Also preparing self document ( if u don't have document) . If u have read it under stand it. Try to ask as much questions as possible. Make a note of it. And start testign.

Read more: https://softwaretestingboard.com/q2a/3308/


Lay a foundation for quality

"For the moment, we are asking the QA engineer to go through customer business requirements and to validate the existing functionality against them as well as familiarize himself with the roadmap."

That is good for a few key scenarios. However, beyond that it may lead to a testing setup that looks more like an ice cream cone than an agile testing pyramid with extensive end to end selenium tests that are slow and flaky.

Instead of just focusing on adding lots of slow and flaky automated UI tests, for an early stage company this is an opportunity to lay down the foundation for quality. After all, tests in of themselves will not fix a low quality product.
Especially if they are written by someone else.
Later in the process.

To improve quality and reduce bugs in software, measure and monitor over time:

  • Product usage by current production users
  • Features are used as imagined by the company
  • Measures are available for usage by key demographics and devices
  • Usage in relation to revenue, adoption or other measures or KPIs
  • Test suite length of run time
  • Application code unit test coverage
  • 100% should be the general rule
  • Test for parameters that are zero, missing, blank, null
  • Application code average LOC method/class sizes
  • Application code average method complexity
  • Average amount of time to fix a production bug
  • Mean time for tickets from entry to deployment
  • Mean time between production failures
  • CI UI Automation code failure rate – target N5 (00.001%)
  • Pending tests – target zero
  • Size of backlog
  • Backlog size change over time – target zero
  • Application performance

Also pay attention to softer and more subtle factors that may be harder to measure such as

  • Naming application code objects well
  • Usability issues related to fonts and colors and sizes for all users
  • Domain specific usability issues for systems and ecosystems
  • Maximizing accessibility to help reach the largest market share
  • Maximizing accessibility for users with different physical abilities
  • Emotions related to color schemes
  • UI consistency
  • Verbal and interactive feedback from key users
  • Balancing all these different aspects is why quality is hard

I think you have made your QA engineer work on the right thing at the moment. As you said the project is going through extensive changes, I would suggest you put your QA engineer to handle tasks related to QA services like:

-Exploring the project requirements and goals

-Writing test policies

-Define the test process based on existing requirements and functionalities.

Not the answer you're looking for? Browse other questions tagged or ask your own question.