What could a QA engineer do when a project is in an early stage?

Question

Good day, beautiful and smart people!

We've been prototyping an application and it is more or less at the MVP stage and the customer, seeing the minimum desired functionality, is taking things seriously and wants further development.

At this point in time, we have a part-time tester with experience in test automation who is eager to apply his skills into practice and try a test automation framework he's been wanting to try for a long time.

At the same time, this is a very early time to invest in test automation as the application changes very frequently both on the backend and frontend sides.

For the moment, we are asking the QA engineer to go through customer business requirements and to validate the existing functionality against them as well as familiarize himself with the roadmap.

What other things could we ask him to do? What would be the best use of QA engineer's time while the project is in an early stage?

Answer 1

If the product is at the MVP stage and your QA is just getting started, there's a problem. As a BA I saw this time and time again with dev vs QA where the dev team only got the BA to do testing when they felt the product was "ready to deliver to QA" which is a notion all developers should aim to dispel.

Most developers went to university and took some programming classes. When they submit assignments, they are graded on how well they meet certain criteria, most of which is whether they pass the professors tests. This is an incredibly damaging practice because it teaches developers to be afraid of publicly failing a test.

Instead, QA should be testing long before these features are "ready." If you have a 10 working day sprint cycle, the things you work on on day 1 should be sent to QA on day 2. Don't send it to QA on day 6 with all your other things, have them give feedback on 6 different features, and have you spend day 7 and 8 fixing it all for a final test on day 9 and a release on day 10.

So if you're at the MVP stage, your QA person should already be intimately familiar with the product because they should have been doing testing on it every day since you could log in to the app. If you haven't, now is the time to get them started on it full time. If the person has tests that can be automated, do so. Don't be afraid to test something because it might change later. In fact, that's a good thing. A lot of times, if the test is structured well, it will still pass even after the change. And if it doesn't, that's fine. You are unlikely to scrap the whole test because of some changes. And if you do, well you need tests around the parts of your app the change the most because that's where the highest likelihood of bugs is going to be.

Long story short, this engineer should be having a healthy mix of manual and automated testing on a full time basis (especially if you have 3 or more developers working full time on the product.) They should focus on the things that are changing right now, as that is most likely to have bugs in it. They should be testing it before it is complete so early feedback can be sent.

Answer 2

Are you aware that “70 to 85% of bugs in software are due to incomplete or inaccurate functional requirements?”

"There's no sense in being precise when you don't even know what you're talking about." -John von Neumann

What would be the best use of QA engineer's time while the project is in an early stage?

Test the requirements itself, not the product...

by asking the right questions to uncover & challenge implicit assumptions in the requirements.It will ensure that the requirements

are complete & consistent and are according to the customer’s needs.This could be the biggest payoff at this stage.

I also think what's precisely distinguishes a QA engineer from a test engineer is that QA engineer prevent bugs in the first place whereas a test engineer just merely finds them after the fact.

I have seen time after time , in projects even in later stages, fundamental assumptions are either not fully correct/ or at least not understood/challenged as a team.

Many who join the project late , either do not bother or have the confidence/courage to analyze & challenge the fundamental assumptions in requirements systematically.

To make testing requirements more effective, you can use the approach called heuristics testing, or testing with a strategy that relies on past data about probabilities. This targeted type of testing often allows for more intelligent investigation of where any bugs or problems may occur, even in requirements testing.

This strategy helps determine what types of errors are likely and how common errors occur in certain parts of the code. It also helps to check requirements against an accumulated base of problems. Be sure to cover all these areas in your app:

Structure (what the product is): Is it one program or many? What physical parts come with it? Can I test it module by module?

Function (what the product does): What are its functions? What kind of error handling does it do? What kind of user interface does it have? Does it do anything that is not visible to the user? How does it interface with the operating system?

Data (what it processes): What kinds of input does it process? What does its output look like? What kinds of modes or states can it be in? Does it come packaged with preset data? Is any of its input sensitive to timing or sequencing?

Platform (what it depends on): What operating systems does it run on? Does the environment have to be configured in any special way? Does it depend on third-party components?

Operations (how it will be used): Who will use it? Where and how will they use it? What will they use it for? Are there certain things that users are more likely to do? Is there user data that would make the tests more realistic?

You can invent your own heuristics and apply them to the whole application as well as to requirements analysis.

Good requirements should be clear and precise, with no uncertainty or ambiguity; should be measurable in terms of specific values; should be testable and complete; and shouldn’t contain any contradictions.

Further Reading: https://www.softwaretestinghelp.com/how-to-test-software-requirements-specification-srs/

Answer 3

Although all observations are valid, from a project point of view, I am missing the motivational and learning point of view. Starting some automation now can also be part of the learning of that engineer, and the team. He will be motivated as he can do something he wants to do. Yes, the product will change. But he can also grow along with the development of the product. Starting the automation in the "future" might even be too late to catch up at that stage. Or very difficult because no one thought about the needs of the testing ("been there, done that"). Complexity grows so will the automation challenge. So from my perspective: start early. Together and alongside the development of the product. Learn and adapt. Refactoring is a given. For all types of coders.

Having said that: the quality assistance member can already help with reviewing unit tests. Help with tools that assist the team. Build mocks. Apart from what already is mentioned.

And returning to the person view: a coding tester is not very different from that other type of coder: they run if you burry them in paperwork... just my 2cnt.

Answer 4

I agree that it may be too early for much automation. But there is still a lot QA can bring to the project to help it be successful at this stage.

If it were me on the project, along with learning everything I could about the requirements, I would be looking at what the customer(s) really do & what their workflow is, what our competitors might offer and anything we've developed previously that might be of use (good or bad) on this project.

I would try to get invited to any/all meetings on design & development so I could understand not only the product requirements, but how the coders are developing & writing the product. Take note where there are heated discussions, confusion or areas of concern - those are areas you might need to give extra attention later.

Be curious & ask questions. Ask about tests you might run, or the labels for fields, or what the customer expects response time to be - ask about everything. This confirms everyone sees things the same way (easier to catch it now, than later). I've been amazed how many times a 'harmless' confirmation question has evolved into a full 20 minute conversation that fixed a major issue on paper instead of when it came to test.

I work in an agile shop - so I would begin to draft up some epics, and if possible some of the first stories. They may need flushed out later, but start getting the ideas down. And begin looking at your requirements - do you have a specific format required for test plans, regulatory documents you'll need to fill out with testing, etc. If you don't know what those are now would be a good time to gather that information.

If you see something new to you, now might be the time to give yourself a crash course in it. (I once did an introductory course on medical coding based on the project I was on... not needed, but very useful when we got further in.) While things are slow for QA is the time to squeeze that in.

Being your QA person is only half time on the project all this may not be possible, but even tackling a few of these things will be good for the overall project.

Answer 5

It sounds like you're on the right lines already but, in addition to going through the requirements and getting familiarised with the roadmap, it's definitely worth getting him to exploratory test the MVP and get a feel of the product as early as possible.

Also, static testing the requirement document would be beneficial as it allows for early feedback and your guy could shape the backend requirements to make his automation easier to implement -- I've worked in places where the automation team would get to name the page elements, rather than the developers or business analysts.

Failing that, creating some tests for the highest priority journeys (usually the 'happy paths') would showcase what the automation is capable of... and would likely impress the customer!

Answer 6

note: I assume you are CTO or similar role or the "boss" for QA stuff

In order to clarify my "idea" of QA during project/product development, I've write down following workflow as example:

• requirements: formalize them, update them, take track on every release, as use case or user story.
• define tests: define how user story are tested, with edge cases should be take in consideration
• tools: find right tool/framework/system to build tests
• tests: core QA work here, for every release run tests or create new tests to cover most of user stories. Test could be of the following types:
  • unit test: code must be tested on edge cases (null, worst scenario) always, everywhere
  • integration test: api integration test, using for example tool like postman
  • end user test: not automatable tests, should take care of core use cases and could be done by hand (or mouse)
• document & retrospective: track releases, tests done / accepted / failure to bring insight on next steps where team can do better

To answer your question: "What could a QA engineer do when a project is in an early stage?", for me is:

• help on requirements phase: understand them, help to track them, and be sure that every requirement should have an acceptance test, write in plain text. Find a way to specify them, track them, organize them and keep them updated in a "test wiki/document" is a huge work. I've found useful define not only user story/use case, but "user journey", to connect use cases together in a meaningful way). All team could benefit from this work and QA engineer can understand better and more quickly what system do and why and organize following phases

• be in charge of define tests and tools phases: decide what to test, when and where is importand! Also QA engineer should be (with CTO approval, of course!) in charge here. Be critic on tool choice, be free to experiment with other tools is crucial for QA engineer to be accountable here

• work on tests: core QA work here. Be responsabile and accountable that every user story is covered by a set of tests (unit/integration/end user tests) with a proportion of 70% / 20% / 10% for type of test. Critical path for every release of the software should be tested manually.

• document & retrospective: don't forget this! Help project/product manager to identify areas of improvment, make proposal and give feedback on problems, so planning features/epics should be more easy. For example: we had a lot of problems in this area, should we invest more on redesign/re-think it ?

Background: developer&QA 10 years, product owner 1 year

Answer 7

I think it's a good time to ask questions about how you're defining quality for the app (ie, how quick it needs to be, how accessible it needs to be, supported browsers) I've also found that test automation can take a little time to get up and running. There's lots of drivers to be installed and/or docker/virtual machine type containers to get up and running, plus decisions on tools to use, security features, hooking it up to the build server, etc, and teaching anybody who wants to contribute or review results.

Have that ready to rock, even if it's just a login test, can be very helpful in the effort to 'automate with the sprint' that so many people are trying to achieve these days.

Answer 8

Need to discuss, the concern team members Business Analyst, Developer, Testing member about the project scope. One idea will be generated how to go into for testing.

Also preparing self document ( if u don't have document) . If u have read it under stand it. Try to ask as much questions as possible. Make a note of it. And start testign.

Read more: https://softwaretestingboard.com/q2a/3308/

Answer 9

Lay a foundation for quality

"For the moment, we are asking the QA engineer to go through customer business requirements and to validate the existing functionality against them as well as familiarize himself with the roadmap."

That is good for a few key scenarios. However, beyond that it may lead to a testing setup that looks more like an ice cream cone than an agile testing pyramid with extensive end to end selenium tests that are slow and flaky.

Instead of just focusing on adding lots of slow and flaky automated UI tests, for an early stage company this is an opportunity to lay down the foundation for quality. After all, tests in of themselves will not fix a low quality product.
Especially if they are written by someone else.
Later in the process.

To improve quality and reduce bugs in software, measure and monitor over time:

• Product usage by current production users
• Features are used as imagined by the company
• Measures are available for usage by key demographics and devices
• Usage in relation to revenue, adoption or other measures or KPIs
• Test suite length of run time
• Application code unit test coverage
• 100% should be the general rule
• Test for parameters that are zero, missing, blank, null
• Application code average LOC method/class sizes
• Application code average method complexity
• Average amount of time to fix a production bug
• Mean time for tickets from entry to deployment
• Mean time between production failures
• CI UI Automation code failure rate – target N5 (00.001%)
• Pending tests – target zero
• Size of backlog
• Backlog size change over time – target zero
• Application performance

Also pay attention to softer and more subtle factors that may be harder to measure such as

• Naming application code objects well
• Usability issues related to fonts and colors and sizes for all users
• Domain specific usability issues for systems and ecosystems
• Maximizing accessibility to help reach the largest market share
• Maximizing accessibility for users with different physical abilities
• Emotions related to color schemes
• UI consistency
• Verbal and interactive feedback from key users
• Balancing all these different aspects is why quality is hard

Answer 10

I think you have made your QA engineer work on the right thing at the moment. As you said the project is going through extensive changes, I would suggest you put your QA engineer to handle tasks related to QA services like:

-Exploring the project requirements and goals

-Writing test policies

-Define the test process based on existing requirements and functionalities.