Industrial standards for SQA framework for a non-technical QA team

Question

I am seeking a software/platform that i can let my team use for QA in software development.

So basically now we have a dev team working on new features and maintenence of our platform(a web app, but soon will also be a phone app). And we want to instituitionalised our QA process, whereby the stages will be: Development - QA - UAT - Live. Then we have a QA team, who is going to be people from non-technical background that will work on QA at the QA stage, before the new/updated product goes into UAT stage. Since they are not really technical people, I would need the user interface to be as friendly and easy to use as possible.

Features that we need will be one column for "What to test for", "Test results", "the artefact"(basically the evidence like screenshot etc). Another problem we might face is that for every update sprints, the updates will be different, (eg. this sprint we might be updating a input free text field, next sprint we might be doing a "who's online" feature.) So I am not very sure how to manage that. I will also need to perform load testing and hope load testing and all the other processes that can be automated will be automated.

If it is possible I might wanna add that I hope it is free.

Are there any automated testing that doesnt require scripting?(actually I half-expected there will be none. But dont dismiss my post yet! My post is more on the SQA framework for manual testing)

What testing goes before the other?(eg is it regression testing before load testing?) And what do dev teams use for load testing/regression testing/sanity test(most likely manual testing right?)?

Any experienced person can suggest how i can go about setting up this QA SOP, and which softwares I can use for automation/bugs reporting etc?

Answer 1

Manual

If you main focus is manual testing then I would focus more on using a good test tracking tool like Jira, Trello, VSTS(MS) or Pivotal Tracker. Most of these have free trials of some variety. I like Jira. I would focus on that plus a well developed agile process and person to manage the process and avoid the things that will always happen without management such as "backlog from hell", etc.

In order to address 'change A this week' and 'change B that week' you need to develop processes that manage that such as agile development and tie them into the tool you use which will usually involve a GUI that has a board with tickets on it for features/changes/bug fixes that can also handle 'sprints'.

btw - free products - should be avoided when you are going to use them as part of your business. They will either break and not be fixed out of the blue, or they will break due to things they interface with changing, or they will suddenly change into paid products, be discontinued, etc. the list of things that can go wrong due to choosing a free product is long. Nothing of lasting value is really free right?

There are not really 'industrial' standards in this area for manual testing that I am aware of. There are certainly testing certification and the like but they have little respect in the community as a reflection of the ability to add value. Software engineering and quality within it is still at an infancy.

Automation

Non-technical people usually cannot maintain a complex automation suite*.
* A suite that runs quickly, is easily maintainable and extendable and adds maximum value

The fact that nontechnical people can't usually use a GUI to build complex application products is also true for automation.

Examples of GUI tools in both areas are Visual Basic and Selenium IDE. These two UI tool examples show how limited our current generation of UI tools are.

I refer to the problem often as 'the thousand test problem' and it is a simple issue that I often pose to vendors that come before me. I recently saw another one of those vendor UI automation 'this will be amazing for your company' product demos. After a half hour demo I asked one question: This looks like a great product for doing 1 test. How would this product address maintainability and extensibility and performance issues when there are a thousand tests. The vendor said that their product didn't address that. Demo over. Thank You.

The problem isn't with the vendors or the tools (I am actually a fan of UI tools for certain situations), the problem is deeper than that. As time passes and products continue to grow and be maintained, test suites grow in size and take longer to run. Also you will usually start to experience intermittent errors, especially in browser based tests due to the nature of using an asynchronous program that is not under your control. Also the effort to update and change the tests becomes greater and greater and often involves a lot of tribal knowledge by one or a few key people. Due to all these issues the value of the tests starts going done. Eventually it actually becomes negative. As your test suites are so slow to run they slow down product development and your competitors eat your lunch.

The solution is to have test automation that is DRY, uses SRP and other programming principles to maintain it.

The other issue that you will run into is that most non-technical folks will use the UI to test every data combination and business case. These tests may be from a few seconds to a few minutes each. Technical programming folks however will (should) break down testing into backend user testing, integrated testing, UI testings and usability/performance testing. Backend tests run in less than 1 second each so when you reach the point of having 1000 tests, UI tests may take hours or days and backend 'unit' tests might take a few minutes.

One option is http://www8.hp.com/us/en/software-solutions/quality-center-quality-management/index.html as it seems to cover the features you want. As hou can see it is complex and not free.

btw I also see the following statements as being indicators of needing more technical expertise to address them:

• "people from non-technical background that will work on QA at the QA stage, before the new/updated product goes into UAT stage." Not wise.
• "So I am not very sure how to manage that". Right, you need technical expertise

See also

• http://www.softwaretestingstandard.org/.

• https://www.softwaretestinghelp.com/what-is-actual-testing-process-in-practical-or-company-environment/

Answer 2

No

Michael has answered your question, as best as is possible, but I'm going to advise, very strongly, against doing what you're proposing. You're not going to be able to have automated testing, as no one is going to be able to set up, let alone maintain, the automation. So, you're going to have no way to do regression testing in any meaningful way unless you're planning on hiring new manual testers every sprint, because the ones that already exist are going to be busy doing all the non-regression testing. You're also not going to be able to do load/stress testing, unless, again, you plan on hiring thousands of people to create the load.

You're also going to have a serious problem with defect management, as the non-technical testers are going to open a huge number of defects, most of which will not be defects, and do a very poor job of explaining how they produced the defect.

So, you have a choice: you can hire a couple of skilled people who can do things like build a test suite, maintain said test suite, build an automated test environment, enhance the testing as necessary, and so on, or you can manage an ever-growing group of non-technical people who won't be able do to anywhere near as good a job as the couple of skilled people.

Testing is a skilled discipline. Would you go and ask how to create an application without any developers? Or how to build a car without any process engineers? In theory, if you threw enough unskilled people at any of those problems, they might come up with something eventually, but it's going to be much less painful, and much more likely to succeed, if you hire people with the right skills. It would be far, far better to make one of your developers a tester than to try and rely on unskilled labor to do the testing.

Answer 3

As other people have already commented, your approach is probably going to fail. Manual testing isn't good for regression testing. Manual testing is good for testing that new features work but not that old features still work. Unless your manual testers run through every feature ever created for your application on every release, you are going to miss bugs and have many hot-fixes.

You really need a combination of both manual and automated testing in order to provide confidence in your application.

If you want to do what you proposed, you can outsource your manual testing to a service like Applause and hire 1 or 2 automated testers rather than a team of manual testers.

Alternatively, you can have the developers write more unit and integration testing and not have a QA department. Several large tech companies do this like Facebook.