Getting Started - Newbie Question

Question

We are an IT company which provides Automated Vehicle Location (GPS Based) services. We provide dozens of web apps in our ecosystem, and none has ever been thoroughly passed an SQA process. Our development team is just 7 people and, considering the amount of feature request, we cannot afford to release bugs into production environment, so we'd like to automate testing as far as possible.

Could anyone please recommend us some starting points on where to look for possible tools to use (free / opensource preferably), tutorials or alike?

Answer 1

There are alot of things to consider. First here are some tool links as there are tons.

Mobile:

http://www.testingtools.com/mobile-testing/

Various automation companies:

http://www.capterra.com/automated-testing-software/

Some brief info on Approaching Automation:

http://www.softwaretestinghelp.com/automation-testing-tutorial-7/

http://www.nalashaa.com/7-key-best-practices-of-software-test-automation/

Automation Tools:

http://www.testingtools.com/test-automation/

http://docs.seleniumhq.org/

http://www.sikuli.org/

Ok, now that you are thoroughly overwhelmed with data here is the most important things to keep in mind. What is the best value for testing. By "value" I don't mean cost, but importance. If you already have a large code base there should be a few critical components to start out.

1. Where is the biggest risk if not tested?
2. What has the biggest client impact if broken?
3. What code is utilized the most (i.e. if this breaks there will be lots of places that break instead of just one)?
4. What is not covered currently by any other verification method?
5. What is repetitive and easiest to script and rerun?

Some of these are manual testing questions and some are automation questions, but your goal isn't "what do I automate" it's "how do I ensure the best quality". Once you can determine that in detail it's time to separate what to automate vs. what to manually test vs. what to take a risk on. Based on what you have said in your description it seems you have a group of developers, if you don't have QA folks I recommend getting one. There is a reason it's a separate job and it is because it's a different way of thinking and approaching things. If developers were able to do all the QA there wouldn't be a need for the job itself. It's important to have someone with the right way of thinking and perspective to bring that into the team dynamics. Maybe you already have this person in which case great (maybe that is you...).

1. Once you have a solid team the answers to the questions will become evident. Risk could be customer impacting, but it could also be something like divergent code bases that are no longer maintainable vs. a centralized approach. It could also be inline styles vs. css or copied code instead of reused code.

2. This should be your client perspective. If it seems really bad but noone cares, don't put it here as it could offset the whole project. The ultimate goal should be customer satisfaction even if there are some things that are "need to do later".

3. This will likely overlap 1&2 but the goal is anything that overlaps more than 1 of these is more critical by nature than something that doesn't. Dead code for instance might just be in here, but it might also confuse and make the footprint confusing which causes more breakages. It's important, but not as important as copied functions that are actively utilized but not maintained, centralized coding becomes a 2 part making it more important even though both are along the lines of confusion and duplication.

4. This will start out huge and hopefully grow smaller over time. The starting point on whittling down this will relate to the other categories. If something seems critical to 3 places than naturally start there first and work towards the lesser impacted areas.

5. Sometimes the quickest way to improve quality is getting the most coverage first. Even if it's less critical if it covers 25% of the application in 1 swipe it might be worth automating it first.

I recommend doing a weighted point system based on categories to get a proper analysis on the overall scope of the application. After that it's deciding which tools and where to utilize them. This should be part of the SDLC though and not treated as an after thought. If you are going to do this right, you need to change the process going forward to include QA as part of the process prior to release.

Answer 2

Selenium is a standard for browser automation, and is free.

But biggest challenge will be to change the culture in your company. You will spend huge amount of time and resources backfilling automated tests to apps which were not designed to be automated. In many cases, changes would be required to make automated tests easier/more reliable: is company willing to spend resources on that?

And if no changes are allowed, you will have to rely on flaky locators like XPath, which will decrease your productivity significantly - see all these folks here struggling with XPath. I give only one advice about XPath is - avoid it if you can. :-) I do.

You will need a champion on the business side to spend all those resources on something which does not add new features for customers, and marketing does not care. Without such champion, you are entering a battle you cannot win.

I recommend to research Workplace SE for questions about workplace politics and process changes. That would be your bigger challenge than coding, IMHO.

Answer 3

The problem with Legacy applications is that most of the time they where not developed with test automation in mind. Adding automation as an after thought is a challenging task.

There is not a lot of information about this, but there is this well received book Working Effectively with Legacy Code.

Change the process. Stop adding more features that are not well tested. Add enough tests in each level of the test pyramid for any new feature. Slowly refactoring existing code to become better testable.

Just blindly using test automation tools will probably not solve the testing problems. It might even add new issues, because automated tests potential become a maintenance nightmare.