Where does Automation creation happen w/in the Development Cycle?

Question

I’m a Software QA Engineer at a small web development company and I need help wrapping my brain around implementing a new process.

As of recently, my shop only handled ‘one-offs’. We would work on projects where the development schedule was measured in weeks and after deployment we would rarely need to touch the project. Since our development cycle is so short there didn’t seem to be a need for functional automation. But, just recently we have been asked by a client to take over maintenance on their site we are about to create for them. This will involve the full Development, Testing, Support, Implementing 3rd party plugs (Which will also need to be tested). In order for this to work for me and to not get overwhelmed several months down the road, I want to start implementing automation.

Since my language of choice is python, I’ve installed the selenium module and have been playing with Webdriver. What I need help with if figuring out where this all fits into the Development Life Cycle. I feel if I wait until I get the full product it’s too late, or if I start too soon I will just be wasting my time.

So if someone could explain their current project cycle or lead me to a few blogs that explains theirs, that would be awesome.

Thanks for giving me any info you can!

Answer 1

If you can, try & promote the idea of test first development (aka TDD, BDD, ATDD, Specification By Example) with Continuous Integration (frequent commits to a pipeline such as Hudson or GO from Thoughtworks which continuously runs the automated checks to see if any of them have broken after a recent commit)

Before Developers write the code, they write an automated test (check) which fails due to there being no code executed. They then write the code to make the automated check pass. Finally they tidy up the code (refactor).

This will result in the functionality / logic of the code largely being tested (checked) by the Developers as they write the code. As a Tester with my current client, I pair with the Developers to help identify the automated checks I'd like to see before any code is written.

These automated checks are largely at the unit, integration & container (requires webserver) levels as these are the quickest / cheapest to run, but we also have GUI level automated checks.

Prior to committing the code to the pipeline, the Developers demo the code & the Tester & BA to ensure it meets the requirements. The Tester then explores the software & code on the Developers machine to exercise edge cases & see if there are any problems.

If any problems do arise, the Developer writes a failing check to reproduce the problem, fixes it & ensure the check passes.

When the Tester is happy, the Developer commits the code to the pipeline & we then test the code in our integration environment.

Ideally, the only problems we find in integration environment are integration problems!

This frees up a lot of our time help refine requirements with the Customer & BAs to ensure we're building the right thing.

This is a really high overview of how we operate at my current client site. It may not be the best process, but it works for us. Some links that might help:

TDD on Wikipedia
TDD Yahoo Groups
Lisa Crispins blog, with a link to her book co-authored with Janet Gregory, a great resource!
Software Testing Club - another great resource for asking questions &/or finding answers
A blog post of mine with some great resources on how testing is changing Continuous Integration

There are many, many more resources - try Googling "agile testing".

Hope this helps,

Duncs

Answer 2

I'd recommend seriously looking at building a framework that has the absolute minimum of repeated script code - this has the advantage of minimising update work. Similarly, I'd consider data-driven scripts with an object-oriented framwork where you're building your transaction objects to harness the application's features.

That way, as the application grows and adds functions, you can plug in function modules with minimal need to rewrite your earlier code.

A good mapping of web objects to automation objects is essential, too - how complex it needs to be depends on the nature of the web application code, as is working with developers to ensure that the web objects they're generating have sufficient hooks to allow your scripts to access them (This can be as simple as putting an item ID into the web object's attributes)

You probably want to avoid functional and flow automation until each phase of development has stabilized.

In terms of development cycle, I'd honestly categorize automation as its own development cycle with design, building, testing and so forth in its own cycles, and with a dependency on a stable application to test against.

Answer 3

Start early, even before the actual software. This will give you many benefits- 1. You have the luxury of evaluating several tools or automation designs 2. Your tools, even if not complete, can help the developers test their code in early stages. 3. Your final product (i.e. automation code) will be better suited for the actual product 4. Bringing up your automation will serve as kind of exploratory testing, exactly when it is needed- during features development.

Answer 4

Automated testing makes the most sense when the interface is fixed and the implementation is likely to change: interface fixed because otherwise you waste time chasing a moving target, and implementation likely to change because otherwise there will be no reason to run your test after the first release. (To illustrate this idea, one reason automation developers like web sites that use predictable element IDs is that they provide a more stable (fixed) interface to test against than a mere DOM hierarchy.)

Your goal, then, will be to choose a point in the development cycle when the interface you test against is stable enough that you will spend more time testing than catching up to changes. There is no cut and dried method for making this choice -- after all, you are trying to predict the future -- but you can improve your chances of success by letting your developers know what you are trying to do and monitoring your developers' progress.