1

We currently use the following processes for testing:

  1. Write a test plan (currently stored in Microsoft Test Manger)
  2. Execute Test plan
  3. Once the tested functionality is promoted to Production, automate the Test plan using Borland SilkTest, for future regression testing

Question : Is this a typical process for a shop that uses automation? As changes to the original test plan change (due to future modifications), do you continue to maintain both the manual test plan as well as the automated one?

Thanks!

Improve this question
2
  • What does the test plan include? What level of abstraction?
    – dzieciou
    Commented Mar 23, 2014 at 14:04
  • The plans are pretty specific (enter "John" into input marked "first name")
    – Rudesyle
    Commented Mar 24, 2014 at 15:19

4 Answers 4

Reset to default
2

It is probably different for each organization/team/product/process, but here is a typical process I have used (starting from new functionality):

  1. Identify the key test parameters such as oracles, surfaces (variables like platform, inputs, outputs, etc), and risk areas.

  2. Explore the function under scrutiny at the time using multiple manual test sessions.

  3. Develop automation to make testing that function faster or identify functions that would make a good addition to a smoke test (fully automated) suite.

  4. Integrate automation into regression test automation suite as smoke test or independent computer assisted test modules that help setup or check specific areas of the application. This step is done as a part of the weekly or biweekly cycle while it is fresh.

  5. Only document enough to identify which automation to use with each regression test area. The function of the scripts are documented as comments in the script itself.

The key here is that regression automation is going to break on a regular basis, so a smart mixture of automated and manual test for regression is best to make it as useful as possible but still maintainable. The scope of the automation will work itself out over time based on the time you have available for implementation and maintenance.

Hope that helps!

Improve this answer
4
  • Everyone's answers have been great. Once you automate a manual test session, do you continue to maintain it over time, even after it's been automated?
    – Rudesyle
    Commented Mar 24, 2014 at 15:22
  • Additional difference I can see between Jeff's process and Rudesyle's one is that Jeff's does not define test processes upfront, but do it in more explorative way.
    – dzieciou
    Commented Mar 24, 2014 at 15:23
  • The point about exploratory approach is well taken. I have also been involved in waterfall government projects that required validation of static requirements in a fixed time frame. That project more closely reflects the process in the description. The intent there, however, was to demonstrate functionality and not a good process for finding defects.
    – Jeff_Lucas
    Commented Mar 24, 2014 at 15:41
  • The topic of automated script maintenance is important. When the script breaks, you have pressure to quickly get it working again, but you need to understand whether the failure is caused by an intended change. Naturally, that requires the automation be maintained "real time". The person running the scripts should be both an experienced tester and an experienced script writer to be efficient - a tough combo to find.
    – Jeff_Lucas
    Commented Mar 24, 2014 at 15:46
1

As the other answers have said, there is no "typical" process. Everyone handles things their own way.

That said, there are some common factors:

  • most places will have some form of test plan repository. I've seen everything from a mix of documents and spreadsheets in a shared directory structure to the high dollar professional tools.
  • most places will figure the test plan gets executed manually. It's rarely a good idea to automate manual test plans - what constitutes a good manual test is not necessarily a good candidate for automation.
  • most places that automate will have a way to identify tests targeted for automation and a strategy to add targeted tests to their regression codebase. The exact strategy varies a lot.
  • places that are serious about automation will not use record/playback whether their tool supports it or not.
  • places that are serious about automation will treat their automation codebase as code
  • places that are serious about automation will keep their automation codebase in a version control repository. It may not match the application code version control repository, but if it isn't matching the application codebase version control, it will be flagged in a way that allows regression to be run against any version of the application codebase required.
  • places that are serious about automation will ensure that time is devoted to maintaining automated regression.
  • automated regression for a new feature will not be built until that feature is stable.

It sounds from what you've said that you've got most of these points covered, so you're probably on the right track with your process. The main thing is that it works for your situation.

Improve this answer
2
  • I'm curious to know why you feel that serious Automation shops avoid the recording mechanism? We use it to capture a test, and then tweak with custom scripts
    – Rudesyle
    Commented Mar 24, 2014 at 15:20
  • @Rudesyle - There's nothing bad about using a recording mechanism to capture the identifiers for a test. If you don't customize and parameterize (and reuse that code), you end up with an unmaintainable mess that's loaded with repeated and near-repeated code. In an app where you've got to log on to do anything, a standard record/playback test will log on, do something, log off. Imagine having to update thousands of those because of a change to the logon screen and you see the problem.
    – Kate Paulk
    Commented Mar 28, 2014 at 11:00
1

Firstly, each company has its own test processes but it is typical in common. Secondly, each company has different development lifecycles so testing processes do differ. I can share with you our current test process:

  1. Overview project structure
  2. Overview diagram of requirements
  3. Get the software requirements specification document
  4. Write a test plan which includes the following sections: purpose and scope, functions to be tested, functions not to be tested, test approaches, test pass/fail criteria, regression test strategy, tools (test case management, bug tracking, performance testing tool, automation testing framework etc), test schedule, risk management
  5. Prepare the smoke test
  6. Create a functional test suite based on the software test plan which was created in the previous step
  7. Prepare the automation test suite
  8. Then agile process (verifying features in the end of each iterations), develop and updating automated regression test suite
Improve this answer
2
  • In step 5 you mention prepare a smoke test. Isn't a smoke test just a quick test where you click on some links to ensure nothing on the screen "blows up" ? A test plan, to me, would detail pass/fail scenarios
    – Rudesyle
    Commented Mar 24, 2014 at 15:24
  • @Rudesyle You are right actually but we have separated checklist for smoketest in our process. But, again, I have described our process :) Smoke testing is usually quick check of basic functionality and does not need separated test suite.
    – amazpyel
    Commented Mar 24, 2014 at 15:27
0

Testing Process:

1) We get any new project there is an initial project familiarity meeting

2) Discuss on who is the client? What is the project duration and when is its delivery? Who are all involved in the project i.e. manager, Tech leads, QA leads, developers, testers etc.?

3) Software Test plan

4) Developers start coding from the design.

5) The project work is divided into different modules and these project modules are distributed among the developers.

6) Tester is to create test scenario and write test cases according to the assigned modules. We try to cover almost all the functional test cases

7) When developers finish the individual modules, those modules are assigned to the testers.

8) Smoke testing is performed on these modules and if they fail this test, modules are reassigned to the respective developers for a fix.

9) If any bug is found that gets assigned to the module developer and get logged in bug tracking tool.

10) On bug fix, a tester does bug verification and regression testing of all related modules

11) Different tests are performed on individual modules and integration testing on module integration.

12) These tests include Compatibility testing i.e. testing application on different hardware, OS versions, software platform, different browsers etc.

13) On passing all the test cases test report is prepared and the decision is taken to release the product!

14) Process of project life cycle.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.