Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

The term "Test Automation Framework" is become a cliche in automated testing.

I have come across circumstances during work when my colleagues developed some reusable methods/function for an automation tool and propose it as if they have developed an automation framework. This is also often portrayed in Resumes.

I personally find test automation framework a very wide term which might include (and not limited to) - language support, IDE support, Test Report Generation, Recovery Mechanism, Test Execution Scheduling, Test Report delivery mechanism etc etc.

Creating some reusable methods in a language should/may not qualify as a framework, though it may add to over all framework. But then how do I let them know that their creation is not a framework?

share|improve this question
add comment

3 Answers

up vote 4 down vote accepted

Well there is the question "What is a difference between a library and a framework" over on stack overflow and I think that the accepted answer, albeit reworded there also applies here.

Wikipedia calls framework a "buzzword". It defines a software framework as

A software framework is a re-usable design for a software system (or subsystem). A software framework may include support programs, code libraries, a scripting language, or other software to help develop and glue together the different components of a software project. Various parts of the framework may be exposed through an API.

So applying a little search and replace magic I think you could define a test automation framework as follows:

A automated testing framework is a re-usable design for an automated testing system. An automated testing framework may include support programs, tools and other automated testing libraries to help develop and glue together the different components of a automated testing project.

share|improve this answer
    
+1 We certainly live in the era of buzzwords. How many of us are 'agile'? :-) –  corsiKa May 12 '11 at 5:39
    
@glowcoder no you mean 'Agile'. 'agile' is something completely different :-) –  Bruce McLeod May 12 '11 at 7:20
    
I would rather be 'agile' than 'Agile' with my process. –  Steven May 12 '11 at 12:29
    
If we didn't have buzzwords we wouldn't grow as a community. ;-) –  MichaelF May 12 '11 at 12:41
1  
@glowcoder and @Bruce McLeod, didn't you both mean 'frAgile™' ? :P –  yoosiba Jul 7 '11 at 14:27
add comment

Test Automation framework need to be evaluated on different parameters

  1. Design of the Framework - Record and play back scripts are not Test Automation Framework. Reason is code generated is not a maintainable, refactored code. They will not generate any ready made reusable code.

  2. Is it extensbile - You need to look at how the framework handles localization aspects. Do you have any factory based implementation where the methods remain same but test methods would be picked up based on a particular locale

  3. Data Driver Testing Support/ Leveraging best practices - Support for passing parameters in XML, Data provider support, Data Driven Testing, Initialization, Pre and Post test methods

  4. Logging aspects/How test failures are handled - Do we have a mechanism to re-run failed cases

  5. Reporting - How the results are reported. Failures reported with reasons. Good Enough for a business user to interpret the results

  6. Continious build integration - Can the framework be used to integrate with build and run it as continious integration build

Edited and made it clear based on Suchit comments.

share|improve this answer
    
-1 as this does not really answer the question. This answer contains random aspects regarding Test Automation Framework –  Suchit Parikh Jun 8 '11 at 17:50
    
I think answer to decide if give piece of code is a good automation framework has various aspect to look at it. Provided suggestions are to evaluate robustness of automation suite. May be if you can provide more clarity we can discuss further. –  Siva Jun 8 '11 at 18:01
3  
I think what @Suchit was trying to say was that the question here is more to do with inter-developer relations. I.e. how do you tell someone their "framework" isn't really a framework? While these points you have are a good starting point for evaluating someone's work, it doesn't bite at the real problem, which is trying to get people to not use misnomers and buzzwords to describe their creations. –  corsiKa Jun 8 '11 at 18:14
2  
-1 I'm with @Suchit and @glow on this... while this is good information, it does not answer the question about how to create clarity between what an automation framework looks like versus just a library of functions and reusable tests. –  TristaanOgre Jun 8 '11 at 18:18
    
Ok folks. Thanks for the clarification. My view is more in terms of evaluating it. –  Siva Jun 8 '11 at 18:25
add comment

It is hard to prevent people from misusing abstractions on their resumes, at work, in politics, and elsewhere in their lives.

Rather than arguing over the abstraction, you need to talk about the specifics in the context of how the test software will be used. So for example, if your manager is asking the test team for recommendations for a test framework, you need to be specific about what you need out of that framework, just as you would if you were shopping for a place to live or a car.

share|improve this answer
1  
Agreed and what frustrates me most is when I see people returning to same patten time and again - "I developed a framework". +1 for your answer. –  Tarun May 13 '11 at 6:18
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.