Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I have worked in Selenium. For developing Web UI Functional Test automation suite I considered below aspects

  • Using TestNG Framework
  • Using Page Object pattern
  • XML based configuration
  • Logging of failed cases

These are based on my learnings. There might be other aspects which can be added to the list.

If I have to develop automation framework using QTP, How does Test Automation Approach differ in this case. I have not worked on developing Test Automation using commercial tools

I have below Questions

  • Do we still use TestNG when you develop framework using QTP ? Someone who has exposure to both QTP and Selenium Can clarify my question
  • Similar to Page object Pattern what is best practice that QTP suggests for automation

Excluding Licencing & Opensource aspects,

  • What do we do different when we develop automation framework using QTP Vs Selenium
  • QTP can be used for non-web testing (ex - winforms)

Any best practices based on working with QTP ? Any related posts / Examples please post in reply

share|improve this question

3 Answers 3

up vote 5 down vote accepted

Note: Last I worked on QTP was back in 2008. My notes are based on that.

The most significant difference felt when I transitioned from QTP to Selenium was the programming / scripting language. QTP only supports(ed) VBScript, hence OO approach to test framework used to be more challenging for me. I still used Page object models. I used to create a text file with the element names and their descriptors.

HP Quality Center works best with QTP for test management purpose (to drive the tests, etc.)

QTP has this advantage over Selenium, that is automation of non-web applications. I used to identify windows with window handles. It was extremely easy to create an automation that included interaction with the web and other windows applications.

SQAFormus' QTP forum is probably the best source of knowledge and interaction in the QTP world.

share|improve this answer
    
Thanks Suchit, Any related posts-examples please add to your answers. Other Aspects TestNG/Logging was it required for QTP ? –  Siva Jun 14 '11 at 19:31
    
+1 Great points –  Aruna Jun 14 '11 at 19:38
    
I used to write logs to text file in the format that I had created. It has been long time so I don't really remember much, but if something comes to my mind, I will add to this response. –  Suchit Parikh Jun 14 '11 at 20:09

Guidelines For Automation Framework Design

Introduction

A Framework defines a set of guidelines for all phases of test automation: Requirement Analysis, Script Design, Execution, Reporting and maintenance. A framework can be a wrapper around some complex internal architecture which makes it easy to use for the end user. It also enforces a set of standards for implementation

Problem Statement

There is no standard set of guidelines available on developing a framework and what all considerations need to be taken during the development of the same. There are different white papers which go over types of framework and how they work. But none of them defines what all factors go in to the design of the same

Design guidelines

This paper covers different aspect of a framework and key features it needs to have based on the requirements.

  1. Selection of a framework – Different types of frameworks that exist are:

o Data Driven framework – Used when flow of the application remains constant, only the data changes. The data is provided by external medium e.g. – excel sheet, XML etc…

o Keyword driven framework – This framework provides generic keywords that can be used with any type of application. It also provides abstraction from the type of automation tool used and type of being application tested, e.g. – it can test a similar Web and Windows application with the same test case

o Hybrid framework – A hybrid framework is the one which takes advantages from both Data Driven and keyword driven frameworks. These frameworks do not implement generic keywords but implement business logic keywords based on the application being tested. For ex – Login, Logout could be application specific keyword that can be used.

  1. Don’t reinvent the wheel – A framework should try and use the power of the automation tool rather than re-defining the whole language by implementing new keywords. Developing a keyword driven framework is time consuming and costly. A Hybrid framework can be developed in a shorter time period and with less cost.

  2. Reusability -The framework should allow highest possible reusability. Combining individual actions into business logic provides re-usability. E.g. – Combing actions like “Enter username”, “Enter password” and “Click Login” into one re-usable component “Login”

  3. Support of different application versions -A framework should allow re-use of baselines scripts in case different versions/flavors of an applications are to be tested. There are two different ways to support different applications

o Copy and Modify – This method involves creating copies of the baseline scripts and modifying them for a specific application version

o Re-use and Upgrade – This method involves re-using baseline script and providing a upgrade code for specific version of application. This ensures maximum re-usability and should be preferred.

  1. Support of script versioning – Scripts should be stored in a version control system like CVS, Microsoft® VSS etc…This ensures recovery from any disaster.

  2. Different environment for development and production – Automation should be considered as any other development project. Test scripts should be created and debugged in Test environment. Once tested then only should be deployed to the production environment. This holds true for any emergency releases also

  3. Externally Configurable – Configurable items of a script should be kept in an external file. This would contain configuration like Application URL, version, path etc…This allows running the same script against different environment. Ensure that location of the configuration file is not hard coded. Hard coded files would allow running on any environment but only one at a time. Keeping the configuration relative to current test path allows overcoming this limitation

  4. Self configurable – Ideally a framework should be self configurable. Once deployed to a system, no manual configuration changes should be required and scripts should automatically configure the required settings

  5. Minimal changes required for any object changes -Most common issues faced during automation are object identification changes. Framework should be able to patch such changes easily. This can be achieved by storing all object identification settings at a shared location. This could be an external XML file, excel file, database or automation proprietary format. There are two possible way to load this object identification configuration

o Static – In this all the object definitions are loaded into the memory at the start of the test. Any changes made to object definition can only be loaded by stopping and re-running the test

o Dynamic –Object definition is pulled as per request. This approach is a bit slow as compared to the static one. But in case of huge scripts where the fix needs to be made at run-time this is suitable.

  1. Execution – Framework might need to cater to below requirements (on need bases)

o Execution of a individual test case

o Execution of a test batch (combination of tests)

o Re-execution of only failed test cases

o Execution of a test case/test batch based on result of another test case/test batch There could be many other needs based on the project requirement. A framework might not implement all of them, but should be flexible enough to accommodate such requirements in future

  1. Status monitoring – A framework should allow monitoring the execution status in real time and should be capable of sending alerts in case of failure. This ensures quick turnaround time in event of a failure

  2. Reporting – Different applications have different reporting needs. Some require combined results for a test batch and some require individual level test report for each test case in test batch. The framework should be flexible enough to generate required reports

  3. Minimum dependency on Automation tool for changes – Some fixes can only be made by opening the script in the automation tool and then saving it. Scripts should be developed in such a way that modification is possible even without the unavailability of the automation tool. This deflates company cost by reducing the number of licenses required. It also allows anyone to make changes to the script without having the need to setup the tool

  4. Easy debugging -Debugging takes a lot of time during automation and hence special care needs to be taken for this part. Keyword driven frameworks which use external data source (like a excel spread sheet) to read scripts keywords and process the same are difficult to debug.

  5. Logging – Log generation is important part of execution. It is very important to generate debug information at various points in a test case. This information can help find problem area quickly and reduce the time to make a fix at the same time

  6. Easy to Use – The framework should be easy to learn and use. It is time consuming and costly to train a resource on a framework. A well documented framework is easier to understand and implement

  7. Flexible – Framework should be flexible enough to accommodate any enhancements without impacting existing test cases

  8. Performance impacts – A framework should also consider the performance impacts of the implementation. A complex framework which increases the load time or execution time of scripts is never desirable. Techniques like caching, compiling all code into single library while execution etc… should be used to improve performance whenever possible

  9. Framework Support Tools – External Tools can be developed to perform tasks that help in framework design. Some example tasks would be

o Uploading scripts from local folder to HP Quality Center

o Associating library files to currently open scripts

o Synchronizing local files with HP Quality Center.

  1. Coding Standards – Coding standards ensures scripts that are consistent, readable and easily maintainable. Coding standard should define all the below listed things

o Naming convention for variables, subs, functions, file names, script names etc… Ex – i_VarName for interger, fn_i_FuncName for function returning interger

o Library, subs, functions comment header. This should include information like version history, created by, last modified by, last modified date, description, parameters, example

o Object naming conventions. Ex – txt_FieldName for a text box

Summary

Automation should be considered as a development project and not just record and playback of events. Starting automated testing with a good framework ensures low maintenance. Guidelines discussed in this paper can be used as input for developing requirements for a framework.

Search following on GOOGLE to get ready to use selenium framework

InfoStretch Selenium Framework for Automated Web Testing

What is the Framework? Tool of Choice InfoStretch Selenium Framework for Automated Web Testing InfoStretch has developed a framework for Web application testing and test automation using Selenium test tools that focuses on achieving a rapid return on investment – often in less than three months. The InfoStretch framework using Selenium has a robust set of pre-built features that include:

Parallel compatibility testing for many browsers from the cloud, with zero infrastructure costs..

Integration scenario testing across UI’s and Web services.

Highly maintainable and repeatable tests that utilize reusable test assets, proper modularity and semantic structure.

Data-driven Selenium tests with external Excel, XML and CSV-based data.

Drill-down, Junit-style test reports that show key automation metrics across multiple test executions with screen shots of failed tests..

Best practices for coding conventions and process guidelines, and coaching on how to make apps "testable."

Integration into many continuous integration/build automation systems such as Hudson.

Few more references:

http://stepauto.com/images/Shankar%20Arigela-microsoft.pdf

http://www.open2test.org/QAT_Tech_Docs/QTP_framework_introduction_v1.pdf

http://www.logigear.com/logi_media_dir/Documents/Automation_Framework.pdf

http://www-106.ibm.com/developerworks/rational/library/591.html#N10223

http://safsdev.sourceforge.net/FRAMESDataDrivenTestAutomationFrameworks.htm

InfoStretch Test Automation Framework

http://blog.infostretch.com/806

Choosing a test automation framework

http://www.ibm.com/developerworks/rational/library/591.html

http://www.byte-code.com/meetup2008/images/meetup_camza.pdf

Test Automation Framework

http://www.articlesbase.com/software-articles/test-automation-framework-378599.html

Designing An Automated Test Framework

http://www.infosys.com/IT-services/independent-validation-testing-services/Documents/business-rules-test-automation.pdf

http://www.automatedtestinginstitute.com/home/index.php?option=com_content&view=article&catid=132:test-automation-training&id=1358:designing-an-automated-test-framework#magictabs_fxunc_1

share|improve this answer
    
I am not sure how this lengthy answer relates to the question. –  user246 Jan 30 '12 at 14:25
    
I agree with user246 comments. Answers need to be precise, to the point. We should not try to address things out of scope of this question. –  Siva Feb 4 '12 at 18:47

I think you already got two rather lengthy answers.

Do we still use TestNG when you develop framework using QTP ? Someone who has exposure to both QTP and Selenium Can clarify my question

Similar to Page object Pattern what is best practice that QTP suggests for automation

I have had experience in designing tool agnostic Automation Test Framework for two of the top Airlines companies. So I will partially be able to answer your question. I have never used TestNG but I have seen it being used for some projects and from that - NO. It can't be used for QTP without heavily modifying/including a few things. And it won't be efficient after those modifications.

If you are fully utilizing the power of QTP you can do lots of things. But often we don't do that. One of the very prime examples of this is descriptive programming. Hence it depends on the application you are automating and also the client.

What do we do different when we develop automation framework using QTP Vs Selenium QTP can be used for non-web testing (ex - winforms)

If we look from the aspect of "Designing the Framewrok" rather than what they themselves can do. Then it finally boils down to the programming/scripting language. You will use vbscript for QTP and java(primarily) for Selenium. In my case I used selenium when I was countered with a cross browser testing scenario.

And as for your example. Yes it is much easier to use QTP to recognize and work on windows objects. But there will be time when you'll have to do that from selenium (which inherently won't be able to do that). In that case this should be handled from your framework (conditional vbscript using sendkey).

Hope this answers your queries.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.