What is a Test Automation Framework?

Question

Some automation engineers use an underlying automation framework as a jar dependency in their selenium web driver project in eclipse. Then, they use the framework to write functional scripting of the AUT.

I have started to learn scripting in webdriver and a bit about TestNG etc, but my colleagues suggested to me learning and creating a fully functional, generic framework that can be used by any team?

What exactly is a framework? Are there any courses/books online to learn automation framework design in Java?

Answer 1

my colleagues suggest me to learn and create a fully functional - generic framework which any team can use?

I suggest you don't. Your coworkers suggest you to start over-engineering. It is fine to experiment.

Automate a test, automate another one. Maybe a reusable abstraction emerges. Maybe this becomes a reusable framework. Practise the YAGNI (You aren't gonna need it) principle and KISS (Keep it stupid simple).

I see a lot of testers make their goal to build a framework, but software testing is about understanding and safeguarding the behaviours of persona. Automation and frameworks are just a tool. So yes practise with tools, but do not make it your goal.

Answer 2

I think you are missing out the definition of "Framework"

https://smartbear.com/learn/automated-testing/test-automation-frameworks/

A framework is comprised of a combination of practices and tools that are designed to help QA professionals test more efficiently.

These guidelines could include coding standards, test-data handling methods, object repositories, processes for storing test results, or information on how to access external resources.

While these are not mandatory rules and testers can still script or record tests without following them, using an organized framework typically provides additional benefits that would otherwise be missed out on.

So test frameworks are not just folder structures, it defines an overall process, tool, rules, technologies, coding standards, and so many stuff

Why we need a framework?

It makes code maintainability pretty easy. Once you have a framework that is efficient, you can code in parallel with high efficiency. For instance, you write code for drop-down selection and another engineer can reuse the code.

If you didn't have an efficient framework(naming convention and directory structure), the engineer should waste his/her effort in finding a solution that was already found out.

Why can't we reuse framework?

That is how all frameworks are being built, we take up existing frameworks and leverage it to create frameworks that are suitable for our project and team. For example TestNG, robot framework, etc.

All the test directory structures for UI tests are almost similar. Most uses page object model with folder structure like util, common, resource, report, etc. We just make changes according to different projects.

Why can't we have one fit solution?

The framework depends on organizational decisions and project complexity. For instance, the organization wants to save money by moving from VMS to containers. They want to run the selenium grid test in containers than in VMs, or sometimes the decisions may be to use the cloud.

So you have to investigate how the test-framework could be made ready for being put in CI/CD or delivering the additional requirements.

But in all cases, there will be a base framework from where we start building it.

And also test frameworks are company proprietary asset, it comes under intellectual property. You can't just share it with the public, the same as any other source code.

How to create an efficient framework?

Start from the base framework, learn from mistakes, keep evolving the solutions.

Most importantly, fail fast, meaning that you should realize the inefficiency of your framework faster. Analyze the drawback and make changes quickly, grow your test suites slowly till you are confident about the implementation.

Any Frameworks recommendation?

The framework depends on project, budget, organizational decisions and many such things. And as mentioned, frameworks are not just directory structure.

Recommendation for directory structure could be taken from GitHub or any such open source repositories. But you have to develop it in the ways you need

Summary:

1. So in your case, you have the base framework as testng+selelnium
2. You can leverage it to a common reusable generic framework by segregating your reusable code.

Eg: in one test file you have a code to read CSV, instead of keeping it in that specific file keep in a separate package called commons so others can reuse your logic when needed

3. Try to see if you can add data-driven testing in it
4. Separate the project to Page objects
5. When using third party tools keep the code in a different package called utils, so that others can reuse it

And mostly, you learn more as you start to do things and when things fail. Frameworks are not final, it is always evolving so don't be scared to make things go wrong.

Answer 3

I think the first sentence on Wikipedia says it clearly enough:

In computer programming, a software framework is an abstraction in which software providing generic functionality can be selectively changed by additional user-written code, thus providing application-specific software.

[ https://en.wikipedia.org/wiki/Software_framework ]

To say it in different words. You colleagues want you to write generic code that could be used on multiple test automation projects, therefore some of the code would not need to be written again for the specific project.

You can further have a look at some generic Java code in this tutorial. E.g. creating a sorting function that could be used with any data type is an example of generic code. In testing, it can look like this: you create a function that clicks a button, but what specific button is being clicked won't be hard-coded, but passed as an argument, thus allowing you to use this function on multiple projects with different DOM locators.

Answer 4

As a shorter way, Its a base structer or the foundation to write automation test scripts for different componets (web, mobile ... etc). A good framework will provide solutions for some of below limitations or the issues.

1. Handling scripts and data separately.
2. Creation of libraries.
3. Extensibility issues.
4. Maintenance issues.
5. Reporting issues.

Answer 5

Are there any courses/books online to learn automation framework design in Java?

In my case, I started learning with online videos about Java (focus in selenium). In my case, it was more helpful (and to be honest more convenient to learn via online video tool) I started with the link from Angie Jones Courses. There you will find good online videos. Also start with the main page AngieJonesTech and get an overview of that. Most topics are about Java and are about the DOM structure this helped in my case to understand the test automation procedure with Java. When you got more knowledge and maybe you made the decision to start with Java and Selenium you can have a look at ToolsQa

I am not quite sure whether you are a german speaker but for the German speaker, you also can find further information about this page TestingBoard

As you can see there is so much to do regarding learning Java. What I want to say is that I would start learning java with test-related topics as mentioned above.

Answer 6

Since you are just getting started with your test automation journey, imagine you are given an application and some test scenarios to automate. You use your knowledge of Selenium and TestNG and you start creating few Classes and Methods to perform operations on the UI controls as per your test cases.

Using TestNG you manage to execute those tests and you get your test results. Voila! You have automated your test cases (TCs).

Now the functionalities in your application grow and so do your test cases to automate. You continue creating more and more classes and methods to automate new TCs. Now about 3-4 months later you find that you have created several classes and methods, there is duplicate code in some places, there is no package structure to categorize/organize your class files and whenever you need to update object locators you find it difficult to find the code where you need to update because you haven't made any rules on how to organize your code in your Automation project.

Now with your experience, you understand that you need to setup some common/utility classes for common functionalities, object locators are properly organized at one place as class variables, a proper package structure is setup to organize related class files into respective packages and so on. You call them as the rules which anyone who newly starts working in your project must also abide to.

A Test Automation framework is exactly for that purpose - a set of rules on how to go about when building test scripts for an AUT.

These rules ensure better maintenance of project artifacts in the long run. General rules of a good Test Automation framework include:

1. How test scripts are written? - coding, feature files, ...
2. How test data is managed? - hardcoded data, data from excel sheets, ...
3. How object locators are managed? - Page object model, XML files, ...
4. How reusable components are managed? - Utility classes, ...
5. How all the test artifacts (classes, data files, configuration files, etc.) are organized? - folders and sub-folders, ...
6. How logging and reporting is handled? - log4j2, Assertions, ...
7. What reports to generate? - html, junit, testNG, ...
8. How execution can be triggered? - Eclipse, Command prompt, ...

Creating a good framework is a skill that comes with experience and not everyone can do it. There may be some recommendations on the internet on building some basic test automation frameworks but they are usually helpful only for simple automation tasks.

The best way to learn is to gain experience in Test Automation by automating several scenarios on an application, understand the problem areas, consider the kind of questions I mentioned above and start designing a framework with the rules that will help not only you but also others to follow a mechanism for building test scripts for your project.

Answer 7

Basically a automation testing framework is used to write a suite of automated tests which are run to give repeatable results back to the devs as quickly as possible.

Some points to keep in mind:

• You'll need to look carefully at your AUT and try to establish as many ways into it as possible, such as unit testing, APIs, front end, database etc. The more ways into it the more options you'll have, and the better your chances at success will be
• Take a look at exactly where the risk is in the AUT, which is usually where the complexity and the proposed changes lie. Try to get your framework as close to that part of the system as possible.
• Java isn't a bad language to learn, but I would choose the language based on what skills are already in the company. You can build a testing framework in pretty much anything if you want, but if the dev team are familiar with a tech then you'll get more buy-in. If there is already unit testing, you may want to build your framework with that.
• WebDriver is becoming legacy now in preference of other frameworks such as Cypress, which are easy to learn, easier to use, faster, more reliable and generally more powerful. You might want to do a side by side comparison of these before you go too far down the track.
• That said, try instead to test through an API. Testing through the front end is expensive, and API tests tend to run quickly and give better errors and more consistent results. Don't be afraid of having multiple frameworks that tests at the various layers of the application, that way you can have options as to where to place the testing.
• Don't try to dumb down the framework or test suite so a manual tester can use it. If a manual tester wants to learn automation, they really have to be able to code if they're going to be effective, especially when it comes to troubleshooting.
• Environments are a critical part of your testing effort. Do not underestimate how much time can be lost getting these right. Get used to having lots of moving pieces too, it's the nature of the beast.
• The more often you run the tests the better. Avoid the approach of running a huge test suite nightly or having test runs that take a lot of time. People will ignore the results if they can't get that rapid feedback.
• Your tests need to be utterly reliable, or people will ignore the results. The importance of this cannot be overstated.

Answer 8

Although test automation is still new, businesses are looking to centralize their testing environment to effectively automate their legacy, web, and mobile applications with a single tool. Here is where the model for test automation starts. It lets us centralize all of their test automation properties, regardless of the tools you use.

What is the Automation Framework?

Automation Platform is a set of tools and processes that leads to achieving automated testing of any application. It incorporates different functions, such as libraries, test data, and reusable modules.

The framework promotes automated testing as a reference for technical implementation. Consider, for example, a case where the testing team involves members who are focused on different automation test codes. And they are unable to understand the standard pieces of code and scripts that have been revised by a team member in the project.

The automation system not only provides the advantage of reusing the code in different contexts but also lets the team develop the test script in a standardized way. As a consequence, the test automation system addresses all issues.

For the test automation framework books, you can refer to:

https://www.amazon.in/Hands-Automation-Testing-Java-Beginners/dp/1789534607 https://www.goodreads.com/book/show/12563795-experiences-of-test-automation

Answer 9

As per definition:

set of practices, assumptions and concepts used to design automation frameworks utilising testing tools which are used for automation testing

."

Primary Benefits of an Automation Framework:

1. Reusability of Test Scripts
2. Flexibility in Adapting Changes
3. Comfort in Training New comers
4. Better Maintainence

Reference: https://www.google.co.uk/books/edition/Software_Automation_Testing_Secrets_Reve/AK44DwAAQBAJ?hl=en&gbpv=1&printsec=frontcover

Example Automation Framework: https://github.com/webdriverio/cucumber-boilerplate

Sample Test Framework in CypressIO: https://github.com/narayananpalani/cypress-test-techniques

Myth about Automation Frameworks:

Myth 1: Automation Tool is nothing but a framework

Automation Tool itself is not a framework-Tools such as UFT One or Selenium are just tools and not framework themselves. When selenium get adapted along with TestNG,Cucumber in a first test in which a functionality being tested, it is converted to use inside an automation framework. Until then it is just a tool

Myth 2: Any automation tool integration can be a framework.

Not all integrations/frameworks are automation frameworks-Tools integration such as QTP to ALM is not an automation framework. It is simply group of tools.

So what really makes the difference as a true automation framework? A framework should be belongs to atleast one category on the below list of frameworks as per the book:

Myth 3: All frameworks are reusable

It is a wrong concept. Many frameworks are simply depends on license usages and limit the user from adapting to future projects. when scripting the frameworks, there are certain features are to be considered.

A framework should suppose to have list of objectives met such as test management,test design, test execution and test adaptation

List of features in a good framework and gTAA (general Test Autoamtion Archtecture) can be referred at: https://engineers-hub.teachable.com/p/istqb-advanced-level-test-automation-engineer-professional-training-with-q-a

https://www.istqb.org/certification-path-root/test-automation-engineer.html

https://labs.sogeti.com/benefits-of-generic-test-automation-architecture/

Answer 10

In a layman’s language, a test automation framework can be considered as a set of rules and guidelines that are used for creating test cases along with some combination of practices and tools that can simplify the automation testing solutions.

Though most of the test automation frameworks are open-source and are easy to adopt for people who have been associated with the QA background, one can still explore sources like software testing help, Browserstack, etc. to learn the process. Also, you can always take your queries to the community for quick and expert assistance on anything that seems complicated.

Answer 11

A test automation framework is basically a bunch of guidelines such as test-data handling, coding standards, object repository treatment and so on, which when followed while doing automation scripting leads to profitable outcomes like increased code re-usage, decreased script maintenance cost, higher portability and so on. These are only guidelines and you can still script without following the guidelines. But you will lose the opportunity to get the advantage of making use of a framework.

Answer 12

I would not go for any long answer here as I know during your struggling phase you would have come across multiple stuff like that. I will simply suggest a short solution that worked for me and further landed me in a product based MNC company with an awesome package. I understand you have limited knowledge but don't worry. IT employees with limited coding knowledge only do come in testing. Rest depends on them if they want to improve they require hands-on and practice..remember "Practice makes a man perfect". Just do the following things, buddy, as I instruct and I guarantee you will notice results in just 20 days.
At last..if you are reading it, do trust me:

1) Invest in yourself: By this, I just mean start buying some important things. You know after working all day in the office your eyes need rest but ears are always open.
a) Buy a small speaker(no headphone pls) max 1000rs just for personal use
b) Buy a pen and notebook. Just make sure to open your notebook and pen cap and sit down. Do whatever you want on your phone but meantime does open your laptop.
c) If possible, buy any java book from amazon. It's a general humane nature people tend to look for the return on investment. So if you will buy a book, it is or sure you will start turning its pages if not reading it though.

2) Start with java first: Remember to be successful in the IT field you at least need to have knowledge on loops. Be it any program but looping is important. Read java from a book and take online help whenever you want. do it for 10 days..from the 11th day you will self start enjoying it.

3) Once java is a bit understandable say 20 percent. Tune in to youtube and go directly for core java interview questions videos. Here that speaker use kicks in. Plug it to the laptop and listen at least to whatever is being said in the video. Further, if you fill its important to note down also write few in notebook and practice code in eclipse.

4) Day 15> Tune in to youtube and search selenium course on youtube. Prepare it for basics.

5) Day 19> Again on youtube, search for automation framework full tutorial and select any long video say more than of 2 hours stretch. Always play using speakers not headphones as it will help you again noting down in a notebook.

6) Practice practice and practice.

Best of luck mate!