Why do many jobs describe automation but then require extensive manual testing?

Question

The question somewhat stems from frustration after seeing recent trends in software testing jobs:

In Job Description, automation skills are given utmost priority. All the interview rounds will be based to assess your coding skills and knowledge of Selenium, Appium and other automation tools. But once you are hired, a lot of job portion will involve manual testing given the urgency of releases in Agile teams. Automation is just something which testers perform whenever they have free time which is very little. Companies do like the term Automation but generally have poor knowledge of proper strategy and unrealistic expectations from it and hence(from where I see it), it gradually becomes the least priority.

Automation Tester profile was considered a very good option for people who like coding and as an alternate to a developer profile. There was a time where most of the companies hired dedicated automation developers but that trend has stopped may be due to low ROI or failed automation strategies. Many professionals jumped into this profession due to the boom since last 4 years but now they are having very limited options for the kind of work they want to do or being told to during hiring. Result is testers are hired saying that their primary focus will be to automate the regression suite and other repetitive work but within few months are asked to mostly manual. Now, of course, manual testing is a great job but why is there a sudden drastic decrease in demand for automation testers?

The ideal job as per the "job description" for most companies for "Automation QA Engineer/SDET" profile would be somewhere around 80% automation with some manual tasks as per needed but the actual scenario is totally opposite. Many automation testers who are passionate about automation are left with frustration of doing something they were not hired for. Are pure (or 80%) automation jobs dying? What is the reason? What is the future of automation testers? Is this trend specific to companies?

Answer 1

Following the Skeptics SE guidelines I would expect a question like yours to be accompanied by sources, otherwise it's just a collection of anecdotes.

To answer your question my own experience and knowledge is the opposite, more and more companies are moving to Modern Testing or some forms of Combined Engineering where testing and quality are owned by the entire team.

In those teams testers are acting as specialists helping the team achieving their goals for example by kicking off automation framework and teaching the team to use them or coaching others (i.e. developers) how to design and perform better manual tests.

If anything an automation engineer is an expensive option for doing manual testing on a permanent basis, but reality is stronger than any theory and sometimes under pressure companies will use the available resources at hand to solve the immediate problem using resources inefficiently.

Is it possible that you simply had "bad luck" filling positions the ended up as being"abused" under pressure?

Answer 2

What I observe is that it is hard to hire good programmers who specialize in automation and the particular set of skills needed for it.

The result is that you end up with either manual testers or full stack devs doing testing. All the other varieties of 'other people writing test automation' (as I term it), lead to a lot of communication and goal issues.

The process of trying to hire good technical QA people usually goes something like this:

company has quality issues
\|/
they need an automation specialist
\|/
most of the applicants are either manual testers or...
\|/
most of the technical candidates would rather do app dev and be paid more
\|/
the selection process is done by application developers who have different priorities

The above process mean that automation specialist skills are frequently poorly understood. For instance many automation programming interviews go down the path of recursion, performance, Big-O, etc. topics instead of the skills actually needed in automation such as BDD, Agile Test Pyramid approaches and Test Suite DSLs.

Also many interviewers who are newer to the field are often still in the mode of academic competition and tests to prove competence. When a QA applicant doesn't 'test well' according to an application developer the result is usually a determination that the QA applicant doesn't have what it takes to code. I see this sad story play out dozens of times for many of my fellow programmers that I've worked closely with. So I do also have that experience to draw on. Not just my own limited experience - although I have several decades of that too and observed this pattern repeatedly over the last 20 years, as have many of my colleagues.

At the end of the day it may be as simple as:

• application code provides todays revenue and is simplest to justify to the business
• paying more for application devs leads to better ones going there

Answer 3

I can give an example from our company which is nearly the same what you desribed. Let me shortly explain how we worked. We got two departments, one was the IT-department - which had testers (manual and automated testers) and we as business department had also testers (more epxloratory testing and manual testers). Most of our work was done twice, but nevertheless we detected a lot of errors when we set deployment on Production. Hence we also deciced to do the automation testing as business department. We started with Selenium and Appium and we also encouraged to hire automation testers (in this case we hired consultants).

But unfortunately the consultants - who are specialised in test automation - at the end did a manual testing job (means creating test cases, documenting errors, reproduce errors whether these are valid bugs and so on). So as you already said his profile was 80% manual and 20 % automation instead of doing 80% automation and less 20% manual tasks.

What were the reasons?

1. New web application changes locators on a daily basis

Well, we got the task to test a new web-application. But this web-application was somehow a marketing web-page. The locators (xpaths,ccs-locators...) were changing nearly on a daily basis. Hence as business department we had more the focus on testing E2E-applications. But this was not an E2E-application. Creating automated test cases didn't make sense at this moment and we hadn't the time to create test cases and changing them on a daily basis. So we did exploratory testing. Furthermore we had to evaluate whether test automation makes sense. We used QaSymphony / Tricentis for exploratory testing and measured at the time for testing, exploratory testing was saving more time than doing automation testing.

2. No need for test automation - the developer does this stuff at all

Somehow this was also a struggle for us. Our Product Owner was saying that test automation is already done from development team, hence test automation was not required (!). But he didn't understand, that a developer cannot replace a tester (organizational blindness). Furthermore the development just tested a small component of the whole application and did somehow mostly unit testing.

3. Budget issues

We got lot of people who left the project or did another role. The budget was also reduced. And test automation costs money, the business department was (at first) not willing to pay the money for test automation because we had to convince the Product Owner & It-department. But there was also a need to test the application - but from point of business department manual testing is engough!

4. No time for test automation - do test quickly

That was also a relevant point for us! As a small testing team we just had 2-3 applications in our responsibility. But the management decided to overtake several applications. And with our small test team it was hard to coordinate all testing activities. So we concentrated on the relevant test cases and started with manual testing instead of establishing our test automation strategy. At the end, our test automation engineer was doing 80% manual testing and just 20% automation testing (and this was just for preparing the test environment and creating the first test cases). The management wanted to see results quickly! Furthermore we had to focus on retesting bugs (since we are the business department we had to do somehow a UAT and verify that the bugs weren't available anymore).

5. Problems with firewalls and other technical stuff

In our company we had also a strict policy that we weren't allowed to introduce test automation and using e.g. Selenium & Appium and connecting them with real devices. At the beginning we asked our consultant to create and run automated test cases on their devices! This was a solution but this took us at least 1-2 months (ordering devices like iPhones etc.). At the beginning this lead to frustration among our testers who were willing to introduce something new.

These (and many more :-) ) were the lessons we made. But at the end let's say that somehow we managed it to introduce test automation on a 50% to 50% basis ( 50% test automation and 50% manual testing).

Answer 4

I am just rearranging the order of your questions so that this could be answered more clearly:

Is this trend specific to companies?

The answer would be more product-specific, and test level-specific than the organization.

Let's see automation approaches in different test levels:

System testing:

Critical applications

Consider critical banking applications, network devices, server technologies, etc. in this case companies won't risk releasing a defect prone product to market with the hope of fixing any production bugs later.

In such critical cases, the company hires domain experts to have the product go through rigorous in-depth manual testing. The manual testers write proper manual test cases and then move them to dedicated automation teams that automates the entire workflow. Here, we can see 100% manual teams and 100% automation, teams.

Non-critical Applications:

In this case, Defect Detection Percentage won't be considered to be much of a critical metric. Here, testers need to cover only the most important functionality coverage ( if we see security as a different level). Here companies don't need dedicated manual testers who do rigorous manual testing. They need more of QE engineers, who identify manual scenarios and automate it and then integrate it into the CI/CD pipeline.

Here we can see 60-80% manual plus automation. This approach focuses initially on finding as many manual scenarios as possible and then change the efforts to 80% automation and 20% manual in the future sprints. (This is what you are talking about in your question)

API testing:

Here we can automate even the critical applications. When there is proper API documentation you can start writing automated API test cases even before the product is developed( Test-driven testing). THe automation scripts could be written by using dummy JSON response. In such approach we have 100% automation test team and 0 manual testers.

What is the reason?

The reason for the trend you mentioned is due to less ROI and more cost to the company. Imagine you have a dedicated SDET that doesn't have any knowledge about manual testing, so they have to depend on manual test cases from the manual team before they start automating. Also, once the product becomes stable the manual team will not have many tasks to do and it will become redundant for the organization. This is why some companies outsource manual testing, as they believe it gives more independents to test teams and provide higher defect detention percentage.

In this case, companies hire candidates with both manual and automation skill, who tests manually still the product becomes mature and then in parallel automate what could be automated

Are pure (or 80%) automation jobs dying?

As I mentioned, the automation percentage depends on the product. Every QA should be proud enough to say that " Automation scripts are only as good as the manual test cases". SO, if you hire a developer of test automation, the scenarios he would cover will be far less than an experience QA.

Companies don't care whether you do things manually or through automation ( Until appraisal). So don't wait for someone to tell you to automate something, just automate and demo it as PoC and surely it will be incorporated in the organization. Be a self-starter, and don't wait for directions.

What is the future of automation testers?

Automation testers are in more demand than ever before, you cannot find a job which doesn't require automation skills. But keep in mind anyone with logic can learn automation but manual testing is a talent that not everyone can grasp. You should be highly passionate about breaking stuff, finding loopholes, wearing the white hat and so on.

Improve your domain knowledge, have a passion for manual testing, have the skills for automation and mainly have the knowledge about CI/CD

Answer 5

I can think of several issues:

• Test automation fails if the test automation engineers are too far separated from the software engineers.
  In a perfect world, fully automated tests allow continuous integration and deployment. That means almost any software change requires a test change to make the tests run through if the change affects APIs, or service architecture, or whatever.
  In a perfect world, few things are tested twice. So the test automation engineers need to be aware what the software engineers are covering with their unit and integration tests, and automate only the rest.
  So you have two kinds of engineers working closely together. Why not label all of them "software engineers" and make test automation a task for the dev team? If you do that, those assorted programmers may be tempted to push the test load to the manual testers and do features instead.

• Test automation requires good requirements.
  All too often I've seen requirements (both in agile and waterfall projects) which did not describe the result well enough to write automated tests. Instead you had the software, and the requirement, and you said "the software appears consistent with the requirements." The test automation would have to be adapted each time the software fulfills the requirements a different way.
  For some projects it would be nice if the requirements were good enough to write the tests without reference to the software. But that culture change must come months or years before the test automation engineers can start to work.

Answer 6

What you see is actually a not uncommon pattern and not limited to software testing.

Many companies advertise the job that they would like to have, not the one they actually do. I am sure that in the heads of the managers there, most testing is automated and a few edge cases require manual testing. They may or may not understand that this is their vision, but not (yet) reality.

This happens in many, many jobs. There's the idea of the job and there's the actual thing. It does not necessarily indicate bad faith. There is usually a combination of a) managerial detachment and b) wishful thinking. Especially if the company is transitioning away from manual testing towards automation, managers often believe that they are further along that path than is really the case.

It can also be that the actual idea of the job was that you should be the one introducing or completing the automation, and the details were lost somewhere in the five corporate departments involved. Especially in larger corporations, so many people meddle with job descriptions that this can easily be the case.

Your best approach is to ask clear questions during the interview. Such as which fraction of testing is currently fully automated and to what fraction they intend to push that within the next year. These two numbers will give you a good idea of how well the people you talk to know the actual process and how aggressively they are working on more automation. (and if someone says "100%" you know that they're bullshitting you)

Answer 7

I think this is because Automation is still relatively new, and a buzzword.

Most organizations love the sound of it, and the 'immediate return' it could promise.

But because it's so new, they just don't know what it entails to set it up and do it right, nor any infrastructure for it; so after floundering for a bit, fall right back into manual testing.

Answer 8

I think most companies are late adopters when it comes to test automation practises. They understand they need test-automation to become more nimble, but they lack the guts to stop current development for three months and turn the ship around.

So most companies that make software that could hire you have a product with atleast a 2-3 year old codebase. Small products tend to start small and testing, deploying is easy by hand.

This is where it goes wrong, teams start with doing things manual. Now three years later loosing these manual habits is hard, automating them is costly. The employees are not motivated to challenge management and are accept their mediocracy or think it is job-security. This is why most companies have product that are pure Legacy (e.g. code without test-automation) already. Read working effectively with legacy code.

Still they try to hire for the future, but are stuck in the past.

My job switching advice: When looking for new employers, not just talk, but look at the code, look at the tests, look at the processes. Ask to work a single day in a team. Validate they practise XP and things like Continuous Delivery. Most software people can choose where they are going to work, as demand is larger than the number of people who can do the work.

Answer 9

I once surveyed 25 testers from Polish testing forum why it is hard to find a test automation specialist. One of them said that this is because companies cheat in job offers:

Many companies looking for testers claim they need someone to automate or able to learn automation and then it turns out that it is only about manual tests, and automation was just PR, which was supposed to persuade a candidate to accept a contract. When you sign a contract, hardly anyone resigns.

Perhaps it's a vicious circle. Companies see that engineers are interested in test positions if it involves automation. So they describe job positions accordingly. Engineers become aware of such misleading descriptions so only less experienced engineers apply.

Answer 10

I think you basically answered your own question.

But once you are hired, a lot of job portion will involve manual testing given the urgency of releases in Agile teams.

A lot of folks have high aspirations, but timelines and resources can limit delayed reward work like automating testing. And people within the organization may have different views on the value of it. Things that don't deliver "right now" can be hard to push past business owners.

Answer 11

To me, they want to make sure QA has some coding idea. Because anyone can be a manual QA by learn the system like how it works, end to end work flow. But the main problem with manual testers, they do not understand code in qa, dev, prod. So all the things they treated as Bug, May be that's not bug the code not pushed to QA yet.

So, a lot of devs want QA who has little idea about coding so that they can understand properly. That is why they are looking for Automation QA although they don't need it or time for automation.

Beside this, Automation has very limited chance to open a bug. I use automation only for regression testing to save time for manual testing and use manual testing for opening or finding bugs.

Both sides have limitation that is why I like do both.