Test Strategy Documents seem obsolete in agile teams. Am I wrong?

Question

Test strategy documents are obsolete?

Is Test Strategy document no longer relevant for testing in the 2020s? No one cares what it says, only that it exists to because some people want it. This violates a principle of the agile manifesto:

Simplicity--the art of maximizing the amount of work not done--is essential.

Am I going to do meaningless work?

I've been asked to write a test strategy. There's no Azure work item for this. You only get test plans, test cases and test suites. Maybe Microsoft's caught on that this isn't used.

The big advantage that test plan, test suite and test case work items have over test strategy in a word processor or wiki is binding. Azure binds the test work items along the at planning, execution and reporting phase and tied to a specific release.

Whatever test strategy we decided to use is embedded knowledge in the test plan, test suite and test plan concepts. So what's the point of making one of these up (except to cover my ass because some old-school auditors want to work in MS Word?

See here what ISTQB wants to write and let me know if there's a need for a strategy and plan

Test strategy is:

Documentation aligned with the test policy that describes the generic requirements for testing and details how to perform testing within an organization. Synonyms: organizational test strategy

Used in Foundation V3.1 - 2018 syllabus

Test Plan is:

Documentation describing the test objectives to be achieved and the means and the schedule for achieving them, organized to coordinate testing activities.

Used in syllabi for Specialist - Performance Testing - 2018 and Foundation V3.1 - 2018 Reference: After ISO 29119-1

No one cares about these documents.

No release ever got stopped because someone read a test strategy. Am I just covering my ass in case of audit, or to say I told you so? Does anyone else wonder what the point is?

Update

I confused the outcome of making a test strategy with the artifact it produces.

Kaner, Bach and Pettichord say in Lessons Learned in Software Testing, 1st edition lesson 143: Don't use test documentation templates: A template won't help unless you don't need it. If I am a skilled tester, I don't need the document because I already know the strategy.

In lesson 144 they say: Use test documentation templates: They foster consistent communication. At times I will need to work with others - from feeding back to the developers, auditors, other testers and the clients.

How do these answers resolve my confusion in 2020?

While I may believe I know the strategy, there's no way to test (1) whether I really know it and (2) it is valid for the current test cycle. Going through this process validates that the test strategy is sound, or identifies strategic errors. While I may believe others know the strategy, I can't tell if they really do and if it's valid. Making a strategy is a learning exercise.

I only hate creating the document because I've exhausted what I believe I can gain from mature templates at a test strategy level. That's possibly a call for us to revisit test strategy.

Getting another team to provide a test environment takes a lot of time. It's their fault if the software fails because of an environment issue. That said, if we could, at will, create an environment equivalent to what the operations team would give us, the time wasted waiting for an environment that's not good would go down. Without talking about the test strategy, this idea wouldn't come up.

The outcome of test strategy is validating if there is opportunities to do the test process better. The artifact of test strategy is measuring if the opportunities to test better are met (with less waste) and satisfy the usual people who want some document (auditors, customer).

It is up to the organisation to choose to use the test strategy for this learning purpose. There's no prescription to do so. As per the agile manifesto,

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Answer 1

"Planning is everything, plan is nothing." -Dwight Eisenhower

Its not the document itself , but what goes in the process while making it , is important.

Also importantly, it should answer clearly & uniquely(in a given context) what we are NOT going to do in fact more than what we are going to do as part of test strategy.

As long it evokes the right questions about right things in the given context and those questions are very well answered from the right people - in short its adding value in the context, it makes sense to develop it and capture that valuable information for future reference and course correction.

Also it should be a living , evolving document instead of static one so that we can revisit it time to time and update as required based on our learning project experiences as a team.

Good Read: How to design an effective test strategy

Answer 2

The idea of what is considered essential or wasteful work depends on the context. What is wasteful for one organization could be value-adding work for another, based on their needs and the needs of their stakeholders. It's not possible to make a blanket statement about the value of a test strategy.

That said, the idea of a test strategy that provides guidance on what type of testing is performed, when it is performed, and who (particularly at an organizational role level) performs it could be valuable to explain an overall verification and validation process to an auditor or to help onboard new employees.

I don't believe that there's an overlap between a test strategy and the concepts of a test plan, test suite, and test case. A test plan tends to focus on a particular period of testing - a test plan for a release or a test plan for a penetration test, for example. Test plans tend to have specific dates, times, and individuals defined as to who will be executing the work and key milestones. A test plan may reference specific test suites and/or test cases, along with supporting tools and environments. Test suites are collections of test cases, which are specific inputs, conditions and operations, and outputs or results.

A test strategy may reference some of these. For example, test strategies that I've seen often define the criteria for a controlled test environment - who builds and maintains the necessary tools and infrastructure, who deploys software, and if there are any rules about the data and software put into a test environment. Similarly, the test strategy may reference different types of test suites - regression suites, performance suites, or suites specifically related to requirements associated with regulatory compliance - that are stored in and may be easily extracted from your test management tool. If you perform different types of testing at different points in your development life cycle, the test strategy would also define these - when you carry out functional testing, performance testing, security testing, and where these different types of tests are executed.

If you find writing this type of information down useful, then perhaps making a test strategy is a good idea. You may also want to control it - put restrictions on who can edit it, require approvals on changes, or set up a regular review and reapproval cycle. However, writing it or controlling it may not add value, in which case it doesn't make sense and you can skip it.