Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It only takes a minute to sign up.
Sign up to join this community
This question was inspired by: Fitting regression testing in a Agile/ Scrum development cycle
Two out of three answers there suggest a hardening sprint every couple of cycles. There's also mention of adding exploratory testing throughout the sprint.
But in Kanban, theoretically you can decide to release at any point. How do you fit regression or other "whole system" testing - like say performance/load testing - into that?
With Kanban you don't really have the problem of the sprint ending before you are happy that a feature is fully tested and working, so there is no need for an equivalent hardening sprint. Instead you would hold the work item in the test state until you have a robust set of automated test that verify the acceptance criteria. Limiting the work in progress means that is there are a lot of testing tasks then the team will need to focus on reducing this before new work items can begin.
Performance and Load testing should be ongoing and part of the test state and added to for each work item that enters into this state if required.
As for exploratory testing, you can arrange a time, for example every Monday 9am to 11am , that the whole team will do exploratory testing and then discuss the results.
You will also need a mechanism for keeping incomplete features out of a deployed build such as feature switch that will allow you to turn on and off a particular feature.
In a lot of cases you work hard to have as much of your regression/performance/load tests automated and run on something like a nightly basis (since those tests can often take many hours to run, you don't want to run them with every CI build, but you do want them run regularly as often as is practical. Having the Acceptance/Regression tests for a new feature running is one of the final phases of getting that item through the pipe as it were, and it should not release if those things are not online and running.
It does seem to me that the deployment cadence is almost always of a lower frequency than the delivery (by devs) cadence. Also, that you don't get value to the customer until you pass a critical mass of components. Also, there is often a piece of legacy software that is resistant to automated testing. Also, (but not finally), stuff comes out of the woodwork that no one would think of automating, maybe because the solution, or technical designs just got it wrong.
These make me think that it might always be appropriate to have 'super stories', that each depend on a number of stories being delivered into the pipeline (not deployed), that allow for exploratory and user testing, and which end with a deployment.
Am I being too old hat?
