What advice can you give for encouraging Project managers, and Developers to factor in tests for purposes of proving an application system's performance, resilience, capacity, scaleability etc beyond the testing conducted at unit testing stage. There is more emphasis/reliance on citing the efficacy of the design concept (and Unit testing) as the sole insurance for meeting such requirement. In short, also assuming that the system will run as designed under the less controlled circumstances i.e PROD like it did in the unit tests. Consequently, we may not see monitoring / evaluations of performance results from the functional tests conducted in the UAT period. We need to use UAT gauge the underlying system performance to see if there are performance limitations under more production-like circumstances when all the processing chains are integrated for the first time. However, we are finding that the project solely focuses on the Business results. See also article Why is separating functional and non-functional test important (or not)?
|
|
Suggest having dedicated Non-Functional (NF) Test Management practice separate from the functional practice (UAT etc.). Also NF needs to be much more closely aligned to (i) Infrastructure Architecture, and (ii) Production Systems Management. Also have the NF testing as separate test phases with separate governance. Have used this approach successfully with a number of clients and it works much better than having combined Functional and Non-Functional. Be prepared for Functional Test Management to want to govern NF - they will always fail to understand it and miss-govern. The separation really needs to be at the CIO level. |
|||
|
|
|
First, you're going to have to understand the Project Managers and Developers that you work with. Not knowing them it is hard to understand what sort of arguments might persuade them. Think back on arguments you've had with them, what kinds of arguments did they use to try to persuade you. In all likelihood a similar form of argument will likely hold sway with them. For instance, if they are always pointing at customer data, you need to form your argument using customer data in some way. Talk to them in their own voice. It's also possible you can also appeal to company culture in some way. If your company prides itself on being frugal, thorough, well-designed or user-friendly those are the terms to use when forming your argument. Finally, I think you need to clarify in your own mind what you're trying to accomplish. Don't test in a vacuum. How is better performance testing, for example, going to further your companies business goals? Can you explain your reasons to yourself or to a teammate in a clear single sentence? Spend some time tearing your own ideas apart, make sure they're defensible. |
|||||||
|
|
I lost you somewhere along the question... Anyway, upper management are usually convinced by showing them the ROI, for example if you can show problems that could have been detected and solved during a non functional testing phase. |
|||
|
|
You need to connect the dots. There are at least three messages to convey: your production system has problems that are worth worrying about; those problems result from your shortcomings in your organization's current practices; and the best way to address those shortcomings is to introduce new tests, i.e. to use your terminology, additional UAT-level tests focusing on the application system's performance, resilience, capacity, and scalability. You did not describe the nature of your production problems. You also did not say whether the project manager and developers are aware of those problems or share your concerns about them. Your first message may need evidence about the frequency/severity/impact of those problems. Specifics are more compelling than a general sense of unease. Sometimes production problems are caused by factors downstream from the development team. Your second message needs evidence that these problems could be traced back to the development team and not, for example, to sloppy practices in the production team or infrastructure problems at your ISP. Finally, your need a persuasive argument that investing time in new UAT-level tests is more important than other things, because there are always other things. Adding tests may slow your delivery time, require spending more money on headcount and/or hardware, and/or delay the introduction of important new features. |
||||
|
|
At my org, we have 'bug bashes'. Besides structured testing, you invite others who are not very familiar with the feature to have a go at it with the sole intention of destroying the software. Anything and everything is fair game. We've found that this uncovers the most number of bugs. :) |
|||
|
|
|
Project managers ultimately report to a more senior person about how the project is going. By performing more types of testing, you can give the PM more ammo to provide to the senior mgmt about how reliable, scalable the system is and - assuming this is a problem across the company - how innovative the project is. To do this you will need to be able to quantify the effect of the extra testing you are doing, otherwise it will be seen as wasted effort. Give the PM a graph of how the speed of the system has improved over time. The PM can then show that to senior management and you both look good. |
|||
|
|
