2

My problem is as follows. Currently I am on a project, where we have a standard product that gets changed/bespoke for each client individually. My task is to create a test approach, in order to test the product, but also test for each client individually. As we are a small testing team, testing for each client individually will become impossible if we get to a large amount of clients.

So, please can I ask if someone has been in a similar situation to give guidance?

  • After you deliver your product to your client, do you still need to maintain your client's changes? In other words, when something changes in your standard product, do you need to re-test for every client you have ever had? – user246 Jul 7 '15 at 14:06
  • Yes, currently we do. But I do not want to go this route. – Eugene Buitendag Jul 7 '15 at 18:48
3

My recommendation would be to build an automation suite, on the base of the product. Then, to have your testers manually test the differences in the product based off of your client requirements. Unless you hire other testing resources, automation will do a lot of the testing for you, in the base product.

  • 1
    Perhaps some of those client specific tests can be automated too, depending on their complexity. – corsiKa Jul 7 '15 at 22:50
  • Thank you so much for your feedback - this was the answer that I was looking for and confirmed exactly what we had in mind... – Eugene Buitendag Jul 8 '15 at 3:57
  • Do you maybe have a test strategy example document that you can share with me - would be really appreciated. – Eugene Buitendag Jul 8 '15 at 4:08
  • Seconding the automation approach here. That's how I've dealt with similar problems in the past. – Kate Paulk Jul 8 '15 at 11:40
0

An automated test suite for the primary product before it gets customized is a good way to go.

You could go a step further and automate the creation and testing of the bespoke components too. If the group that handles the bespoke part is also your testing team then simplifies the politics. If your automation scripts are checked into source control then you can always go back and recreate a particular bespoke component. Knowing what was done is so valuable when testing and troubleshooting. And! There's great satisfaction in saying, "I ran the script the way I always do. If there are problems, it wasn't me."

You didn't state if the bespoke parts are compiled differences or configuration differences. Either way, scripting the compiling, composition, and testing of all components will make your life much easier.

  • Any idea how a test strategy document for this will look like - any examples maybe? – Eugene Buitendag Jul 9 '15 at 5:19
  • I trust you have test strategy documents that describe how you test the uncustomized version. The bespoke components would be additional items on your current strategy document. How to setup a composition and testing workflow is too long for a comment or any one question. This book was incredibly helpful to me: amazon.com/… – Green Jul 9 '15 at 11:52
  • Does that answer your question? – Green Jul 9 '15 at 11:53
0

One of the approaches that can be used to aid in automated testing of all the client specific builds is to adjust the design to simplify both test and the production of client specific builds.

  1. Influence the design such that all the clients have the same code, with all the functionality present, but with which functionality is available to the specific client being determined by one, or more, of licence file options, licence server replies, registry entries, configuration file contents. You can then have a test suite that tests all of the product functionality with all of the functionality enabled and a client specific test that simply has to confirm which features are available and unavailable.
  2. Another option is to have the base product structured into core items and customer specific items and a configuration tool that reads the client specific details and outputs them to both a set of compile time options and a set of test options/limits. Then your test suite will consist of the core functionality tests plus those tests enabled/tailored for the client.

Either, or both, of these options will probably require some restructuring and rework of the product code base but in addition to improved test coverage and reduced test time, both well worth the investment by themselves, they will also result in the following improvements for the coding department:

  • Some clear delineation between core code and client specific code
    • this will speed up the generation of client specific code,
    • simplify source code version control,
    • possibly allow some division of responsibility between programmers,
    • ease the roll out of bug fixes, etc., e.g. an important change in the core code section should possibly result in notifications or updates for all customers major changes in customer specific code should not but can you currently easily tell which is which?
  • Either approach will tend to make the generation of the customer specific build, and the associated tests, more of a configuration exercise than a programming one possibly allowing the support or project staff to do it freeing up your talented programmers for more productive work and speeding the response to customer specific requests.
  • It will often allow the identification of the code that should possibly be refactored into generic functions rather than a multitude of customer specific functions. This in turn will often allow testing and bug fixing before the customer has made up their minds about something. e.g. If alarm sounds and duration vary between customers then a generic alarm module can be specified, developed and tested that takes an input file name identifying one, or more, specific format of audio file plus reads a specific setting from a configuration source. This can then be adjusted, without code changes with just configuration changes at any time up to, and possibly beyond, customer delivery. Since you have, hopefully, already tested all the valid inputs and the fall backs for erroneous inputs last minute customer changes only require testing that the customer is happy with the settings that they have selected.

As both a software engineer and a tester I would always rather work a little harder and smarter once, then be able to move on to something else, than use a quick hack over and over again and possibly risk forgetting for one, or more, customers/occasions.

  • Steve, thank you for your comprehensive and very well thought out answer - I really appreciate it. Do you maybe, as a tester, have a Test Strategy/Test Approach document that incorporates the above? – Eugene Buitendag Jul 9 '15 at 10:28
  • Unfortunately such an approach really belongs in a system/project design or, better yet, department/company development strategy document - these tend to be proprietary but with a little searching you should be able to come up with some academic papers. There was a lot a few years ago about design for test and test team involvement from the first design meeting. – Steve Barnes Jul 9 '15 at 11:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.