Advice when putting together a cost benefit analysis when considering changing defect tracking systems

Question

we use a homebaked defect tracking system in my org. As a QA I have worked in various different organisations previously and find the current in-house solution to be sub par. I initially started to raise feature requests, but at this stage the list is so long that the cost of developing them would equal the cost of buying an OTS solution. Add to that the fact that we are behind on that long queue of requests - and even more doubtful we should be investing our time/money.

IMO - We develop our own specialist product - dev'ing defect tracking systems is not our core function and is not something we should be wasting our time on.

I've been told that if I'd like to see us move to an off the shelf solution I need to come up with a cost benefit analysis to prove the value.

I have my own set of features and cost savings which I think we would benefit from in moving.

What I'd really appreciate is input from people who've been through similar situations, moving from inhouse to OTS defect management systems.

so here is the short question - How did you convince your team that moving to an OTS was the best solution for the company in the long run?

Thanks!

Answer 1

I had been in similar situation. Since, I worked on different test case management tool, I was able to provide drawbacks / missing featured when I worked on Inhouse test case management tool.

First you need to understand the drawbacks of current tool. Second important thing to look at is migration aspect if you finalize on your new tool.

I did a similar exercise of Test Case Management Tools. Couple of points also apply for bug tracking tool as well. The criteria for analysis is based on

• Does the tool provide Web UI interface to update/manage test cases and Bugs. This would be better than using a client app to connect to server.

• Tracebility over bugs logged. Does it provide option to link bugs with test cases which have failed

• If you are using a different product for test case management then you need to look at how you are going to integrate it with bug tracking tool

• Reporting / metrics from bugs logged. Type of bugs (code/design/deployment). How easy it is to generate reports. What are inbuilt reports provided with the tool

• Cost of the tool, usability aspects in terms of (adding images/screenshots/copy-paste) etc..

• Effort required to develop the tool / maintain / support the tool. You have lot tools which are relatively cheaper and you can quick migrate from your current system

• You can start looking at free tools, tools which are relatively low cost. You might have a cost problem if you start looking at HP, Microsoft tools.

• The missing drawbacks in your current tool, you can provide estimate on effort spent in tracking them manually or deriving metrics out of it manually. You can propose how these efforts can be saved in new tool

• I have also seen cases where management is aware of drawbacks in tool but they were not interested in moving to new tool. Reason was it was not a priority as per organization goals. It also depends on maturity of leadership within the team to improve team productivity with better tools and practicesl

Found one more tool related to this discussion. BugTracker.NET is a free, open-source, web-based bug tracker - http://ifdefined.com/bugtrackernet.html

Answer 2

A big part of selling them on switching systems is knowing your facts and being prepared.

You can start with a feature list comparison. Vendors will publish their own feature comparisons, but theirs will be biased toward their own product, emphasizing their own perceived advantages and de-emphasizing or refuting their weaknesses. Do your own feature comparison based on what you think you need, not on what the vendor wants you to think is important.

If your existing system has been around for a while, it may be integrated with other systems. You should evaluate whether those integrations are still necessary, and if so, whether they can be accomplished with an off-the-shelf system. Of course you might also consider integrations that are possible with the new system but not with the old system (in its current state).

Some organizations use multiple bug tracking systems for different audiences, e.g. one system for customer support and other for internal bug tracking. If your current system is used for multiple audiences, consider whether that is still the appropriate choice. In some circumstances, it may make sense to have different systems or multiple instances of the same system tailored to their specific audiences.

Finally, put together a transition plan. Include your own deployment/development effort, data migration effort, and time to produce training materials, and training classes. Switching bug tracking systems will require people to relearn user interfaces and workflows. Some people don't like change, and others don't like for you to inconvenience their busy lives, even if it's for the best in the long run. Showing them that you have a plan will help take the sting out of switching systems.