Issue Tracking Spreadsheet... anything wrong with this?

Question

So we are a small company, and we have a lot of communication between teams.

That being said I am actually the first official QA of the group, and will be starting the QA group and leading the way for it.

We don't have a budget for tons of software so I've been making due with Google Sheets. And so far it seems to be working for triage purposes. We will start putting in issues that are still left over (plus new ones), after moving to production, into Jira.

Does this makes sense? I feel weird using just a spreadsheet for this, but it works for us as it's easy to share and is free (which sure doesn't hurt).

Right now I've been laying out my test charters in the first page (don't know if that makes sense, I'm new to the exploratory testing sort of way but it seems to make the most sense for me being the only QA. Large test cases I don't really have "time" for) and then adding subsequent sheets for each "Round" of testing, and anything that wasn't fixed gets moved to the next sheet. Also we use Color Highlighting during the "round of testing" to identify "Fixed Issues - Green" "Ready to Retest - Blue" "New issues - White" and "Failed Retest- Red".

Does anyone else use excel sheets a lot for testing/issues? Does it work for you?

Edit: Side note: For columns right now I have:

Issue #/Bug Type (IE Functional/Error Handling/UI/Steps to Reproduce/Expected/Actual/Severity)

Is there anything else that may be helpful?

Answer 1

Niels van Reijmersdal and Dhiman already mentioned the most important point. I'd like to give a look into my experience as my situation was quite similar to yours now some years ago.

I'm on a small freetime project (a browser game) and in the past years we tried to build up at a QA too which is a bit more problematic with only users who do the "work" in their freetime and are not teached into testing / QA. We also tried several ways to handle our issues from sheets over different issue trackers (sheets [excle and google], self written php/mysql page, mantis bugtracker, redmine). In this post I will also havbe a look into JIRA (was used in my university bachelor project first) and HP QC/ALM (we use in the company I work for).

Sheets / self written small PHP/MySQL-"Tracker" The first things we tried was using sheets and a small webpage for tracking open issues. For the beginning it was okay just to collect the issues and have a list of it. As you we used several colors to identify the status of issues. The advantages are clear: No / Low cost and available to all without real work. But quite fast we moved away from this for the following reasons:

• Not much space for descriptions <-> Loosing track of the issues due heavy descriptions. Always moving the cells to show / hide the descriptions and so on or have several related documents to see the details
• No history of issues. That was the most important point why we moved away. We could'nt track the history of an issue
• Bad search to e.g. look if a bug already tracked
• No syncronisation (at the beginning, using excle sheets)
• No privilidges for different users to change status etc. (Everyone could edit anything)
• No work flow "restrictions" (Even if we had a defined workflow you could change the status as you wanted and skip phases, e.g. by changing the fields to "Closed" and noone would notice.

So I would advise you too use an issue tracker instead of sheets.

Issue trackers We then moved to mantis bugtracker. It was a huge step for us to have a good look at the tickets, privilidges to seperate at least some actions to different users. We could define a small work flow for issue status. But after some time one of us got in contact with redmine and two people (one QA at his work, one learned software engineer) tried it in a test instance. After only a few hours we know that we want to change because we could define the workflow, user groups and priviledges very detailed. We could define own fields, integrate version control, had a better handling (e.g. changig the status of multiple tickets at once) and much more. As there was also an import plugin to import from mantis to redmine we could take over our tracked issues. Okay we got some trouble because we moved from ISO to UTF8 and the status could not be exactly transmuted but it was quite good and saved us to import the issues manually.

The comparison between JIRA and redmine is like two people arguing about the car they want to drive. The one wants a green car type A while the other one wants a blue car type B. What I want to say this this: The difference between them isn't huge. But for my personal I like redmine more as I think keeping track of issues is much easier and the administration and workflow definitons too - or short I think redmine is more powerful. I got in contact with JIRA at my university bachelor project and we had the problem that I was the 11th person in our team. But JIRA had only 10 free license at this point (some years ago, maybe changed). So I showed the team manager and one of the JIRA administrator our redmine project and hold a presentation what they could to with redmine - there were some points they had problems with in JIRA which they could do in redmine easily but I don't remember them, sorry. After the presentation the admin installed redmine, moved the issues from JIRA to it and we used redmine from them on and could save the money for further licenses.

As already said there are no huge differeces but I think redmine is more flexible and it's free of charge regardless how many users you have. You wrote in your question that you already have JIRA so you I assume you have a server for it. Then you would also have a server for redmine and no need to spend money for it. I don't know how large your team is but if it will grow you may get in trouble with JIRA licenses.

If you would be open source (what I don't think as you spoke about company) github with the build in issue tracker would be a free alternative too, but it is not so powerfull like JIRA or redmine.

A side note on HP QC/ALM. As it's quite expensive it won't come into focus at all but in my experience it can only plays its strenght if you use all aspects (requirements, issue tracking, testing) of it. Just for the issue tracking I really don't like it because of the handling. I don't think this would be a viable investment for you.

Conclusion Yeah this answer sounds like a payed commercial for redmine but it's just a short summary of my experience keeping issues tracked in small teams with no or limited money. If you are happy with JIRA use it. But if I understand your comments right you are in testing phase at this moment. May you could give redmine another try to have a look on which tool is better suited for you - maybe you don't see a difference at the beginning / all but I think this comes with the detailed requirement of your team to the tool. Would probably to broad for this here, e.g. on my freetime game we don't use the "expected time effort" tracking of redmine but in my university we had to.

But I would emphasize the posts of Niels van Reijmersdal and Dhiman that you should use a tracking tool instead of spreadsheets.

Answer 2

Working with spreadsheets is not bad and strange, it all depends upon the scope and type of tracking you want to do, if you have a limited scope of test cases, issues and requirements, then you can stick to the excel sheets but in long run you will need some tracker which will make your work a lot easier and will itself perform some of your tasks which do in excel.

With the use of tracker you can create customize reports, track issues to closure, send automated mails to Assigners, Reporters and other stake holders. The most important benefits which you will get with any such trackers are:

1. No one is allowed to bypass the process, only an Assignee can close a task or a coding task must be first reviewed by its reviewer and than it can be sent for testing. This you can achieve with custom workflows.
2. You can link your issues and tasks for creating a better grouping and hierarchy, so that you have a clear view that which development task has how many Coding Defects, Bugs and Requirement Issues linked to it.
3. With the thoughtful use of tracker, you even don't need to maintain a separate sheet for the RTM, same can be done over the tracker and it will again save your time and effort from maintaining 1 more document.
4. Having a history and comments section, which is not that easy to see and maintain in excel sheets. No need to worry about the version of tasks and issues.
5. Track down the estimated effort and the effort being spent over a task, as such calculations are done by trackers automatically
6. More than that it will give you an actual feel that you are working in a processed manner and task is moving from start to complete phase by its own (this may sound little non-technical but this too is important factor). Once you use a tracker you will know it's features and importance in your day-to-day work.
7. Working with different SDLC models and adapting your issues according to that model with in no time.
8. All the team members will see the same data (although it can be controlled through security configuration) and have their own access, thus no need to maintain separate sheets for different group of peoples.

All these features are well provided by JIRA and its a good tool to use and work.

At the end, there is no tool now-a-days that doesn't supports MS Excel, so you can always Export and Import issues from the tracker into an excel sheet and then create your own charts and reports (Pivot table) easily.

Answer 3

Possible issues with using spreadsheets are that:

• you are dependent on an external company for software for access to your historic tickets.
• the only workflows are manual - no enforcement
• you will eventually run out of space or the sheet will get too big to handle
• difficult to add attachments such as screenshots of the problem, etc.

I would suggest setting up an instance of trac issue tracking software on a server within your company.

• Free
• Open Source
• Web interface - no client software to install
• Version control integration for:
  • svn
  • git
  • hg
  • bazaar, etc.
• Automatically email the originator on changes to their ticket
• Automatically email team member on ticket being assigned to them
• Querying/filtering the data & reports exportable as .csv & .tsv or as an RSS feed
• Custom roles and workflows
• Timeline & Roadmap features

Answer 4

One team, one system. Do not create separate spreadsheets, be transparent instead. I would Keep track of your whole project in a single tool.

Jira is build to be defect/issue tracker. This is what it was original designed for. Just use it! :) Before development for requirements, during development for defects and communication, after development for maintenance requests.

If you want to separate pre-production defects from productions defects use the version field. Add some ALPHA/BETA versions or maybe a 0.1 version codes.

Jira can also prioritize issues and you can also add non defect/development tasks into. You can also build custom workflows in Jira for your own new ticket types, this way you can make any workflow you see fit, with unlimited custom fields. It should be able to handle anything.

Answer 5

We have a smallish engineering team. There is an in-house issue tracker built many years ago that we use for customer and internal bugs; however, for testing of features, I use excel. The excel spreadsheet is uploaded to sharepoint. I log the issues with a hyperlink to my documentation (steps to reproduce). The developer reviews the issues and documentation and updates the spreadsheet with fixed or any questions they have. Not very 21st century, but as you know, it all goes back to the team, the environment and what works best for both at that time.

Answer 6

We use both. We use (highly customized) Bugzilla to track/triage bugs and feature requests, and excel spreadsheet to track items related to upcoming release. Some bugs (all but the simplest changes) might have multiple entries in the spreadsheet, and sometimes bug is split into two phases - and few items are deferred for next release (and if so, new bug as "phase 2 of bug 1234" are created).

Bugzilla allows us to keep the history of the changes as bug goes through the workflow process, including audit many months later.

Spreadsheet allows us to track more minute details which will be all (most) implemented within 2 weeks so are in fresh memory and there is no need of preserving change history (they are just flipped from requested to implemented, according to priorities). It also allows us to see overall picture how the release is shaping up across all bugs/change requests included in the release (and with all the minute details which is not worth entering into bugzilla).

If we would start now, we would probably used Trac, because (beyond what Steve Barnes mentioned in his good answer) it includes also wiki for technical documentation, integrated with bug database, and changeset viewer in the browser. Wiki documentation is a killer feature IMHO (I am big on docs), but we started with a separate wiki before Trac (our project is rather old as you can see) and there is no will to redo the customizations from bugzilla in Trac, even if everyone agrees it would be cool.

Jira becomes quite expensive quickly, especially if you want to use some more expensive plugins for bigger teams.

Answer 7

Spreadsheets are still relevant for test management. Previous concerns regarding the use of spreadsheets on separate machines like lack of shared access for regular updation and version control have been satisfactorily addressed by the use of cloud-based spreadsheets. All the team members can access the spreadsheets and still, you can control access to different sheets for different members. You can easily attach not only a screenshot, but also insert video links for issue reporting. Of course, it is important to maintain the critical path within the regression test suite so that there is no duplication of effort.JIRA is an effective bug tracker which can be used with spreadsheets. I am a technical writer for OnPath testing and I have written this article on how to manage your regression test suite with spreadsheets.