6

I'm writing this as a developer hoping to get feedback from the QA community.

We've got a client facing web application, for a very small development team. Assume we have a product manager and developer, with the Product Manager handling most client interaction, and the developer fixing most issues. There is no dedicated QA team.

Recently we had a couple client reported bugs (via email), that included a screenshot representing the problem in the web app. It was clear that an error alert occurred, on what page, etc. But it was not clear the exact steps how it occurred.

As a developer, I've been unable to reproduce the bug, after several attempts, each trying as many possible variations of the section in question, and having them all work as expected. It works in every case I've tried. The difficulty is that all the bug report includes is the screenshot, and the Product Manager is requesting it fixed obviously.

My question is, how should I as a developer handle the situation where a bug is "not reproducible" after a detailed investigation, specifically when the Product Manager expects it fixed? Obviously it should be fixed, but who's responsibility should it be to find the reproducible steps when it isn't specified and can't be determined? I've asked the PM to help find the steps, and he responded that he wouldn't be able to do any more than I to find it, so would be not helpful to try.

At what point is it considered acceptable to stop the bug fix investigation when the Client Request obviously needs to be handled somehow?

Improve this question
5
  • Can you go onsite to visit the customer? Can you VPN into his machine? How valuable is this customer - what is the impact of the bug not being fixed?
    – Phil Kirkham
    Apr 28, 2014 at 16:09
  • As a developer, I have no interaction with the client. The PM generally has email interaction, but seems to feel it unprofessional to query for more info (admitting we can't fix the bug on our own), when it should "just work" for them. Generally, as a small business, all client requests are considered high priority and immediate focus.
    – Miro
    Apr 28, 2014 at 16:11
  • 2
    It may also be one of those "works on my machine" defects where it will only occur due to some interaction on the Customer's machine. You may also tell the PM that while you have tried to duplicate it you can't, can keep it open for checking but if the Customer has had it happen again or has any further information that would help you immensely and get a fix in place sooner. You need more than a screenshot for a defect, you may need to educate your PM about that
    – MichaelF
    Apr 28, 2014 at 18:23
  • It is nothing more than a business cost/reward/risk decision. Ask the PM how much he is prepared to spend attempting to fix it. When its spent, tell him if its fixed or not. Your job is to spend the money (time) as wisely as possible. Hope he does not say its an open cheque book....
    – mattnz
    May 14, 2014 at 3:09
  • When you say "each trying as many possible variations of the section in question," do you mean you tried running unit tests? As I understand, unit tests are kind of built for you to see what happens when you pass a certain parameter to a method.
    – moonman239
    Nov 6, 2015 at 20:35

6 Answers 6

Reset to default
7

First, it is absolutely professional to ask for more information when you've made a number of attempts to reproduce the problem and haven't been able to do so. You've effectively eliminated the most obvious potential problems with your attempts to reproduce, so the cause is something less obvious: it could be the customer using a different configuration, it could be browser-specific, it could be related to the interaction of browser and operating system, a compatibility issue with something the customer is running on their machine... there are many potential causes.

For something like this, I'd ask your Product Manager to contact the customer with a request for more information, and at the same time (if possible) offer to add more logging to the application so that the next time the problem occurs, the system logs will give you more detail on what happened.

Improve this answer
2
  • 1
    Right on. I've seen this happen many times - and often a 15-minute webex/screenshare session with the person who originally reported the problem is all that it takes. If that is still unsuccessful, take some action - like additional logging or asking the originator to duplicate in your dev environment (if that is possible).
    – jruberto
    Apr 29, 2014 at 23:31
  • 1
    Good answer. In addition to the Product Manager, you can often enlist Customer Service in getting more information from the customer who reported the problem. Joining in on a phone call or webex might be in order, too. And perhaps this instance will help in your justification for a professional QA Team ;-)
    – Joe Strazzere
    May 1, 2014 at 1:39
2

This situation does happen occasionally with our application. What we do is to capture as much information as we currently know about the issue and to continue to collect information until we have a reproducible set of steps. It helps if the issue is duplicated at more than one site since we tend to get a little more insight that way. The issue remains an open issue until we have enough information to identify a solution. Also, be sure to document the details of the attempts that you have made to duplicate the problem. Considerations: permissions differences between the customer's environment and yours, data corruption, timing differences, use error.

Improve this answer
2

At my company around 150 Full time employees with 15 developers on staff, and a support team who receives all the inbound calls, if the bug is not easily reproducible, it is researched by a tier 3 support team. If they are unable to reproduce the issue, they will normally check with other system experts such as the QA team. If we are still unable to reproduce we close out the ticket is un-reproducable. If it is a high or critical priority issue our support team will try and put logging in place around the issue. Very rarely do they get a developer involved, of course it sounds like our company has a lot more resources to handle these types of scenarios, but as a developer it is kind of hard for your to fix an issue that you can't reproduce. If it were me, I would push to try and get on a call with the customer get additional details, in an attempt to make the system better.

Improve this answer
0

I've asked the PM to help find the steps, and he responded that he wouldn't be able to do any more than I to find it, so would be not helpful to try.

You are in an unfortunate, but fairly clear situation.

You do not have enough information to reproduce the bug.

You need more information and steps to reproduce from the PM and or the customer(s). If political or organizational issues are screwing this up, that is unfortunate (and very common). But it doesn't change the facts. You need more information. Keep the message the same and keep calm. Talk to your PM and be clear - you cannot fix a bug you cannot reproduce and you cannot currently reproduce. Sound like a 'broken record' and stick your ground. Reject "well, do your best" messages politely with "I've done my best and I can't reproduce it". then say nothing and allow for an awkward silence.

Improve this answer
0

I had been in number of such cases, in some cases I was able to use reverse engineering techniques and try to replicate them in house and then ask client for confirmation if my inhouse environment matches with their environment, if yes provide them temp workaround or fixes. Some times temp workaround helps.

If you have a page and alert screenshot, and you have access to code base, then probably you can try combinations of test cases to mimic it.

In most of the the cases, i have seen, its environment which is to be blame, I have done some work for small business, and quite often their software is more screened compared to software from big and famous companies,some times its firewall which strips headers, sometimes they close connections too fast etc ,sometimes they use extremely legacy browsers etc etc.Get the wireshark trace or fiddler trace or maybe browser dev console trace will help you a lot.

My suggestion is from the screenshot/data, go the code where it generates those alerts,and see under conditions those alerts can be generated in house etc etc and then provide them some temp workaround till you get some fix.

Improve this answer
0

Perhaps an alternate way to replicate the said issue would be to request the client to share his / her Application User id and assuming you already have the timestamp of when the issue occurred from the screenshot shared by the client, go into the application / middleware logs and identify the issue from the logs (based on shared user id and time of occurence) and backtrack in the application logs to identify the sequence of events, n keep backtracking till you hit a different function. Granted its a somewhat tedious process, but until you check the logs you'll never know if its an issue with the application code or an issue with the test data used by the client, or something totally different such as a browser/pc/application/environment parameter or setup issue.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.