Should Testers first discuss a bug with the developers before writing a bug report?

Question

In my previous project, whenever the QA team used to find bugs/defects, they used to communicate the problem/issue to the developer sitting next to them, verbally. In this approach, most of the time, the developer used to refuse to acknowledge the bug and used to tell us not to report it.

Or whenever a tester finds a bug and is sure that it is a real issue, should he log the defect into the defect tracking tool and then write an email to the developer to fix it?

Which is a better communication approach for testers?

Answer 1

As a developer with 15+ years experience, I'd say: please report the bug. I'd much rather have a ticket in the tracking system than a vague memory of a hallway conversation, or an interruption of what I'm working on. As Joel Spolsky put it:

At any given time, I can only remember two bugs. If you ask me to remember three, one of them will fall on the floor and get swept under the bed with the dust bunnies, who will eat it. …

One of the biggest incorrect facts that programmers consistently seem to believe is that they can remember all their bugs or keep them on post-it notes.

(The flip side of this is that bug tickets need to be good. (Spolsky's post also has good starter advice on that.) Vague, irreproducible bug reports are a sure way to build a bad relationship between QA and development.)

Answer 2

In my own experience, I've found that this really depends on the culture of the team or company that you're working with.

On a team with good communication skills, and where everyone works together well, you may very well be able to just tell the developer about it, they take a look at it, and say it'll be fixed in 5 minutes. If it's longer, maybe they'll ask you to send an email. Myself, if it's created during our current sprint, I'm not horribly worried about logging it into the defect tracker. If it's currently in production and being fixed, always log it though.

On other teams, and if the relationship just isn't quite there yet, or even if the culture is just different, this can be very different. On my current team, most of the developers won't even look at it until it's been logged and approved to be worked on.

Long story short, it's probably a good idea to always log it, but, attempting to communicate with the developer and explain it better rarely hurts.

Answer 3

We prefer to talk to developers first. Then, we together decide if detected error is a result of some quirk in processing and not even worth of bug, or, if it is worth a bug, we negotiate who will enter it to Bugzilla (we call if "buggify the feedback").

Often, developers prefer to write the bug description, because that way they can better describe the problem and what needs to be done to fix it. If non-developer enters the bug, sometimes write-up is confusing and developer need to clear up the misunderstanding anyway.

Because all code changes need own bug number, we do not allow changes without associated bug. So error report can be fixed without adding new bug only if error was caught in test environment (QA) and it relates to a bug from current iteration which is currently being tested in QA (bug status reverts from "TESTING" to "DEVELOPMENT"). Even a minor change to PROD code needs to have a bug number and is prioritized, so unfortunately, even trivial change might take more than 5 minutes (unless it is a change related to current iteration).

All code changes are also reviewed, which also slows down the fix (but we believe improves quality).

On very rare occasions, we make quick changes to PROD for very urgent fix, but then we add the bug and fill in the process later.

Answer 4

Do not discuss defects immediately with developers. Mainly because context switching is bad for their focus and could take up to 5-30 minutes to recover from.

How to communicate a defect depends in which stage of development the defect is found.

When I find a defects during testing of a task of the current iteration then I document the defect in the feature and move the feature back into the Todo phase. I communicate this action daily during the stand-up meeting.

Unless this defect is not caused by development in our current iteration. If it is an older defect I report into our bug-tracker and get it placed on top of the backlog. I prefer a zero-bug policy. The product owner should decide if we fix this defect or not. The details of this defect are discussed during the planning session.

Even if you do not use the Scrum meetings I would suggest you find a good moment to discuss the defects with the development team. Do not bug your developers while they are working on something else. If you are sure its a defect, log it into the defect tracker. Unless its a priority one issue, then scream and all hell breaks loose. :)

Answer 5

I think, it is always good to discuss things in person. You are lucky you have that things going on at your work place. In many companies all communication happens only via a system and that at times creates lot of confusion. What you write and what the other person understands may be completely different.

For things like developers don't consider the bugs, I think you should start to improve on you skills to convince people!

Answer 6

As a corollary to what David said, testers have the same memory limitations that developers do. If you don't log the bug, chances are very good it won't get fixed. This is especially true if it's not a severe bug, as chances are very good that the developer(s) are working on fixing severe problems first, and won't fix less severe ones for a while. And by documenting the bug, you're saving the next tester who finds it time, as they won't have to document the bug themselves.

With that said, after opening the defect, there's nothing wrong with then talking to the developer to get/give more information, and possibly putting that into the defect report as well.

Also, to be blunt about it, there's a bit of a CYA factor here as well: if you report a bug that doesn't get fixed, and gets released, things will probably go a lot easier for you if you can point to the bug report to document that it was found. Someone may have decided it wasn't worth fixing, or time constraints may have kicked in, or whatever, but it's far better from your point of view to be able to point to a bug report than to say, "Yeah, I saw that, but ..."

Answer 7

Should Testers first discuss the bug with the developers before reporting it?

From my experience in several different organizations I think the answer given the current question details is "it depends"

It will depend on general factors such as:

• size of company
• size of development group
• current deadlines
• location of developers
• industry
• development approach, e.g. Agile, Waterfall, etc

and on intangibles such as:

• personalities of the developers
• personalities of the testers
• management style of co-operation vs. competition
• how busy dev's are at a given point

and on specific factors about the bug being filed such as:

• severity
• type, e.g. design, layout, database, workflow, etc.
• effect on data integrity
• effect on revenue
• effect on user experience
• effect on branding

This factors all interact and at the end of the day, for every company you have to find a balance between, at one extreme, asking about every little (or large) thing noticed and at the other extreme, never asking any questions of the developers.

As a former developer and current QA, I've experienced several cases (as QA) where just showing and sharing a bug with a developer can lead to them asking additional questions, wanting to look at the console, the DOM, the database, etc. and both of us learning a lot more. There have been many times where this has lead to a bug not being filed. In other cases the bug is filed with additional info that will be useful for dev. They often just notice related information based on their knowledge of the application.

It is good to look at the larger picture - how can QA and dev and product all work together to have good open conversations and agreed-on approaches for these things. If you have good conversations about bugs and their classification (often one of the most contentious items) then discussions about design and features will often then be easier. This also leads to consideration of other factors such as doing retrospectives in Agile environments which provide the opportunity for such discussions. If you don't do Agile I'd try retrospectives anyway. They are good for any kind of team in my experience.

Answer 8

I imagine this would depend on the bug being raised. If it's a glaringly obvious bug (pages not loading, basic function not working, etc.) then I wouldn't waste a developer's time confirming it and do a write-up. However if I come across something that may be considered an edge case or I'm unsure of the intended functionality (mostly found during negative testing), I'll begin a dialogue with the developer working on that specific story and discuss the bug's validity.

This is going to vary on the product you are testing as well as the dynamic of the company, our QAs sit in the same area as the Developers so it makes communication easier.

Answer 9

It depends...

What is the overhead associated with reporting a problem? Some organization have big enterprisy processes, that requires lots of clicking, mandatory fields and multiple actors between the time a problem is open and the time everyone is done with it. You might also loose sight of important problems if they're drowning in a sea of minor (or untriaged) ones.

What is the overhead associated with not tracking a problem? More than one person might stumble on the same 'not a problem' problem, and ask the same questions. Tracking it can help - but only if people are actively and efficiently looking for duplicates. Which gets harder the more 'not a problem' problems you actually file.

How easy is it to get a second opinion (from a developer, or an expert)? If you're working remotely and communicating by mail, you might as well file something. If you're sitting next to the developer that's going to triage the problem, asking is fast and convenient for everyone.

How sure are you it's a bug? This one is pretty obvious, although timing comes into play as well - are you testing the finished product? Or did you stumble on something that's not quite baked yet and you're wasting time reporting problems that are just not implemented stories.

What are you reporting a bug for? If your performance is measured by the number of bugs filed (yes, some organization do that - please change job), then you want to file as many as you can. If you only care about stuff getting fixed, then not tracking minor - but real - bugs that you know from experience development won't have time to address is the right choice. Compliance and customer communications are factors as well.

Conclusion

I don't think there is a 'right' answer to that question - it depends what you're working on and what the development process looks like.

Personally - and as a developer - I like the agile manifesto ("working software over comprehensive documentation" and "individuals and interactions over processes and tools") - so 'reporting' can be verbal and tracking in a tool is to be avoided unless it's the most efficient way to get stuff done. But it's just an opinion.

Answer 10

If there is possibility to discuss the bug, it could be valuable, for example you will have direct information of the cause and who will be the assignee of the bug. But generally it is not mandatory. It could be done as well after the bug is reported.

Does not matter if you discuss the bug with developer or not, probably you will have to report the bug anyway (If this is a real bug).

Behaviour you described is immediatelism, and:

• to discuss the bug and not to report it could be done if they getting immediately fixed (for example in extremem programming methodology). If they are not fixed on the go, you have to report it, so the development will have a clear list of tasks which should be done.

• to discuss the bug could be done if you are not sure if this is bug in the code or bug cased by something else (testing environment, data, etc.). I can just imagine, that this could be caused by extremely unskilled testing team. Find a senior tester then and ask him not the developer.

• If this is a bug and it is practised in bigger team of testers/developers, it could be a bit harmful for development cycle, as some problems could be "hidden" and later forgotten. Do NOT do it! This goes directly against testers job. Do not let some bugs to stay HIDDEN!

• And finally, If this is a bug, this behaviour could be disturbing for the developer, which have to switch to different problem and this is just yet another one source of your future bugs. Do NOT do it! This goes directly against developers job. Better is to ask some senior tester.

Better you have to:

• Be skilled, responsible, autonomous and self-confident tester.
• Ask senior tester first.
• If you need, discuss the bug with developers/dev team lead and announce, that you will report it.
• If you need, discuss the bug with developers/dev team lead and ask for developer who should be assigned.
• If you need, discuss the bug with developers/dev team lead or developer and ask for more info you will put into the bug log.
• If the bug is not yet fixed, log the bug/issue anyway.

Answer 11

If the tester thinks that it is a defect, most likely it is a defect. The developer may be correct that the process is working as intended, in that case the defect is possibly in the documentation, user interface or testing plan.

It should be logged, even if is testing error. This will lead to better determining where documentation deficiencies exist.

Answer 12

Definitely make a ticket.

• If the bug is "don't fix", you can refer another tester or developer who finds the same bug to the existing one, and save the developer some time.
• If the bug is a quick 5 minute fix, it still takes 5 minutes to do, and you have to remember to properly re-test. If these little distractions add up and are not documented, things like this make both you and the developer feel like you've done nothing all day, which is bad for morale. Since nothing is documented, others might also feel you've done nothing all day, which is bad for performance reviews.
• If the bug turns out to be a not-so-quick-fix after all, that's even more of a reason to track it.

Making a ticket is only a problem if it adds a lot of overhead for either you or the developer, or the ticketing system is otherwise broken. In that case the solution is to acknowledge that your highest priority bug is your broken ticketing mechanism which needs to be fixed immediately. An example of a broken ticketing system is if the ticketing system is abused to penalize developers for bugs found.

The one exception to always raising a ticket is if the tester is still being trained in the business domain, and in some cases can't tell if a certain behavior is a bug or not. In that case they need to ask someone, preferably someone from the business side, but asking a developer works too. If it then turns out it's a bug, make a ticket.

Personally, my preference is if the tester tells me about a bug, but leads with the bug number of the already created defect. As a team lead that allowed me to quickly figure out if it's worth looking at at all, or to point them to the culprit, or to apologize for screwing up, or to tell them to retest after story XY, at which point the responsible code would be obliterated anyhow.

Personally, I don't see a point in sending an email about a bug. If you can put it in writing put it in the ticket, if not come over and talk.

Answer 13

It depends.

Some of the factors I've found impact whether or not I report something formally or not:

• Development Methodology - If I'm working in an agile environment I'm more likely to work informally with the developer at first. In a waterfall environment, I'm more likely to report the issue because the developer has probably moved on to something else, where in an agile environment I'm likely to be looking at code-in-progress.
• Team Location - If I'm in the same room as the developers, I'm more likely to work informally with the developers. If I'm in a separate location, I'm more likely to report informally.
• Is the problem with code in active development? - If I'm looking at something that's in active development and seeing a problem or have a question, I'm more likely to ask the developer, or send an email or instant message. If they can get me an answer or fix within a day, that's generally as far as it goes. If it's going to take them a while to get to the issue, I'll report it formally.
• How big is the problem? - This is very much a judgment call - even if you open the new module and get an access violation, it may not be something you need to report formally: there have been many times I've queried something like this, and had a response like "Oh, crap, I missed that, I'll get you a fixed build right away" or "Oh, yes, that part isn't ready yet. You'll need to use this work around." Neither of those cases justify a formal bug report.
• Who developed it? - Sadly, not all developers are reasonable. I've only had one problem child in my career - a developer who would build exactly what the requirements said, and anything that wasn't in there didn't happen, so if the conceptual sketches didn't include the number of pixels between fields, the fields would be built without spacing, and the report that the fields were unreadable as a result was "not a bug" - and for that developer only, everything was documented at an exhaustive level or it wouldn't get fixed. For others, I've often asked if they want something documented as a bug or not, and then done as they've asked.

Typically, if I'm not sure whether to report something formally, I'll email or message the developer, allowing them to respond when they're ready rather than break their flow. It will always be reported in some fashion, whether formally or informally.

Answer 14

My view as a developer is that even if you are unsure it is a bug then report it. The upshot of this is (some of this has been mentioned by others)

• If you are unsure if it is a bug then you do not have enough product knowledge to perform the testing. This could be for several reasons, including partial training, poor documentation and insufficiently detailed specifications.
• If you immediately interrupt the developer that is 15 minutes lost productivity due to context switching (not counting the conversation time).
• Bug reports can always be closed with a status of As Designed. That way there is a track of any conversation and who was involved. A related issue can be raised to improve documentation.
• If it is a bug then an issue number should be required when the fix is checked in.

Finally, and I don't think this has been covered yet, it is ALWAYS the testers responsibility to create the bug report. The report should ideally contain

• A description of the issue (this should be as an average user would perceive the issue).
• Detailed steps to reproduce the issue
• The outcome of these steps
• The expected outcome
• Maybe some screen shots
• A log file or debug output if this is built into the product
• Additional technical information if the tester has a strong idea of the cause. Note that there is a big difference between this and the description

Answer 15

If it is a clear and obvious breach against uncomplicated acceptance criteria, then report bug as soon as possible. Feel free to bypass the developer, BA or product owner.

Otherwise, do not report the bug unless you first talk to the author of acceptance criteria (eg: BA or product owner), and the developer. If you do not: it is possible that you will raise a bug that is actually a feature, or delay important conversations that will probably lead to changing of acceptance criteria, or worse a developer will pick up the 'not a bug' bug and 'fix it', wasting lots of everybody's time.

Answer 16

If my dev counterpart is available I try to talk with them first. That gives them the chance to request extra information that I might not have thought of.

If my dev counterpart tells me that that area is not ready for testing, I tell them I will enter a bug just to remind myself to come back and look at the area once they believe it is ready.

If I hear that that bug will not be fixed or is working as expected, then a bug report becomes a tool for support to reference should this become a call generator.

Answer 17

Since I work at a team that doesn't have specific requirements for projects, I always communicate it with the developers and showing them how I reproduced the issue. They will acknowledge it. Sometimes they would say, let me check on that.. Coming back, they would say, oh I have fixed that please test it. Before they can fix it, I have already documented the issue and assigned the bug to them. I would always say it is for documentation

Answer 18

Report the bug. A QA person's role is to be an independent verification authority. Someone who understands the products requirements and who is focused on quality. Make sure you document your problem scenario/reproduction steps so that your bug report stands up to scrutiny.

The problem with discussing bugs ahead of logging them is that it introduces human emotion into the process before the process has had a chance to begin. You then short-circuit all your metrics, and allow developers to game the system, or worse, emotionally abuse the QA personnel. This reduces both product quality and morale.

Why does so this happen so frequently? Because different developers take bug reports differently. Some take them in stride, others consider them to be a form of criticism or black mark, causing them shame and embarrassment. Of course this is ridiculous, but people are human, and come from all kinds of backgrounds. If this is a problem in your organization, it needs to be addressed. But the bottom line is -- please encourage developers and testers to follow the process. There's plenty of time to discuss issues after the bug has been reported.

Answer 19

To stay simple, and echo what others have said- (20 years experience here)...

1. Find the steps to reproduce
2. Take your screen shots or grab logs, etc.
3. Find the root cause if you can
4. Report the bug. Get it in the system. Perhaps do some limited searching to make sure it isn't a duplicate.
5. speak to the developer at this point or when first able
   
   <-fin->
   :)

Answer 20

The PM, the designer, the QA, the developer and other stakeholders; each of them have different perspectives and expectations from the product, even when working with a perfect analysis. An objective look really matters. So, sometimes, especially when I am about to release a new feature or unsure of the requirements of my task, I ask for QA's time and ask him to spend a few minutes with it. He can spot simple mistakes that I am not aware of. The developer is not interrupted, some bugs are early diagnosed and fixed, the QA now needs less time for tests in the future; even a bug-fix release might be avoided this way.

But the standard and preferred approach is to report the bugs. A reported bug is official; it will be followed, fixed and verified. No one is not interrupted, no post-its or to-do lists, no remembering. The bug stays under record for the future.

Answer 21

I think you should absolutely discuss it with the developer. I work in an environment where we have a mainframe connected via MQ and other middleware to various front ends (web apps, POS machines, smartphone apps). Just because a scenario fails doesn't mean the code is bad. Maybe the middleware developer isn't passing the correct data to the front end. Maybe the XML between the hosts is garbage. Our new thing is offshore testers working overnight failing everything left and right in their scripts because they don't know our platform or how the apps should work. It's maddening. Speaking with a developer will help get it assigned to the right area, and identify what is failing and where. Assigning it to a random dev is a great way to pass the buck around and for everyone to say "not it" while nothing gets fixed.

Answer 22

Just repeating some of what has already been said, but as another voice with 15+ years of combined QA & Dev experience, I would agree mostly with those who say that it all depends on what works best for your company/teams. However that said, I'm a fan of open dialog between Dev & QA, and I personally think teams that have that in place will fare better in the overall shared ownership of quality. In my experience, talking to the developer first when possible is a good way to basically help confirm that what you're experiencing is a bug (some simple examples might be- a host is down for an expected reason you didn't know about- or maybe something else you were doing you thought was unrelated had an effect on your test), and also may help you get more info and narrow the repro description. In general just- dialog could help you both understand the issue more.

Now, that does NOT mean 'if dev says you had a misunderstanding of requirements' he is right and you are wrong. You seem to be concerned about whether or not you should talk to Dev based on the fact that they might disagree with you. I think that is completely separate from whether or not you should bring an issue up with them first. You are not Dev's underling- you are their peer. You WILL have disagreements...regularly- whether you bring it up with them first or not. You will have people who take things personally, who take combative approaches instead of working together, etc (on both dev and QA side)... it's just part of your job as a good QA to deal with those situations as best you can. Sometimes that means trying to find common ground with the dev, if that's not working you can take other approaches (build consensus with your team/manager first maybe- try the same with other members of the dev team/dev team manager, etc). Bottom line is- if you feel strongly that what you are seeing is a bug and needs attention (or at least to be documented and followed up on), and you can make your case for it- then file it and stand behind it.

Lastly- I would just say, sometimes it's just a common sense call. Sometimes you might just know devs are busy and shouldn't be bothered based on the severity of what you're looking at- and you just file it. Or you know it's an issue and just file it. There's really no right or wrong here; if your teams don't have anything defined, you can try a little of both and see what gets you the best results.

Answer 23

Yes, A developer would be able to advise how to do further testing. The important thing is that it is before you write the report. Not instead. The developer is not responsible for deciding if a bug has been removed.

Answer 24

No. Testers should not discuss with developers because they will convince testers else they can fix that bug before we filing in bug tracker. If developers do like that then tester can't explore in organization whether its small/big its not a matter.

If the bug is valid one then it should file in bug tracking tool.

Answer 25

Best practice is first report the bug in defect tracking tool.

At the time of "Bug review" meeting with developer,discuss about the raised bug is valid or not, set severity & priority is proper or not, what the details developer wants to reproduce etc..

Also Defect tracker tool provide the comments section where developer put their comments while changing the status of the bug as rejected, deferred, duplicate etc.

Answer 26

If possible, do not think that you are a tester, first think you are a team member who is working hard to make things right. The frequent communication between development and testing teams helps generating more ideas on each side. Developer can suggest about how to test a particular module better and at the same time tester can show how to correct the defect. Open yourself for new suggestions and sharing ideas.

A testing team isolated from all the other development team, cannot be productive. When a tester adjusts himself/herself among the developers and develops a mutual relation, a good environment of team created and when all the developers and testers work together, it’s a win-win situation for both parties.

Prefer agile methodologies, work together, do pair testing, work with developers, discuss and meet frequently, document less, give equal importance and respect to everyone’s work.