35

Whenever our developer fixes a bug, he or she just posts "Fixed in version x.x.x.x, please test".

We, the QA team, have requested developers to be as detailed as possible about what caused a problem and how he or she fixed it, but they still keep doing it (we don't want to think they are lazy or that this is not our business, of course). It does not always make sense for a developer to do it, but we think that knowing the strategy behind a fix, what caused it, can really be beneficial for the overall quality of the product and, we think, this is something that both teams have to contribute to.

This additional information, for instance, may help a tester to find similar issues in other parts of the application or can provide ideas for exploratory testing of a fix.

How can we convince developers to explain a bugfix in a more detailed way?

  • 8
    IOW, you already have the code. You have the ticket, which should have all of the relevant information that you gave the developer to fix. The ticket shows the observed behavior, steps to reproduce, and expected behavior. What more do you need? What more exists? What extra information is there? If you have all of the rationale behind the code change, and all the code that has been changed, what value is there in having an explanation of the code change? Either you can read the code and figure out the 'why' yourself or the 'why' doesn't matter to you, no? What am I missing? – Shane Sep 15 '17 at 18:17
  • 11
    You expected more, ok. But what exactly DID you expect? You want 'developers to be as detailed as possible about what caused a problem'? Ok, that's the steps to reproduce. You about as detailed as you can get about what is causing a problem. You also want 'developers to be as detailed as possible about how they fixed it'. That's the code. Literally cannot get more detailed than that. So what are you looking for? I'm only hammering away on this point because I'm betting the dev is having just as much difficulty understanding your request as I am. – Shane Sep 15 '17 at 18:42
  • 6
    @Shane for example, for that problem I've noted above it could be "This bug was caused by this opened Chrome issue that effects the table elements when a browser window is resized. I've applied a workaround suggested here.". This provides valuable information for testers especially in case of intermittent bugs or problems that have an unclear cause. Hope you can see my point. Thanks. – alecxe Sep 15 '17 at 18:57
  • 3
    @DanHenderson Those aren't the causes. Those are the actual fixes. – Shane Sep 15 '17 at 20:38
  • 3
    @Shane Well, no, for example with the cause I listed of "referenced constant instead of variable" that would be where the variable is what should have been referenced, so the fix would be "adjusted foo to reference variable". Likewise with my example cause of "started loop at index 1 instead of 0" I meant that in that hypothetical, 0 was the correct index to start at, so the fix would be "modified loop to start at index 0". But my point was that if the bug is "this link turns pink when I mouse over it" then the cause is not "mouse over this link", it's "wrong color set in hover style". – Dan Henderson Sep 15 '17 at 21:48

15 Answers 15

44

I would operate under the assumption that your developers are documenting this somewhere, since I would also assume that they are good developers who care about good development practices. As a developer myself, I strongly believe that developers should be expected to describe what they've done in a language other than code. It doesn't have to be comprehensive, and I believe they should, if they don't already, care to take that time and effort to ensure others understand what they changed without having to read the code. As the saying goes "if you can't explain it simply, you still don't understand it", and that's a dangerous place to be if you're touching code.

This may be as simple as writing a good commit message, or if there's a code review involved, providing a short summary of what was changed and why along with the code review. The code review artifact would then be a useful tool for testers to reference.

I would also be surprised if, for all but the most trivial bugs (e.g. "The label for field X is spelled wrong"), a good developer didn't put any information about root cause in the bug report after investigating and finding the bug, perhaps along with an explanation of the correct fix. (If there's a change control board involved, this is often necessary to justify fixing the bug in the first place--often it's worth describing several possible fixes based on the rough effort for each.)

If your product has written design documents, I would also expect that some bugs would require updates to the design, especially if something major had to change to eliminate a failure mode.

I would have to imagine they are capturing the desired information in one of the above (commit messages, code reviews, or design docs, though your question implies they are not putting it in the bug reports). It may be as simple as doing some spelunking of their other artifacts to find this information, or talking to the developers see where they're putting such information.

Personally, I would go to them under the assumption that they are putting this information somewhere other than their heads, and just ask where you can find it. If your developers truly aren't leaving any breadcrumbs about what they're changing other than the code itself, I believe they are poor developers. It's cowboy coding. In that case, then it seems some introductions to basic development practices are in order, but it will be hard to drive that unless they want to be good developers.

  • 7
    This! If you use tickets, either have the devs describe in natural language what was fixed and how OR agree that they make remarks if the fix could affect other parts/affects a certain generic sub-system, so QA can decide whether to re-test that whole affected system/part. It's not the job of QA to read your code, heck in many projects QA is external and doesn't and shouldn't have access to source code at all. – Frank Hopkins Sep 15 '17 at 15:20
  • 3
    Maybe I'm missing something, but the dev is already doing what you suggest here. Isn't the OP asking for more than a simple decent commit message? – Shane Sep 15 '17 at 18:19
  • 2
    So much this. There should be design documents, code comments, and so on, which are kept up to date. This helps QA, and it also helps with the bus problem. – Kevin McKenzie Sep 15 '17 at 19:01
  • 2
    Further- if this discussion about how the fix was done happens in the issue tracking system then if the issue comes back or isn't entirely resolved or possibly related issues arise then there's some documentation of the previous steps taken and why. Understanding why certain changes were made is quite possibly as important as understanding how the changes were made. – dash-tom-bang Sep 16 '17 at 2:41
  • 3
    @Renzeee As I compare the different answers it seems as though the other answers are saying "It's not a big deal to ask developers to do it" while this answer is saying "Why aren't these developers doing it regardless of how QA is involved?" A subtle difference, but when you're trying to convince someone to do something, showing how it's in their best interest as well is a powerful persuasion tactic. As to the offensive part, the (now removed) line was perhaps unnecessary but didn't seem to be offensive - it simply pointed out that this POV isn't reflected in them. – corsiKa Sep 18 '17 at 14:52
14

Request a meeting between upper management of QA and DEV and explain the goals of what you are asking. I think the description you provided in the question is a good start at explaining the why of the ask.

This additional information, for instance, may help a tester to find similar issues in other parts of the application or can provide ideas for exploratory testing of a fix.

QA wants to better understand the fixes for bugs so that we can:

  1. Better test the original issue
  2. Better discern and test the other areas that might be affected
  3. Better understand the systems in play
  4. Be more efficient in our testing

There's always two sides... what the dev intended to fix and what actually changed. Ideally those two things are the same but QA wouldn't have a job if it was. That's not to say that DEVs are bad or dumb. Many of the systems are very complicated and we're trying to move at a brisk pace. Mistakes are made and DEV and QA need to work together to increase the number of defects found and reduce the number of defects that escape.

Sometimes having to explain something to someone else makes you think of things you didn't think of before. Explaining it to QA educates them more about the various systems in play and how they interact which makes them better testers.

Overall, the goal is to better educate QA on the fix so they can test more effectively and efficiently and detect more bugs. I think everyone would get behind that. The problem, I think, comes with the phrasing of the ask.

to be as detailed as possible about what caused a problem and how he or she fixed it

I think in some/many minds that is asking for a lot of "extra work" to be put into each bug. I would ask for a brief description in the bug. If QA looks at the brief description and needs more info or has more questions, they contact the dev and get that info. I think that's a good compromise between too little and too much info in every bug.

If I were to phrase it, it would be something like

Please briefly explain:

  1. What the fix was
  2. What other areas were affected by the fix
  3. What other areas might be affected by the fix
  4. Any special guidance for testing? Are there any concerns over the fix? Special areas of focus for testing?

I don't think this is a big ask of DEV. I think it's something they work into their process. When they finish fixing a bug, they look back at the files they changed and summarize the fixes into answers to the 4 questions above.

I think after v1.0 of what gets put into each defect, you meet back a few months after it's implemented and review. Are DEVs doing this consistently? Is it too much work? How can we minimize the documentation DEVs need to enter vs how much info QA needs to do their job well.

I DON'T think the answer is to have QA look at the code changes themselves. You are then assuming that QA knows all aspects of these complicated systems that DEV works in daily and I don't think that's a reasonable ask.

  • 2
    That's a much nicer and more diplomatic way to put it, thanks, gonna try it! – alecxe Sep 15 '17 at 17:20
9

Most other answers suggest that requirement to be better in testing is understanding the code - but that requires skills what most QA testers do not possess. Deep understanding the code is not required. In most cases, code changes are way too complicated and obscure for QA to be able to glimpse anything relevant from just reading the code in detail.

What IS relevant for QA is to be able (but not required) to browse through the code changes not to understand the changes, but to see what parts of the system were affected by the changes (which webpages, libraries handling which functions, etc), which are good candidates for testing. This would be only as addition to the description of changes/test in bug ticket.

In my own experience, while I was able in few cases to read the code (as a former developer for our system) and build a test case to fail the fix and prove developer that fix is wrong, it is very rare situation.

Your developers should already use a tool for code review, so use it too (if you feel description of changes in ticket is not sufficient) - and if they are not, code review is the cheapest way to improve the quality of the code.

Not sure how it is in your organization, but in ours, every bug has a champion ("internal customer") who works with other experts to write how exactly to test the changes (in addition to unit tests added by developers). It is this "bug champion" responsibility to identify what and how should be tested (not QA's responsibility). This information is preserved and sometimes used in lieu of documentation to better understand system behavior in specific corner cases.

Remember QA is there not to "assure" the quality after the fact, but to maintain the consistent process and communication between all involved parties to design the quality into the product, and provide information about the status of the quality.

Responsibility of QA is to make sure that agreed-upon process was followed (code was reviewed by senior developers, unit tests were written and passed, regression test passed, developer and champion determined what should be tested), and requested test was performed as described. Task of QA is not to replace all that skills and knowledge what developers and problem are expert have. Task of QA is to make sure all relevant parties participated in the process, not to solve problems for those parties.

Quality cannot be bought, but has to be paid for.

  • "but that requires skills what most QA testers do not possess" it is a skill they should. How can you test software if you do not understand how it is build. Just seeing which classes have changed can already be a worth a lot. – Niels van Reijmersdal Sep 15 '17 at 13:32
  • 3
    @NielsvanReijmersdal - reading and understanding the code is a required skill for QA? Seriously? Our app is huge complicated website. Do you require QA to have developer-level skills in AngularJS, gulp, JS minification, Apache, load balancer, DB administration, ORM, not mentioning coding and managing dozens of services, cache, running on our more than 100 Linux servers? I know about that stuff only because I was a developer. There is no reason for all QA staff being expert in all this. We have one who is informed (not expert). Niels, do you have developer-level skills? – Peter M. - stands for Monica Sep 15 '17 at 13:42
  • 1
    Not sure if you read my answer, but I said exactly what you commented: that QA should be able to read the code enough to understand what parts were affected by changes, but understanding the details of the changes in not necessary. Yes, children can learn simple code, but I would not want average child to configure our apache server or database. :-) – Peter M. - stands for Monica Sep 15 '17 at 13:51
  • 1
    Your definition of QA is very different than any place I've ever worked or even heard of. QA in most companies is the team that actually does the testing of the software being built, not coordinating the testing or making sure process is followed. That's the job of QA leads/managers and probably the project manager. – JeffC Sep 16 '17 at 0:14
  • 1
    @JeffC - Ouf course QA does quite a lot of the testing, but not all of it. All depends how complicated is the tested app. Ours is rather complicated. Validating outputs in some parts requires 4 year degree which QA staff does not possess, so we rely on teams of in-house experts. They do have managers, but those managers manage their primary responsibilities (which are equally complicated) and defer to QA for QA-related stuff. Kinda matrix. – Peter M. - stands for Monica Sep 18 '17 at 14:37
7

It seems that this question has two answers depending on the qualities of your QA team.

If your QA team can read code, they can look at the commit(s) where the bugs are fixed - if you use a version control system. In this way your team can analyse the changes themselves, meaning you don't need any information or time from someone else.

If your QA team cannot read code (or at least only a minor percentage of the whole), you should keep asking the developers to provide more information so that you can spot the bugs earlier from now on. It will help if you keep giving good arguments for the developers to provide you with more information, as it could feel for them as wasting time because the bug is already fixed. Possible arguments are:

  • Improving regression testing. If you know a bug is caused by f.e. a for-loop not being executed for the last element, you will add more test cases to your regression test that cover that.
  • Improve automated testing. The same argumentation holds as above.
  • It will help you spot these bugs earlier the next them, resulting in the bugs being fixed sooner the next time, meaning the product can be released sooner.
  • You can perform "white-box testing" next to "black-box testing". If you know what parts of the code has changed, you can use different testing techniques that are more applicable. It will take more time (most of the times) if you use black-box testing techniques, because you don't know what parts of the codebase have been changed. If you use white-box testing techniques, you can test the more precisely at those parts of the codebase that have been changed.

Of course, it could still be that the developers are not convinced and they do not provide the extra information the next time. But trying to get such a rhythm into the process will hopefully result in them doing it upfront (before you need to ask about it) in the future.

  • This read the commit changes :) I always use them to check if similar bugs cluster. – Niels van Reijmersdal Sep 15 '17 at 12:56
  • 2
    @NielsvanReijmersdal some of our testers cannot read the code (in our case this is JS/AngularJS app under test) and reading commit messages can actually be beneficial in theory, but UI devs don't have a standard way for that as well - there are only ticket numbers and things like "fixed" and "improved" as well. Thanks. – alecxe Sep 15 '17 at 13:16
  • 1
    "If you are joining the company ... you're going to learn to code .. whether you are in sales, finance or operations .. you will know how to code," -- Jeff Immelt. Although I think it is handy if everyone can make their own Office macro's this might be slightly overrated. But if you work in a software department not knowing how to read code is a sin! You might not be great at it, but even small children can write and read code. If you do not understand what your co-workers are doing...wow! Also read: money.cnn.com/2016/08/04/technology/… – Niels van Reijmersdal Sep 15 '17 at 13:30
  • 6
    I have worked in places where testers do not have access to commit notes or source code. – Kate Paulk Sep 15 '17 at 15:23
  • @KatePaulk - sad, but nevertheless in such a place perhaps a plugin to the SCCS or the ticket system can copy the commit message to the ticket. – davidbak Sep 16 '17 at 0:29
6

I'm a developer, and I do add additional information in our bug/feature tracking system on completing an item whenever I think it might be useful. I target this mainly to testers, but much of the same information might also be useful to another developer who comes across some code I touched and wants to know more (after reading source comments) about why it is that way.

For some simple fixes, there really is not much more to say. (Problem description: "The foobar icon is green, but per requirements it should be cyan." Resolution: "Changed the default color for the foobar icon to cyan.") But for maybe around half of all Defect fixes, something from my analysis and/or fixes is worth noting. After all, I'm the one who invested the most time in understanding the original issue, deciding exactly how to fix it, and analyzing whether my changes might potentially affect anything else.

I don't expect a tester to be able to understand code, but even for someone who does read code, a brief English explanation of the bug and fix can be quite helpful. Sometimes in complex code, the sequence of events involved could run through many different functions and stored data, and yet have a single line fix. Looking at that single line (and any added comment), you might be able to see why the change is "correct", but it wouldn't always be obvious how that relates to the observed bug.

So here are some questions you might try posing, to be more exact about what you need than just asking for "more information":

What conditions cause the bug?

This is slightly different than "How can the bug be reproduced?". A bug report will normally include at least one way to reproduce a bug, and possibly a few scenarios that seem to cause the same problem. But quite often when I discover the cause of a problem, I'll realize it's actually more general than the reported scenario(s), and/or would only happen depending on something else not described in the report. For example, a bug report might note the issue is reproduced when you "Do actions A and then B to widget C", but there's actually a problem whenever you "Do any particular-class-of-action A' and then action B to any widget with property D, while at least one E is active".

Or even better, I might start with an issue that seemed to happen frequently but not reliably, and come up with a way to reproduce it consistently.

This is critical information for a tester to properly test the breadth of the change. For example, if I described the original bug as above, a tester might want to also verify that the problem is fixed for a sampling of actions A' and widgets with property D, and that the system continues to work correctly in the same scenarios except where no E is active.

You might rephrase this question, "Are there other ways of reproducing the bug?". A subcategory question is, "Were there any existing configuration settings that would have affected the bug?"

Was anything besides the observed bug fixed?

Of course entirely unrelated changes should not be attached to the same ticket. But maybe two different features were using a common function that had an error that manifested in different ways in each feature. Or maybe the fix involved refactoring some code in a way that solved more than one issue identified during the work.

The test team should definitely be made aware if any positive "side effect" like that was noted, so they can test that out as well.

Which features exercise any of the changed, added, or removed code?

This is the flip side of the previous question. Sometimes a code change has happy side effects, but there's almost always a risk any code change might break something else. Developers should make reasonable efforts at analyzing changes to try to determine what else might be incorrectly impacted, but might miss something, or simply conclude that the remaining risk is acceptable.

When a few other features specifically use some of the same changed code, it may be worth keeping an eye on any automated tests for those features, and/or running a few simple regression tests for those features.

At one extreme, the answer may be "modified a core library function, so effects could be seen anywhere". (And maybe the modification is "obviously" correct, but some other code accidentally relied on the old incorrect behavior...) It's probably not feasible to run all the manual regression tests you have documented and/or can think of when this sort of thing happens, but at least you'll be aware of it as a possible issue.

Are there any non-user tools relevant to the bug or fix?

We developers like to create or find ways to get all sorts of information about what the software is doing that are not meant to be visible to the end user. This includes things like log output, execution traces, and means of inspecting internal data live.

Many of these things would need to be configured or otherwise activated, or might even require an external program. But they're often relatively easy to learn to use, just because the main purpose is to make things easier for us.

Only user-visible behavior truly defines whether software works or not, but sometimes peeking at the software internals in these ways can help verify more exactly that a fix or feature is working as intended. In particular, maybe in a certain scenario you would expect some variation in the output, but there seems to be an undesired pattern in that output. Since it may be impossible to tell from one try if anything's wrong, it may be more useful to investigate the steps in computing the output.

As a bonus, knowing a few of these sorts of tools for your software system can come in handy when writing a new bug report or failing a fix or feature. If you include some data from such a tool that appears, in your best estimation, to be highly relevant to the actual observed bug, the developer assigned to investigating it might appreciate it.


Hopefully explaining exactly what sort of information would be useful to you and why will convince developers to start attempting to provide it.

If developers still resist changing, you might want to discuss these same reasons for wanting more information with a manager, so expectations on the process can be established. This might be more or less formal, depending on your team's size and style.

5

As a counterpoint to the many answers saying that testers can review the changes in source, I have worked in more than one place where testers have no access to source code. Their only source of information on what has changed is whatever information is in the ticket.

In this situation, developers absolutely must provide testers with information on what changes they made, and cover things like:

  • What configuration settings the changes impact or apply to. Many bugs only occur under specific configuration settings, but I've seen first pass fixes cause issues with different configurations due to the assumptions made by the code.
  • Any related parts of the application the developer is aware could be impacted. Experienced testers will have a general idea what modules in the software are interrelated, but there is no guarantee there will not be subtle dependencies the developer can see but the testers wouldn't know about.
  • Any related issues the developer may have fixed or refactored as part of the bug fix: if there is an ongoing refactoring effort, this is crucial for testers to know so they can check the other changes as well.

It's also much faster for a developer to add a set of notes to a ticket with this information than it is for the tester to dig through the code to work out what might have been affected, simply because the developer can make those notes while working on the item.

In my opinion it comes down to all parties respecting each others time and effort. The developers I've worked best with have been the ones who give me the information I need to effectively target my testing and then work with me when I find things that don't match my expectations.

4

You can't expect developers to repeat what they coded in natural language. It would take time and effort they don't care about. At best you will get two sentences which reveal hardly anything (usually - I have had cases where a one-line comment from a developer triggered a few alarm bells).

The good news is: there is a fairly easy manner to check fixes as a tester. (Of course, the fact that you're asking the question makes me somewhat fear you don't have this possibility). One caveat is that you must be able to read code.

Assuming that your team works with code versioning, every check-in by a developer (changesets in TFS, commits in GIT) gets a unique number. If the developer makes the link with the defect during check-in, you'll even see the link appear in the defect item as well. If that's not the case, you can always consult the commit history, where you will see check-ins related to fixes.

Once you know the number, you can simply open it and see the code changes that occurred. Looking at before/after comparisons in this way can really give you excellent insights as tester. It can also help you greatly in reducing and focusing testing effort, e.g. if you see a list of hard-coded values in code, you'll know if a certain value is missing before you even tried it via the GUI.

Another plus is that this approach takes no extra effort from developers. Of course, after looking at the changeset you can always go and ask a developer a more specific question which will get answered more readily than Tell me everything you did for this defect and how and why.

  • 4
    -1 for the premise that you can't expect developers to follow good practices. If they don't care about it you may not be able to make them, but that's different. – c32hedge Sep 15 '17 at 14:43
  • 2
    Reading the code to try and understand how it works is the exact opposite of what you want from an external QA team! Once you have read code, you have an idea of how things work that may or may not be true based on code interactions and assumptions. The only reason to have QA in the first place is to actually test what the code really does. – Paul Becotte Sep 15 '17 at 15:42
  • 1
    @c32hedge OP asked for developers to communicate root causes and fixing strategies. Commit comments are fairly common, but I have never seen an elaborate explanation of how and why just because a developer feels like typing it out. On the contrary, if you know the code context of what has changed and talk to a developer then, you're saving time for both of you. Also, as a tester, I usually don't believe developers so I want to see for myself. ;-p – FDM Sep 15 '17 at 18:58
  • 1
    You're assuming both that the QA folks have access to the source code, which they may well not, and understand the object under test very well, which they may well not. If development changed a module somewhere, for example, how do you know what depends on that module from the commit alone, unless the developer puts that information into the commit message? – Kevin McKenzie Sep 15 '17 at 18:58
  • 1
    You are making a lot of assumptions... some have already been addressed in other comments but one that hasn't is that you are assuming that dev fixes one bug per commit. I don't know about you but I find that very rare. Usually they fix a lot of bugs and then commit all the fixes at once. Now how do you peel back what was fixed for this one bug? If dev can't be bothered to spend a few minutes explaining what was fixed and why to contribute to the overall quality of the project, then that's a significant problem IMHO. – JeffC Sep 16 '17 at 0:08
4

There are two questions here:

1) How do we convince developers that testers need to know what they've changed?

2) What's the best way for them to convey that information?

I'll focus on 2, as this is what we're doing. I may come back and rant about 1, though there are many good answers here.

For 2, I'm assuming, hopefully correctly, that there is some sort of code review being done, and some sort of function test. (If you're doing function test and don't understand what changed, that's a bigger problem then just this.) I'm assuming you're doing integration test/system test/something where you're using the system as an end-user was.

So what we've done is have everyone involved in the testing invited to code reviews, and they restructured the code review so that the first few minutes are devoted to an overview of what changed, and then the rest of the code review is diving into the code. So we can be there for the first few minutes, understand what was wrong, at a level we understand, and what changed. This both helps us understand the system a bit better, but also what we need to do to verify the fix, because sometimes, the external symptom and the internal cause are not easily related.

4

I do not think it is reasonable to ask the developers to always add a very detailed report.

Working software over comprehensive documentation

Are you sure you are always going to use the data? This just sounds as a documentation waste. I read it as we the QA want you to log all the information possible, because just in case we might want to use it sometime...

Do you really need it for re-testing? If yes:

  • Ask the developer in person (or over chat) when you a really interested or if you are going to do something with the information. Explaining why you need it is better than just asking to supply it always. Good reasons are risks estimation and or doing a root-cause analysis.
  • Do a root-cause analysis with the developer together. Now you can log the cause and actions to prevent similar issues in the future. Handover data like you are requesting loses 50% of the information even if the developer really tried to give everything.
  • Link commits to bug-tracker-id. Most if not all bugfixes there are only a couple of lines changed. A tester should be able to scan for dependencies or risks for other systems now.

Sure I do understand why you want to information for further testing, but more data in a bug-tracker is not the solution. Sitting with the DEV team is. Talking to the DEV team is.

  • 1
    Who says it needs to be comprehensive? :) – c32hedge Sep 15 '17 at 14:44
  • 1
    Doing Root cause analysis together (QA, DBA, developer) might work in simple systems, but in really complicated system like ours, it would be waste of time. Our system is so rich in features/services (over 100 servers) that only very few senior developers can hold it all in head - and none works for QA. Good for you if your system is simple enough that more people can understand it in full. Yes we do root analysis, but it is compartmentalized by part of the system. – Peter M. - stands for Monica Sep 15 '17 at 15:46
  • Most bug-fixes to complex systems are still only a couple of lines and most often do not touch the whole system. The buggy code is just the symptom of some larger missings in the process. I would suggest the root-cause analysis could be performed by the single developer who created the fix. Spending 15 more minutes to think about why this situation was caused is worth the time. If the developer don't do this by themselves as a process. I think QA should help them with it, certainly on complex systems, as preventing similar issues sounds way more important than with smaller systems. – Niels van Reijmersdal Sep 16 '17 at 8:04
  • A root cause analysis is most likely going to take a long time and you are asking QA to sit in on most every one. That is going to be a waste of time most of the time for QA given that they don't generally understand the code. What OP is asking for is a brief explanation of the fix so they can be better testers. Most testers don't have a couple hours a day to sit in with dev and watch them work. This is likely made nearly impossible since the dev team is offsite and likely in another timezone. – JeffC Sep 18 '17 at 13:54
  • Maybe you are over complicating your RCA sessions. QA can be great RCA facilitators. Sessions longer than 30 minutes most of the time do not add more value. Code is never the cause. I like fast adaptive cycles, not boring long meetings with whom ever likes to hear themselves. Read the linked RCA format, as it is short and good for a single person or a small group. – Niels van Reijmersdal Sep 18 '17 at 19:20
3

I understand where you are coming from, asking this question.

It is fairly important for us as testers to understand the fix. Not necessarily from a coding pov but to understand it as much as we know how to assess if the change will or will not be impactful on other areas of the application. We`ve had success in just talking to the developer and stating the above(some understand others are a bit more hard-headed, cannot win them all unfortunately)

On the other hand, there are means, we as testers, can understand if a certain fix on a specific area of the SUT will have potential to cause regressions on the code.

Some specific means we can inspect the changes by ourselves:

  • Continuous integration means that we should have access to the commits/branches/deploys that the DEV team are doing, and we can see what the changes have been and what areas are affected
  • Certain tools, like TFS and MTM scan the change made, and can suggest you a set of tests that cover the impacted area.

On how you should convince devs to share that, my answer would be that you are self sufficient enough to find out by yourselves. Do not rely on a DEV to babyfeed you stuff when you can find it out by yourself, and to me this feels a bit more satisfactory(and you can learn a bunch).

3

So the actual question was:

How can we convince developers to explain a bugfix in a more detailed way?

In my opinion the best way to convince somebody is not trying to convince, just put them in your shoes. For example, once in a while, before version release for example, you may ask the managers to get help in verifying closed bugs. Managers will assign developers to verify the bugs in order to help QA Team. When developers will try to reproduce and test bugs they will understand what details are missing in the ticket or in the commit message.

So my message here is,

Don't convince, let them join you and be in your shoes.

  • I think a lot of people think that just running through the documented steps and making sure that they work now is good enough. OP wants to go deeper (as any good QA should). It will depend if management has a QA background as to whether they see anything wrong with this. – JeffC Sep 18 '17 at 13:58
3

This is a communication problem.

As defined in the Agile Manifesto

Individuals over processes and tools

Most of the answers point to tools and code and process. I would push for:

regular in person conversations about the fix details so you understand the fix.

If you get too much push back about having the time to go into the details then as a team you need to have a talk about why you need to do that. This is the role of retrospectives in Agile environments. If, after attempts that are not leading to things improving, it's time for a discussion with management - but be careful: if they "fix" the problem you may have also just enabled command and control which is a big problem in most workplaces.

The ironic thing you will find over time is that as the QA person asks questions of the "fixer" it will start to turn up questions that the developer hadn't thought about... conditions they had not consider... and sometimes it will lead to a "oh yeah, let me address that too".

This is the path to quality

1

I'll start with the issues, then work into how to fix it.

The issue here is bidirectional communication. Both parties are failing to communicate with each other. You're going to need to do a better job of communicating to the devs what you actually want. For example, let's take your example of "Fixed in version x.x.x.x, please test" You say you want more details. For sake of argument, I'll play the dev and play along. My next reply looks something like:

Fixed an off-by-one issue in the UnderlyingServerManager decorator which caused it to produce an incorrect hash on Widgets whose memory address was a prime number - 1.

Now you have details. There's only one problem. None of those details are useful to you. You don't know what an UnderlyingServerManager is (hint: it's underlying, buried deep in the code where non-coders cannot see it), you don't know what Widgets are, and you may or may not know what the implications of the incorrect hash could be. You have the details, but they didn't help all that much.

What you are really asking them to do is to help you do your job. They know that they made a fix, and you know that there may be implications associated with that fix, but you don't know what those implications are. In fact, since you mentioned many testers don't know the language it is coded in, many testers don't even know where to start! You need the developers to do the first half of your job, so you can do the last half.

Now this is not an unreasonable desire. It has been said time and time again that developers cannot just code and throw the results over the wall to test. However, I chose my words to point out that the handover of information you are looking for is not on a clean border between code and test. You're looking for them to distill the code related information into something that's already fully in the language of test. That's a lot more work, and the developers may or may not be up for it.

The fix needs to start with you or someone above you in QA learning what tasks the developers have on their plate and how stressed they are. You need this information for two reasons. One is the obvious one: you're going to be asking them to do more work to make your life easier. It's good to know how much bandwidth they have. The second reason, which is more important, is subtle. You're going to need a communications channel opened to properly discuss what needs to be done. A channel that is capable of gleaning their work load is one which is in a position to find ways to help them out.

"You scratch my back, I'll scratch your" should be the name of the game. Perhaps you find out that there's a QA step that's actually getting in the way of them doing their job well. See if you can offer a trade: simplify that QA step for them, in exchange for opening the dialogue on how to get "more detail." Get them to want to help you out, and the process is easier.

The next step is to realize that what you are asking for is not "more detail," but "less detail, in a human readable form." They gave you the detail -- it's in the code. What you want is a version of that which has already been paraphrased into a form that your testers can work with. I point this out because the developers are not going to know what you need. You're going to have to work with them, back and forth, until they understand what information is meaningful to testers. You need to train them to speak your language (or learn code, so that you can speak theirs). Generally speaking, they wont want to, which is why we started with the first step. Sweeten the deal by finding ways to make their lives easier.

Make sure management knows that you're doing this, and make sure to laud the developer's efforts once they start doing it right. Cooperation is not easy in a corporate environment, especially a remote environment. Make sure their bosses know that the devs are pulling their own weight.

1

It is information valuable to other coders so why not share it with test.

If I am pretty sure the fix is limited and does not require a full set of regression testing I will share that.

If it was a database change that could affect other programs I share that.

Test could be using some manual queries that need to be changed.

I had a case where it did not work in test but it did in development. Turns out test did not copy a config file. If I had told them there was a change to the config file time could have been saved.

  • 1
    This doesn't answer the question. The question is How can we convince developers to explain a bugfix in a more detailed way? and you've described what you do... not how to get others to do something. – JeffC Sep 18 '17 at 13:56
  • @JeffC Explain the value is not a way to get others to do something? – paparazzo Sep 18 '17 at 14:23
  • The question is not whether there is a value in sharing the information. I agree that there is a value as does OP. Just because something is valuable to other devs doesn't mean it's valuable to test. You are stating that you find it valuable but the question is how does OP get the devs he works with to see the need or the value in sharing that info enough to spend the time to document it. – JeffC Sep 18 '17 at 16:34
  • @JeffC My question was rhetorical. Sorry you cannot understand how this answer cannot be used to solve the stated question. – paparazzo Sep 19 '17 at 6:40
-1

Let's be completely honest here: The most probable reason the bug got in was that a developer made a mistake. No problem, we're all human, and if developers didn't make mistakes there would be no jobs for QA engineers. You're asking people to put down in writing, in a permanent record, "I made a mistake", and it will probably also be a mistake that was very obvious after the fact, embarrassingly so. And you are surprised that people push back on this? Let me flip this question around, if a customer discovered a bug would you consider it reasonable to be expected to explain to that customer how you personally, with responsibility for quality, allowed such an obvious bug to slip past you?

Things like this are best discussed face to face in meetings at which no fingers are pointed and no names are written down. You can get all the information you need, and people will be willing to provide it, but you must go about it in a sensitive way, rather than gathering ammunition for a witchhunt.

  • 2
    You missed the point of the question... the point is for QA to gain additional information that allows them to test better. It has nothing to do with public admission of fault, accepting of blame, etc. .. and yes, if I led a team that allowed a big bug through to production I would expect to be asked how it happened and how I was going to prevent it from happening again. It's all part of improving the process and being accountable for the work we do. – JeffC Sep 18 '17 at 13:46

Not the answer you're looking for? Browse other questions tagged or ask your own question.