0

This is "a site for software quality control experts..."

Why do most software-based companies seem to have a wall to prevent customers from submitting or monitoring bug reports, via support or directly to development? IOW, rarely can customers submit bug reports that (even if 'good' in the technical sense) have more than a snowball's chance in hell of being communicated to development. Why?

If you are a software quality control expert, why do you think your/most companies follow this pattern?

(Feel free to disagree with my rarely/most claim, but my question is about the why; please read it as, "Why do many..." if that matches your perception.)

(I perceive that a wall is the norm, and it surprises me, and so I'm curious why things are this way and if it surprises others too, or if they've noticed it. I had a hard time deciding where to ask this, and this board feels like a good match both technically, and in terms of a higher ratio of people who'll have good, interesting answers.)

(Added later:) Perhaps some examples would help?

1)Here's a case where development can't communicate back to a customer: I found a bug in iOS, and I DID manage to communicate the other way - the bug I found was (I'm told) communicated, indirectly, to engineering. Today, I heard that there was a new version of iOS, and I was asked upgrade and confirm that the bug still existed* - which indicates there's no communication in the other direction - just the support guy noticing that there was a new version of iOS. (Furthermore, I just realized, he apparently didn't bother to test to see if the bug was still reproducible before urging me to upgrade.)

*Well, even though Apple had, eventually, reproduced the problem, his request, verbatim, was, "confirm if you are still having the issue with siri and not being able to go back". (FYI: The issue was not being able to go back to see or edit what Siri had heard." When I first reported the bug, the first tech said he was unable to attempt to reproduce the bug because he didn't have an iPhone that could run Siri. He only had a virtual iPhone that couldn't run Siri. He said there was no way for him to get access to a phone in order to attempt to do so, and when I asked to speak to someone who could, I was turned down initially. But then I was transferred to a great (off-site?) rep who was willing and able to attempt to reproduce the problem on another iPhone with the same OS version in which the regression bug seemed to have appeared. And he was able to reproduce it.)

  • Some (relatively uninformed) WAGs: Is the fraction of 'good' ones so small that most companies just don't bother to look at 'em at all? It seems like it. Insufficient resources? Lazyness? – Matthew Elvey Sep 6 '19 at 1:58
  • Perhaps some examples would help? I've just added one. .......... Another WAG: The fraction of "bug reports" that are actually PEBCAK errors is very high, and the customer service staff stops forwarding even what seem to them to be legit bug reports because development seems to 'always' be complaining about PEBCAK errors. (PEBCAK? Google it!) – Matthew Elvey Nov 3 '19 at 3:29
0

The main answers to your question are lack of scalability with user reported bugs, and management (and users) emphasis on new new features vs fixing problems.

  • A typical consumer product might have hundreds of thousands of users up to more than a billion for successful ones (for example Office)

  • Each product have hundreds of features, or more

  • Bugs manifest themselves in different ways and are not necessarily related to the same feature. For example there might be different buttons that trigger the same underlying bug.

  • Multiply this by different platforms and versions of the platform and application. Some bugs might be relevant only to rare combinations or old versions of both

  • Remember that a feature is usually being developed by a rather small team, a team of ten can be responsible for a rather large set of features.

  • Simple math will show that a typical product is being developed by many teams, for big products this number can be hundreds with cross dependencies

  • Each development team is responsible for developing new features for the product, fixing problems found internally and fixing customer issues

Now think of the amount of bugs flowing in from the customer side, many lacking proper information, others caused by misunderstandings or PEBCAK as @Matthew Elvey calls it, all needs to be mapped to right team and usually to the right person.

As we all know automatic bug analysis is still a new field at a research stage so each bug will need to manually investigated and routed by a human, this will probably not get rid of duplicates with different symptoms since domain specific knowledge is necessary

We ended up with many bugs needing to be triaged by a small team.

Next comes the question of what are we going to do with this information, a typical company could for example:

  • Let the team triage and fix all bugs, practically this will halt all new development

  • Hire another team (or increase the existing team) to handle all bugs, this has enormous cost and still significantly slows down development

  • Since this is a quality forum I would bring up the option to develop better quality products, usually this will have the same affect as the previous options

  • Do a superficial triaging of public bugs, focus on the critical ones, prioritize them not only be severity but also based on the product's roadmap (why fix X if we are going to replace it soon with Y) and business decisions (support N versions back of the product and OS)

From my experience a variation of the last option is what most companies will choose for obvious reasons, customers and shareholders both wants new and shinny features for a reasonable cost under reasonable schedules and it's simply impossible to achieve if you listen to all of the customers bugs.

This will be combined with a simple channel for reporting problems since, again, how would you scale reporting problems back to users ? those channels comes in the form of chats, forums or support personnel tickets.

Now if you consider @virolino's complaint about Microsoft, with some of their products being very complex with over billion users what do you think public sees ? a huge amount of unsolved complaints hiding some bug fixes.

All of the above is true for consumer products, enterprise products are a totally different thing and usually allows direct opening of issues and even access to a specific developer.

| improve this answer | |
  • Accepted. Your reply prompts this 'new feature development' idea : Automatic bug triage could be greatly facilitated by requiring a few questions be answered as part of the bug filing process, e.g. Reproducible? For example, ask if the issue is bad documentation or bad UI. In either case, the submitter could identify the bad documentation or UI component. Even if no fix was suggested, if numerous issues referenced the same part of the program, that would help with triage. New issue reports could be triaged based on AI trained to recognize past good bug reports. – Matthew Elvey Nov 10 '19 at 3:46
1

I'm don't think is related to one of QA, but more one about marketing and how you want your company to be seen. In some ways the choice is between being "Totally transparent" or "Controlling the narrative".

Of the companies that I do see that are completely open, they are often based around open source software. In this case, the openness is beneficial primarily because of the people they sell to. Those customers could in theory go and clone the software and set up their own, but they choose to use the companies version because of the support. Being transparent in this case, shows the customers what they are paying for.

There are also lots of things that you might not want to be seen by the outside world depending on what kind of business you are in;

  • replication details for how to make the software do something it shouldn't. (Makes it easier for hackers)
  • Customer details which might have been captured in screen shots (You just gave away your customers to your competitors)
  • Actual data that can either give away what your customers are doing (You just lost your customer because they went out to of business to their competitor)

However, for me, the big problem is the people who would use these Open forums. Firstly let consider your staff;

  • Technical staff (I'm one of these, so this is probably a bit harsh)

    • Employed on their experience using computers and ability to solve complicated problems.
    • Can be defensive/emotional when our code is attacked.
    • At school kids are usually good at either written subjects or maths/science; developers most come from the ones that just scraped through every assignment with an essay. Writing is hard for us (This post was typed, deleted, read-back and rewritten in about equal portions).
    • Interruptions are bad for us. Our problems tend to require in-depth investigation while we build up a mental map of what is happening inside the system to cause that outcome. If you interrupt us we often lose some det.... Ooh squirrel.
  • Customer facing staff

    • Employed for their ability to relate to people
    • Understand that people aren't attacking they are just venting their frustrations
    • Are far better at expressing themselves
    • Tend to have better planning and organisation skills so they can multi-task (e.g. They might break a problem into smaller tasks and deal with each one at a time), which leads to them being better at handling interruptions.

I know who I would pick for dealing with my companies customers.

You also need to consider those customers too. If you log onto any forums, you will find a mix of rational, helpful people, and others who are frustrated by an issue. When frustrated, sometimes some people do not make rational arguments. If you open up your bug reporting, you are inviting people who your software has caused frustration, to vent that, and sometimes that isn't helpful.

Your bug reporting should be focused on the technical problem at hand. You want it to be a clear description of a repeatable process to see the bug. Those reading should not have to waste time getting through the surrounding issues, ("we need to fix this for a release next week", "My manager is going to fire me", "well it started when we moved to the new office", etc). These are valid points for the customer, but not for QA/Dev.

You cannot rely on your customers to be focused in this manner. I have seen one instance in particular where the customers bug reporting is simply "Call me now", and on one particular incident has followed up with "It's no use hiding" when we didn't ring him straight back. Would you want that publicly displayed outside of our controlled narrative?

| improve this answer | |
  • Good answer to why users should not be communicating directly with developers, especially the benefit of customer service-oriented staff. On the other hand, that was a secondary part of my question. My main question is why they can't communicate about issues via support/customer service either. >>Your bug reporting should be focused on the technical problem at hand.<< That's what "bug reports that [are] 'good' in the technical sense" referred to. – Matthew Elvey Oct 16 '19 at 2:48
  • ... in other words, I mean reports based on advice and formats like developer.mozilla.org/en-US/docs/Mozilla/QA/… – Matthew Elvey Oct 16 '19 at 2:51
1

Users should not have to bear the burden of sending detailed bug reports to software developers. They've paid for working software and its most definitely not their job to produce detailed bug reports.

Why not enable direct user ⇄ development communication? Mainly because users are usually not tech savvy enough to provide actionable complaints and developers tend to have a difficult time trying to decipher what users are actually telling them. On the other hand, I've seen that the majority of software development organizations do make customer / technical support available to end users. The sole purpose of this support is to consistently translate what a complaining user is saying into a concise but detailed issue report which is easily understood by development and also to give assurances to the users that their issues will be resolved or at least looked into.

But from my own personal point of view the only way to go is to proactively log abnormalities and scan logs for any potential bugs, crashes or issues and fix them quickly (no need to wait for issue reports from customers).

| improve this answer | |
  • I like your final paragraph. It's hard to understand what you mean, mostly. I'll take a gander at editing your answer to reflect what I am hearing. – Matthew Elvey Oct 16 '19 at 2:27
  • I'm sorry but you edit made the text less understandable to me. I'm not sure if the original point of it is still getting across. – Prome Oct 16 '19 at 7:04
  • I'm sorry it does that and would like to address it. FYI, sqa.stackexchange.com/review/suggested-edits/30638 shows the changes and the two users who agreed with my changes. It seems you're not a native speaker and I am. In several cases, I changed English that was not correct to a correct version, and I guess in some cases I used idioms, or phrasing that is less commonly taught, or messed up. Let's see... (We can delete these comments once addressed.) – Matthew Elvey Nov 2 '19 at 18:21
  • (You know you're welcome to make further edits, right? But I think, given the two 'agrees', a revert would be a bad idea.) I used "user-development communication", which I think is a concise term for the idea I wanted to express - communication in both directions, that is between one (or more) users and one (or more) developer team members, but is actually not very clear. Perhaps "user ⇄ development communication" is a good use of a non-standard character? Or perhaps I was way too concise? – Matthew Elvey Nov 3 '19 at 2:20
  • I switched from Customer and Company to the words I used in my question, which are more broad - user and development team. Does that not make sense to you? – Matthew Elvey Nov 3 '19 at 2:23
0

companies seem to have a wall to prevent customers from submitting or monitoring bug reports

That is actually NOT really true. I understand that you have some specific companies in mind, but please remember that most open-source projects allow you to do exactly that: get involved in the development, at least by submitting bug reports. Many of the open-source projects are actually backed by (big) companies.


On the other hand there are the companies which don't actually care about the customers. Consider Microsoft: what people actually complain about vs. what actually happens with their products. Reports of problems are very detailed throughout the entire Internet, and yet their products become worse with every generation. Remember the forced restarts and the lost work in the Windows 10, for example?


In the middle you will find the companies which organize a more-or-less good customer support, where any average Joe can complain in his own words about the issue, and then the issue will be translated to technical jargon internally in the company. These companies might be able to give you the chance to provide a technical description, probably by providing you with an e-mail address.


Monitoring the bugs by the customers is another "monster". That is considered by most commercial companies as strategic secret information, and releasing it to the public is considered very dangerous. No offense, it is just business. You would probably do the same, if you would run your own company.

| improve this answer | |
  • > companies seem to have a wall to prevent customers from submitting or monitoring bug reports <. It certainly IS true that that is my opinion. You're welcome to disagree, but saying it's not true is a bit much. – Matthew Elvey Oct 17 '19 at 17:22
  • Did you actually read the entire answer? I have the feeling that you only read the first few words... Moreover, i made no comment about your opinion, being yours or not. I only referred to the statements you made. – virolino Oct 17 '19 at 17:49
  • Your hunch was wrong. Perhaps you believe that most software-based companies produce open source software.Your comment referred to (quoted out of proper context) a statement I made that consisted of my opinion. It is a fact that it remains my opinion that "most software-based companies seem to have a wall to prevent customers from submitting or monitoring bug reports, via support or directly to development". – Matthew Elvey Nov 10 '19 at 3:19
  • I'll be honest and say I'm not aware of any company that flat out does not allow bugs to be reported. But let's make sure we are always following SE's "be nice" policy - we're all here to increase the sum total of the world's knowledge, and we can't do that when we're hostile (even if we don't mean to be.) <3 – corsiKa Nov 10 '19 at 8:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.