What should stakeholders or customers see and not see in the test report?

Question

A rudimentary question, actually. But exactly this led to a lively discussion in our teams.

Example:

We use Owasp ZAP for security testing, at the end we can of course create a detailed report (mostly 250 pages), or a rudimentary report which we then use to show it to stakeholders or customers.

Example 2: We use in some test projects Katalon Studio also here one can create accordingly yes Reprts, and these one can also adapt. So we always create separate reports for developers and even customers, stakeholders.

Unfortunately, there was a request from some customers to see more details in the reports, but should a customer see too much? We have our doubts, because it will also lead to misunderstandings.

Questions:

• Have you guys found a solution to how your teams approach this?
• Should reports be customized for stakeholders, customers?
• Is too much negativity in the reports harmful, or does it provide the clues that stakeholders, customers need?

Answer 1

Should reports be customized for stakeholders, customers?

Of course, you don't want to share something that's of no interest to someone. That's a waste of everybody's time.

Is too much negativity in the reports harmful, or does it provide the clues that stakeholders, customers need?

Do you add extra layer of negativity in the reports? Or do you just provide feedback about how things are right now? If someone can't digest the truth, why should it be a problem on your side?

I don't think you can come up with a universal solution here, nor do I think such human-centric problems are ever really solved. But you can talk to the customers and listen to what they need in the reports. Then you go and deliver the information to them.

We use Owasp ZAP for security testing, at the end we can of course create a detailed report (mostly 250 pages)

250 pages seems really long at the first glance. However, it might be exactly what those people need. They should be able to tell you. Perhaps you can share a few examples of reports you can create and ask them if one of them fits their needs.

Unfortunately, there was a request from some customers to see more details in the reports, but should a customer see too much?

It depends. If you are dealing with security topics, some information might be too sensitive to share outside of the company or team. You need to know that.