What is the best way/tool to share automated tests with non-technical stakeholders?

Question

I've been tasked with finding a tool/solution to present our automated python test cases to other stakeholders in the business (I won't write actual python for time-saving purposes).

Our tests are structured as follows:

[Python test file]
User journey #1
- Code line 1 ("login")
- Code line 2 ("fill_in_the_welcome_form")
- Code line 3 ("verify_the_form_details_were_saved")

[Python DSL file]
Code line 1 ("login")
- Click "Login" button
- Enter Username value into username field
- Enter password into password field
- Click "Proceed"

Code line 2 ("fill_in_the_welcome_form")
- Fill in field 1
- Fill in field 2
- Fill in field 3
- Click "Save" button

Code line 3 ("verify_the_form_details_were_saved")
- Verify value "x" was saved into field 1
- Verify value "y" was saved into field 2
- etc
- etc

So we have reusable blocks of code in the DSL level that we can piece together like Lego in the test files and they are named in clear English sentences.

I can say to the stakeholders that "We have a test that fills in the Welcome page" but that's not helpful. Plus they can go to GitHub and look at the test files in our repository but that's fairly long-winded. Plus each line isn't very descriptive. "Fill in field 1" doesn't say anything about exactly what value is being inputted. The only way they could find this out is by going deeper into the code and checking what test data we provide.

We have tried documenting the test steps in Confluence/JIRA/Spreadsheets etc but it's a massive overhead to manually update all the documentation every time the product, and then test code; changes.

In your experiences, have you used a test management suite or tool where you can just document test steps? Preferably something that can be interacted with via an API, so the documentation can be easily updated from the code level. We use JIRA for everything, so any kind of plugin would be amazing.

Sorry for the long post but hopefully someone can advise. :)

Thank you, Jack

Answer 1

I think there are two parts to this.

1. Write self-executing, living documentation through using BDD code as @kirbycope indicates.

2. Figure out a way to educate non-technical folks that 100% coverage never happens at a functional level (don't call it an 'education' of course, call it an opportunity for a deeper conversation about value or some such bs). 100 coverage isn't usually practical once you consider the number of devices, browsers, versions, landscape mode, network speeds, etc. not to mention different data and the possibilities are essentially endless. So the focus needs to be on what is worth testing. And what is worth testing at the UI vs unit test level. How important is speed of code to production vs disruption to users from bugs that aren't fixed quickly vs how long the test suite is taking. Conversations and education about options and choices are the successful outcomes here.

Answer 2

I use the "living documentation" and clear language of Cucumber. The Sceanrio Outlines will show what data is being as input. You can give everyone Read access to the repo (works for my smaller clients) or write a script to pull it into a spreadsheet (works for my government clients).

Scenario Outline: eating
  Given there are <start> cucumbers
  When I eat <eat> cucumbers
  Then I should have <left> cucumbers

  Examples:
    | start | eat | left |
    |    12 |   5 |    7 |
    |    20 |   5 |   15 |