Do we need to do cross browser testing for minor text changes in blackbox testing?

Question

Background: When a new user opens our web page for the first time, we show them how to use the webpage. For each control they need to use, we display a popup next to the control to tell them how to use it, switching popups as they switch controls. Two types of users can make posts: Mods and Regulars. Mods have more privileges than Regulars (Eg. Ban spammy or rude users and more).

Requirement: The text for one popup says "Welcome to this software!". We are changing the text to "Welcome to our App!". The size of the popup remains the same but the length of the text is now shorter. Note that the popup has only plain text.

Question: Do we need to test this change in multiple browsers or is just one enough? Do we need to test this for both Mods & Regulars?

My Opinion: One browser is enough because its a minor text change and it's highly unlikely that it will break the layout or UI. Moreover, because this is an infrequently used feature & not critical to the functioning or usability of our application, we do not need to test it in multiple browsers.

We probably do not need to test this for both Mods and Regulars. Only testing for Mod or Regular should be enough, unless our app has different code for generating the popup text for Regulars and Mods. If that is the case, then our app design is probably bad. When you want to show the exact same popups to Mod and Regular for a particular page, then why have different logic to generate those popups?

Are there reasons I'm missing that we should test all these cases? Or other points that would help make the argument that we don't need to?

Answer 1

I think the general answer is "be reasonable". Just because you're doing blackbox testing doesn't mean you should do extra useless testing--in this type of situation, you're likely better off looking at how the code is implemented and deciding on the scope of testing accordingly. I've seen too many blackbox tests that were long and painful to execute but didn't add any value because they were doing lots of checks that each "looked" different, but in reality were testing the exact same code path over and over.

Now, if you looked at the code and found that there really is different code for displaying the exact same message based on which type of user is logged in, then you would need to back off, question your assumptions, and do a lot more testing. But probably in that case a redesign is probably a much more valuable use of time. In that case, a simple code check would have revealed a far bigger problem with the product than whether this text always appears correctly, namely, that it's potentially a spaghetti, unmaintainable codebase if it requires significant testing of unrelated features even for such a small change as the text of a popup.

If someone is pushing testing all the possible cases for this change because "we should test everything", have them read Perfect Software; and other illusions about testing by Jerry Weinberg, which is a very eye-opening book on how to approach (and not approach) testing. In particular, Chapter 3 "Why not just test everything?" makes some very good points about why "testing everything" is a common fallacy. The part on "Instant reviews" in Chapter 16 is very related to the example you gave that I alluded to above, about observations that tell you much more about the quality of the product than the actual test (or review) in question.