28

I have been working in testing for almost 4 years and had never had to ask this question because I worked in highly commercial and critical products.

Whatever bugs I raised were marked as “to be fixed”.

Recently I have taken QA ownership of an in-house REST API implementation

One of the acceptance criteria was that if a user searches for a project which does not exist, the result should be

project "xxx" not found

However, if I search for a project named “xxx\n”, the message that is retrieved is

Project Resource not found

The developer is the PO of the project and he is insulting me, saying that no one will enter emoji, emoticons, or special characters in the search field. Even if someone enters nothing, the error message is logically correct.

But the error message is not as expected per the acceptance criteria. Is this a valid bug?

The work environment is getting toxic as corner case bugs like special characters, spaces, etc. are being treated as silly.

What should a tester do in this situation?

Improve this question
11
  • 16
    What happens if user searches project by copying it from somewhere and there's a trailing newline/space/unicode non printable character?
    – Tomáš Zato
    Commented Oct 31, 2019 at 10:59
  • 11
    Obligatory XKCD - Little Bobby Tables
    – Mawg
    Commented Oct 31, 2019 at 14:40
  • 12
    It would concern me, as it suggests that the API is not escaping things correctly, and there could be a potential security hole in there.
    – TRiG
    Commented Oct 31, 2019 at 21:43
  • 4
    If it's not per the specs, it's what you call a "valid" bug. But not all valid bugs are worth fixing. The part about your environment becoming toxic sounds more like something to ask at The Workplace. You will want to add more details though. Personally, it sounds like you may be over-sensitive in this case. It's not uncommon for a dev to tell a tester that a bug is too "edge-case" to worry about fixing, but it also depends on who the customer is!
    – Mars
    Commented Nov 1, 2019 at 2:00
  • 2
    @user253751 It is a bug unless and until the developer convinces someone to change the acceptance criteria. Which is not QA's job.
    – gnasher729
    Commented Nov 1, 2019 at 13:10

6 Answers 6

Reset to default
14

One of the Context-Driven Testing principles is:

The product is a solution. If the problem isn’t solved, the product doesn’t work.

Another way to say this is that software should work for its user, not for some generic/arbitrary/commonly-used set of definitions of how things should be.

When he says

 Even if someone gives, the error message is logically correct.

It can mean that the users of the API - probably computers/software - are satisfied with the error. Maybe the consumer contracts of the API have driven such error handling. Maybe some conversation or email has driven it.

enter image description here

Or maybe not!

If you know the current agreement is different from the implementation, then you have a good case for a change request, aka a bug to be fixed. If you don't know about any sort of agreement, you can ask how the dev/PO knows it's OK and how you can identify beforehand in the future if something of this sort if OK or not. This conversation can either serve for you to learn more about the product development or to show to the dev/PO that there is a risk in assumptions not-based on users' expectations.

For more details on this, I would suggest the book Bug Advocacy, by Cem Kaner.

enter image description here

Improve this answer
19

Take a deep breath

step back

and look at the big picture

Talk to folks / your boss about standards. Have a meeting. Agree on standards including items such as special characters. Take short term initiatives this month to reduce long-term repetitive pain next year.

and allow for humans

I've lost track of the number of times I've: typed weird characters by mistake; clicked the wrong link; clicked 'ok' by mistake; dropped my keyboard; had my cat walk on my keyboard. These might seem silly or funny or whatever to some, but they are amazingly common. They've happened to me; my wife; friends at work; personal friends - all that's a lot for a pretty small group. Two weeks ago i dropped my keyboard and my display went like 400x600 and it took me 2 hours to fix it - and I was quite motivated because it was (supposed) to be a screen share remote pairing session for an interview!

Then live by the agreement you make

Also:

  • Don't enter bugs outside of the agreement
  • Don't breach the agreement for parts you didn't really like but agreed to anyway
  • Ask in person, such as 3 amigos meeting (product-dev-qa) when you meet an edge case
  • Generally prefer asking product owner over developers

To avoid toxicity and being seen as silly, take initiative in these areas and you'll be seen as quite serious without seeming toxic (although it depends on how you act)

Remember to focus on the business and the business goals, not just 'tests and bugs'. When you relate bugs to lower revenue or less customers it means more to the business.

Improve this answer
3
  • 7
    The developer itself is the PO , that's the main issue.
    – PDHide
    Commented Oct 31, 2019 at 0:02
  • +1 for less customer = more for buissness
    – PDHide
    Commented Oct 31, 2019 at 0:18
  • 2
    Second the 3 Amigos! It may be more difficult since dev functions as Project Owner. Is there a UI/UX person, business analyst, other neutral dev or qa person you could include? If things are toxic, I wouldn't try this without a neutral party too. Over the years I've found a general walk through of my questions & test findings works well. Dev sees the issues; PO gets a better perspective, and with 3 instead of 1 we come up with even better ideas & solutions. As a result specs change, stories are updated or a bug is logged, or several of those things happen - but it was a group decision.
    – CKlein
    Commented Oct 31, 2019 at 11:40
4

For an API, I would consider this a bug. An application is possibly going to be parsing the text so it should be the same for any invalid project name.

However, as a backend developer myself, it's definitely possible that it just isn't important enough to fix, compared to other issues.

I think your main problem is not a technical one but the fact that you're being ridiculed for it. This is toxic and shouldn't happen no matter whether you found an important bug or not. I'd focus my attention on this instead of the debate what is or isn't a bug.

Improve this answer
2
  • 1
    "it's definitely possible that it just isn't important enough to fix, compared to other issues."I was thinking the same thing. It may technically be a bug since the behavior doesn't match the spec. but does it really do any harm? Or is it more 'cosmetic' in that the text isn't an exact match but the software behaves & functions fine? If so, what is the value in having resources fix this vs. other things required by the customer that are to be delivered this sprint/release?
    – CKlein
    Commented Oct 31, 2019 at 11:30
  • 1
    "Cosmetic" bugs can make customers hate the product, and fixing them can very much improve how they perceive the quality of the product. Especially if "cosmetic" bugs make the product look amateurish.
    – gnasher729
    Commented Nov 1, 2019 at 13:12
3

Acceptance criteria are not set in stone. They change as we get new insights. Now in your example case, the question is: "is it good enough?" Personally I think it is.

I wonder if the toxic environment might be because you are acting too strict constantly. If you do not act agreeable most of the time people will stop taking you seriously. Know when to stand your ground and when to accept that the application is not perfect, but is good enough for now.

The work environment is getting toxic as corner case bugs like special characters, spaces etc are being taken as silly.

Talk with people and figure out why people are behaving as they do. Make it clear you do not like it and agree on a more acceptable way of communicating. Keep in mind Covey's habit 5.

Habit 5: Seek First to Understand, Then to Be Understood. "Most people do not listen with the intent to understand; they listen with the intent to reply."

https://www.franklincovey.com/the-7-habits/habit-5.html

I think you might be trying to getting people to understand your side of the story first, without understanding and acknowledging theirs.

Improve this answer
0

Case 1: If the error message is not as per the acceptance criteria ,and if there is a separate error message mentioned for special characters input ,then it will be a valid bug

Case 2: There is no specific error message provided by acceptance criteria,and you are thinking that displayed error message is not the specific error message then discuss with a dev team as improvement ,it definitely not a bug ,it just improvement to that feature

Improve this answer
4
  • 1
    Even if it is a "valid bug", it's also worth noting that not every "valid bug" is worth fixing!
    – Mars
    Commented Nov 1, 2019 at 1:53
  • 1
    @Mars With that attitude, rubbish builds up. In this case, it sounds like a trivial fix, so more time has been wasted by discussing the matter instead of just fixing it.
    – gnasher729
    Commented Nov 1, 2019 at 18:58
  • 1
    @gnasher729 In this case, I think it's more time discussing policy regarding such bugs than the bug itself. And a lot of bugs are trivial fixes, that then require a re-test, test reports, someone to sign off on those reports and redelivery of the app....
    – Mars
    Commented Nov 4, 2019 at 23:58
  • 1
    @gnasher729 Also, this fix doesn't sound completely trivial... you need to decide which characters should not be allowed, where to filter for it, etc. Do you stop the user from sending it in the first place? Do you process it on the backend? What about different encodings, if someone just happened to copy and paste a weird project name? Quick hacker fix is probably just to filter for newlines on the backend and would take about 30s, but in a corporate environment, that could eat up a few man-days and involve multiple people...
    – Mars
    Commented Nov 5, 2019 at 0:17
0

Per definition of what a bug is:

A software bug is an error, flaw, failure or fault in a computer program or system that causes it to produce an incorrect or unexpected result, or to behave in unintended ways.

One should write a unit test for the method returning that string - matching the criteria with given string format, which has the name of the project included - and while it fails this test, this is a bug.

Improve this answer
3
  • Hi Martin thanks for replying , the question is more on a ethical and professional level than theoretical . Does such bugs matters or should be investigated
    – PDHide
    Commented Nov 2, 2019 at 18:37
  • @PDHide while there is a SRS, software requirements specification, which states how the output should look alike that - and when there is a test case, which asserts how the output should look alike, this is technically a bug and needs to be fixed, despite it is rather of cosmetic nature. This bug might be of low priority, hence it is barely cosmetic, nevertheless it had been projected how it has to look alike - and so there is no room for discussion (without SRS and test-case, this might be rather open for discussion).
    – user39770
    Commented Nov 3, 2019 at 5:04
  • 1
    I mean, if it would be a bad or wrongful specification, one could still provide a reasoning "why not" (eg. if an error message would disclose relevant information to unauthenticated users), but while there is nothing wrong with the specification, this specification is "the law", to be enforced with test cases.
    – user39770
    Commented Nov 3, 2019 at 5:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.