1

I was talking with my senior tester and he asked me a question I couldn't answer to his satisfaction

When I am testing, how do I recognize a problem?

I answered that when I am testing a product, I am testing all relevant factors. I check for console errors, I check the error log, pagination, breadcrumbs, any php errors, database errors, whether I can cause a sql injection attack. If anything seems bad then I report a bug.

My senior tester asked:

How do you know when something "seems bad"? How do you tell the difference between good and bad?

I answered that when I am testing a product like customer experience, with a registration page, if I entered the details and click on submit button and I see that the name couldn't register yet, this is a bug and seems bad.

My senior tester asked:

What makes that bad, though? Why is that not OK?(Those probably sound like odd questions; that the answer is obvious. What's obvious, though?)

I said that the name is not registered and I got a php error so this is a bug.

He said:

You know it's a bug only because an error message appeared? What if an error message didn't appear? Would it still be a bug? How would you know?

I said that if an error message didn't appear, then I would check the users list, and if the username doesn't appear it should be a bug and then also I check to login with the particular user. If it couldn't get to login, then it should also a bug I also check the console error too.

What makes it a bug? How do you notice other bugs?

Incorrect results happens, some mistakes in design and coding.

How do you know they're mistakes? How do you know they're incorrect?

Because its a functionality error

The programmer tells you how he believes it should work, and it doesn't work that way. So the product is inconsistent with a claim the programmer made. Problem?

I don't understand what my senior tester is looking for. He is asking me how I know something is a problem, but none of my answers are good enough.

What are the standard ways of deciding if something is a problem?

  • To me a bug is when the system works in a way different than expected. How do you know what is expected? Well, there are different ways to get that information: formal requirements, users, similar products. The book "Lessons Learned in Software Testing" has a whole chapter on that. – dzieciou Apr 3 '17 at 11:20
  • Edited to make less opinion-based and more answerable. – Kate Paulk Apr 3 '17 at 11:30
2

In general we (software testers & engineers) use Oracles to recognize problems in software.

  • An Oracle is a way to determine whether the program passed or failed a particular test.
  • Oracles can be reference programs, documents, requirements, heuristics, or other inputs of information: all of which are fallible to some degree. This means all Oracles are only partial (they don't tell us the whole truth, just part of it) and are incomplete.
  • Since our Oracles are incomplete that means testing also requires us to use our own judgment about what we think is correct or appropriate behavior for a program.

I don't understand what my senior tester is looking for. He is asking me how I know something is a problem, but none of my answers are good enough.

I've used this exercise before and when I do I'm trying to understand how much the tester(s) understand about the challenges involved in the work you are doing. Understanding how we recognize potential problems leads us into how we design tests to reveal those and other similar problems.

This foundational aspect of software testing is called "The Oracle Problem".

A few other references:

0

Short answer: The client/PO/user (whatever name you want to give to the person/people who says what should be built).

Testers have (primarily) the responsibility to give knowledge about a product to all involved in the project. This knowledge includes functional behaviour (and if it deviates from previously defined spectations - specs, e.g.) or processual.

Not rarely, the spectations were not complete or we have realize that they were not the best approach. Testers can bring these problems to the client, but he is the one responsable to determine what to do with this new information (or he can delegate this decision to any member of the team, if he sees it as a better approach).

0

What is a software problem?

A software problem is an issue that affects your company

How do you know about software problems?

This is the heart of the question and the list of how you know about problems is long and includes:

Bad Things:

  • There is an error displayed on the page
  • Users are complaining
  • Revenue is dropping
  • Entering invalid or missing data breaks the system
  • New user sign-up is dropping
  • There are syntax or grammer errors
  • Affected data is incorrect
  • Logs are recording errors
  • Functions don't match requirements or documentation
  • The item can be seen to have not been added / updated / deleted
  • The user cannot perform the expected actions after it
  • The display and layout is hard to use on the device being used
  • The process is too slow
  • Unit, Integrated or Feature tests are failing
  • The database is not correct after the operation
  • Different devices / browsers / versions have issues
  • Experience with subtle usability and accessibility issues
  • Knowledge of regulatory requirements such as Section 508 Accessibility and Sarbanes–Oxley

Also: understanding the company's goals and missions and what it values as quality and what categories of items it consider to be bugs.

The answer perhaps to 'how do you know to look at those things in the first place' is also that when you test a system you follow some fairly paths to find issues such as:

  • You focus on the sad path and assume the developer got the happy path working for them
  • When you add an item, you try editing it.
  • You looks and carefully read all the screens and messages.
  • You enter invalid data and missing data
  • You try different scenarios and personas
  • You use boundary data
  • You use different devices, browsers and versions.
-2

Thats why you need proper test cases. Only way you can tell a functionally is wrong or right by referring to your test cases . Test cases are written based on the spec or SRS. So again you need to focus on user requirements.

  • This answer assumes that test cases are the Oracle and that they always maintain their accuracy. I've never seen either of these to be true for most of the time. – Chris Kenst Apr 3 '17 at 18:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy