90

What I would do with any other bug - report it, write the bug report.


51

1% of 100 users is a very different issue to 1% of 1,000,000 users - make your team and stakeholders aware of the issue (preferably in writing, with a defect report) and then they can make the decision on the priority / severity of the issue. It might be a minor issue for you, but a huge issue for the company.


36

Is it still needed to write acceptance tests that target verifying those business rules implementation? Yes, it is absolutely necessary. Your unit tests cover the business rules in isolation. Your acceptance tests verify that the application properly implements those business rules from a customer perspective. There are many cases where the unit tests ...


30

This is called Risk Analysis. By the book, the over-simplified step is to analyze Impact x Frequency. Things that happen rarely but with huge impact can be prioritized, as well as things with little impact but too frequent. For a deeper understanding, I would suggest watching Michael Bolton's talk on Risk Analysis. There he questions our biases that may ...


26

There's already 2 correct answers but I can't stress this enough. You found a bug, you file a bug report. It doesn't matter who it affects or how. It could conceivably affect 0 real users and still be a bug and you still file a bug report. A QA's job is not to determine whether or how quickly bugs are fixed. A QA's job is to find bugs and make it known ...


12

Wonder why you want to present this to management? What is your goal here, what do you hope to achieve. Do you want to show them they have unknown quality issues? Normaly I would expect the product to have either an issue tracker or a backlog. Just put the issues on the list, discuss it with the business owners and let them prioritize. Together consider the ...


7

There are plenty of answers here talking about how all users matter. Essentially, this is a good reason to care about rare bugs, but I thought I would give a developer's perspective on some other reasons. First, and most importantly, bugs reveal a mistake in my thinking. It is important to correct my thinking so that I can correct the rest of my code. I ...


6

QA and UAT have different goals. From a commercial stand point QA is there to make sure the clients will accept the version during UAT. After UAT, taking it into production and making sure the client does not run into (too many) critical issues. Client UAT is there to verify clients do not take a version into production that does not meet their minimal ...


5

Yes you should have scenarios which verify the unit tested business rules. Some reasons: good unit tests mock and stub the datastore. Acceptance tests make sure it is configured and works correctly. unit tests usually test logic but don't allow for usability and accessibility unit tests don't test whether the application works on the vast array of devices ...


5

Although the main purpose of BDD is to enhance conversation between people involved in a project is can be used to described any level of test. Besides UAT, one can describe an object behavior (unit test) using Given-When-Then: Given I create a Cache object created with the arguments "..." When "15" seconds passed Then the cache is empty The point of ...


5

It really depends on many different things like: What is the severity of this bug? What is the effort of fixing it? 1% of how many users? There are a lot more things to consider, but a bug is a bug. It needs to be reported and the decision should be taken by the managers. So, I would simply report it.


5

How do you know bug will affect only 1% of the Users? Even if bug severity is low (impact low number of user as you say), business like Product owners might increase bug priority and fix it. If you do not create bug report for it, they are missing important information about the SUT. You should always report bug, once you find it.


5

A bug that is considered minor today may become critical tomorrow. As an example, many external HDDs only implemented USB mass storage properly, and had buggy USB attached SCSI implementation. That was a minor issue until UAS driver made it to the Linux kernel, resulting in data loss or inability to use such drives with Linux PCs after an update. Having a ...


4

Here are a few considerations: There are multiple reasons why a unit test might fail. Unit tests have to be maintained just like the system under test does. When a unit test breaks, it might point to a bug in the system under test, or it might point to a bug in the unit test (e.g. a timing issue), or it might even point to a unit test that no longer makes ...


4

As you say - 'it depends' but a few comments from my experiences: If you supply exact steps and data then what is the point of UAT? You might as well get your testers to run the scripts. I'd rather give the users some training on the system and give them scenarios to follow - which will have been developed with their input. Should anyone be able to run the ...


4

You state you are junior. I wonder if you are a "team of senior + junior QAs and you are one of the juniors of the team" or you are "alone in charge of this". If it's the first case, don't do any report, just tell your senior QAs teammates what you see and ask them for advice. From now on for the rest of the answer I will assume that you are in this latter ...


3

If QA and DEV environments are equal (have the same dependencies installed, the same build, etc.), there is no much sense to run acceptance tests on both of them, since it will be just duplicated execution and additional overhead in terms of analysing test results on two environments instead of one. Acceptance tests are the tests which provide you ...


3

Have you considered using post actions to either pull the UATs from the tasks or push the UATs to the task? This is relatively easy to accomplish if you have some experience of configuring your own workflows. It allows you to generate the tasks required based on the Acceptance Criteria (i.e. each would create a new task) and copy the details of that into ...


3

I am looking for a specific tool which will help me document what I see when I perform web site usability evaluations. At the end of each evaluation, I need to create an report for my client. As your demand above, I would think a tool called qTrace could help you out. It's a complete screen capture tool that helps a tester easily submits clear and ...


3

BDD is suitable for all levels of testing, Gherkin not so much. Writing unit-tests with the overhead of Gherkin (e.g. English feature files, regular expressions and functions to execute the tests) is extra work, but work that does not have the value of conversation as only developers read them. Describing your tests from a behaviour stand-point is very good ...


3

Jira, like Trello, VSTS, Pivotal Tracker and others are story ticket management systems. I've used them all and prefer Jira the most due to its UI, usability and integrations with both other Atlassian tools and other vendor tools. However I don't consider these tools at all suitable to actually details the steps to go through to execute a given test, that ...


3

What I'd do is, as many already said here, to write a bug report. In it you describe what impact it has (not only in terms of affected users but what it causes as well) and with what frequency it happens. As soon as it's described, you and your team can decide what to do next. Perhaps the business representatives, or the team, decide what priority this bug ...


2

I would likely just use Word, using a template to contain my questions. When I purchased a house earlier this year, the Inspector used Word. The resulting book looked exactly like what you are proposing for your final report.


2

As a general principle, testing is about risk/reward, which comes down to cost in terms of money/time/resources (people/machines) in various combinations. The test coverage curve will be some sort of log curve , where the most benefit in testing occurs in the initial set of tests. The first 500 tests give 62% coverage, the next 500 tests only give an extra ...


2

On some projects testers certainly can benefit from knowing the status of unit tests. 1) To initiate discussions. If unit tests have not passed before the testing / release, I would like to know why. 2) On some cases testers being able to see the status of unit tests and ask these questions has positive impact on maintaining unit tests. Sometimes testers ...


2

Acceptance testing is the final step of validating your product. There are certain advantages of acceptance testing. In fact I've got a guest post on importance of acceptance testing, hope might be useful and could be a good add to this question. Here's the link to article http://qainsights.com/what-is-acceptance-testing/


2

There are a lot of projects out there. I have used FogBugz, BugTracker, JIRA, and TFS. As for templates, I have found this very handy: As a {role}, I want to {feature/action} so that I can {value} Conditions (optional) Item 1... Item 2... Design/Dev Notes (optional) Item 1... Item 2... Acceptance Criteria ...


2

Tomasz had a great start, let me add some details and pieces of advice- Don't use Skype as a recording device, it has an inherited semi-random delay of itself. A clock is not good enough as an accurate synchronization device since your stream will probably have low(ish) variable frame rate limiting you accuracy. A good approach will be to embed timing into ...


2

User acceptance testing, by definition, must always be done manually. This is the last step of testing where end users verify that the software functions the way that they expect it to, and lay out any deficiencies prior to the software being put into production (preferably). If you are currently automating UAT, it isn't UAT but is rather a form of ...


Only top voted, non community-wiki answers of a minimum length are eligible