Hot answers tagged

27

Fibonacci series is just one example for estimation efforts. Some teams also use series as below: 1, 2, 5, 8, 20, 40, 100, .... 1, 2, 4, 8, 16, 32, 64, .... The idea is to use an exponential scale for estimating efforts. The reason is the larger the story point, the more uncertainty there is around it and the less accurate the estimate will be.


22

They reflect that the degree of uncertainty grows as you look further out and at bigger tasks with more dependencies. For example, today you can be reasonably confident about how much effort is needed for a small task. You may be highly confident that you can finish it within a day, and, critically there is little uncertainty about the factors involved. So ...


11

No It absolutely is not valuable to measure bug count per hour of dev time. It is especially bad to measure bug count per hour for individual devs. Devs on more complex or difficult code will generally have more bugs/hour than devs working on cleaner, more straightforward code. Similarly, devs will produce more bugs/hour at the end of a long workday than ...


8

It depends, and there are no industry standards. Seriously. Any metric can be gamed (and will be, if you use it for assessment). I'm not aware of any standard approaches, not least because the teams are - or should be - evaluating themselves regularly and looking for ways to improve their own processes (if they aren't then they're probably using SCRUM-but......


7

This is process related - nothing you can solve as regular QA tester. Your QA manager need to talk to DEV manager how to improve communication between devs, QA and customers, and how to track relevant info to gradually improve your whole process. Bug tracker is just one part of it. But as QA, you cannot make developer enter a bug to tracker, or do anything - ...


6

By the success of the company. Buy-in for QA will need to come from the top rather then the justification being looked for in data. You will, over time, be able to point to things like some major bugs prevented from reaching customers performance issues anticipated and planned for unusual bugs discovered for certain conditions more new customers more ...


6

We thought of some sort of limited coverage measurement - checking if collaboration points like app to database, app to external service are covered by tests. Most code coverage tools allow filtering by file name or location; you can create multiple reports or checks for certain areas of your codebase. Here is how Jacoco does Naturally, these would not have ...


5

Common estimating methods include numeric sizing as well like 1 to 10 or sizes like XS, S, M, L, XL, XXL, XXXL or Fibonacci sequence 1, 2, 3, 5, 8, 13, 21, 34, etc. The reason for using the Fibonacci sequence is to reflect the uncertainty in estimating larger items. A high estimate usually means that the story is not well understood in detail or should be ...


5

First of all, you need to define better what you want to measure. Imagine a sports team. One could say that "improving" means "winning more games" - and one could say that a software team is "improving" when the product is making more money or has a bigger user base. However, in both cases, one can break down what makes the team ...


4

The deliverable is information about the system. The purpose is to help people make better decisions, based on information about the system.


4

Coverage is always coverage related to some model. This often gets skipped over, which leads to much confusion "you said you had 100% coverage so how come there's a bug?!" When you're looking at unit tests, then it's possible to use code coverage as an indicator (there are tools that can measure what percentage of the lines in your code are exercised when ...


4

Agile teams are cross-functional teams. They estimate the work from design to delivery often on a story based level. This includes the testing work, since testing should be part of the definition of done. Story points are the relative size of the complexity of the task at hand. Read more about estimating with relative sizes in this blog. I as a tester part ...


4

Test Efficiency is a measure of the relevance of the bugs being reported. A low efficiency would imply that the test team are reporting many bugs that aren't worth fixing. This is pretty limited, and simplistic. I've had better success with a pie chart that shows the "resolution" for all of the resolved bugs. This shows the resolution categories like "...


4

Black box approach in and of itself is specifically designed to come from the user perspective backwards. The difference between black box and white box testing is knowledge of the underlying code and components. Therefore when you are prepping for black box testing you should be coming from the user perspective who utilize the application. This is any ...


4

As Kate says NO The common term for kind of metric from those with experience is: A bad metric I'll go even further and say any kind of bug count is likely to be gamed The industry has done thngs like this is the past, e.g. 'lines of code to measure productiviuty'. You can imagine where that leads! Although maybe how few lines would be good? NO! That would ...


4

Spoiler: No scientific reason. Fibonacci grows very fast, so people will have fewer options before reaching enormous values; thus it incentivizes breaking work down in smaller pieces. If the smallest typical work takes 1 hour, a big piece may take 8, 9, 10, ..., 16, ... even 32 hours. However, if the smallest piece of work takes 1 story point, and they ...


4

A notion I don't see in any of these answers is that in a simple 1-10 range, people can get bogged down in whether something is a 3 or is it really a 4? And what if another person thinks that it should be a 2 instead of a 3? By using a Fibonacci sequence, you eliminate a bit of that "hair splitting".


3

Testing efficiency, among other metrics, is a merely a guideline. It does not tell the whole picture, it has to be put into context in order to make some sense. It is more of an indication used by management when they make an assessment. E.g. when a senior manager asks a project lead, "Recently we have put some investment into fixing defects, how does it ...


3

There are the following types of test coverage criteria: structural criteria data criteria fault-model criteria requirements-based criteria criteria based on explicit test case specifications criteria based on statistical methods for random test data generation criteria based on mutation-analysis All criteria except the first one are non-structural. ...


3

I would suggest you look at their Device Fragmentation Scope of testing, (white box / black box) test aspects, App store testing, versioning, Security testing, memory usage and usage of best practices (mind maps, automated tools, carriers etc..) Tools used / Automation Efforts Previous projects in similar domain / expertise Meaningful Measure of Number of ...


3

I'm a workmanlike Tester who's had the good fortune to work with some excellent people over the years. When I hire people, there are some key characteristics I look for: Eye for detail This is the obvious one. Testers need to be able to see the little details that others miss. You know those Facebook questions? The ones that go: Spot the mitsake : 1 ...


3

Telemetry can't always pin point problems, but many times it can indicate the existence of a problem. If you expect some problems to occur you can sometimes add smarter telemetry and better analysis of other pieces of information, for example users skipping the Delete button and closing the application all together. A complementary approach is using A/B ...


3

How many functional test cases have been automated - Regression Automation coverage by different teams. first you must have a requirements traceability matrix to check if all your test cases cover all your requirements, then you may proceed with this. BUT, this doesn't say anything about the performance of a QA How much increase in Automation coverage has ...


3

Posted about this just recently. Reference to the original is at the bottom. Key Idea Suppose you want to know how long a task will take. You take a look at the problem and estimate that it will be one hour's worth of difficulty/complexity. At the end of the hour, it's not done. You've realized something about the problem and it's as if you are starting over....


3

This question is answered in a blog post by Jeff Sutherland (co-creator of Scrum). It's rooted in a US Department of Defense study on estimation. Rand researchers then studied the effect of the numbers estimators can choose and found a linear sequence gave worse estimates than an exponentially increasing set of numbers. There are some recent mathematical ...


3

Can you detail why you need those metrics? It sounds like you just want to have metrics - but do you have any plan with them afterwards? Personally, I'd first focus on getting the integration tests on point on a story-by-story basis. That is, during implementation, have the developer and tester sit down to discuss what is being unit tested, which cases ...


2

The problem with any form of hard metric in a field like software QA is that - to use the answer I give so often here - it depends. Each hard data point is really only helpful in the context of the development project it links to, and even then there are variations. I'd suggest you take a look at the answers to the question user867 suggested as a possible ...


2

I would give them a percentage number of test cases automated of all test cases that can be automated. The key here is to develop the full list of test cases that "can" be automated. What test case management tool do you use?


2

Sonar works well for this. We've been using it for about a year on a suite of products written in .NET. Monitoring metrics for individual tests is outside of the normal use case, but I believe it can be done if that's what you want. It's also quite straightforward to add your own plugin to work with custom metrics if you decide you need to do that.


2

I use HP LoadRunner(fyi) so I have 90th percentile, but the question does not change. I explain to managers consuming these reports: The Std. Deviation number should be low. If your test is fairly consistent in load you want to see this number low because it means most response times are close in number. The Mean is great but only accounts for what 50% ...


Only top voted, non community-wiki answers of a minimum length are eligible