I have a data migration project going on. There's a semantic mapping tool in the middle to translate between two DBs. I'm testing that.

However, some issues that I find are fairly...blocking. E.g, one that causes the wrong number of values to appear in every column in the new database. Fairly big.

As a result, any other tests I write that get masked by this issue I can't know if the test is right yet, or if once the blocking issue is fixed we may see more issues, or whether everything will pass.

The project manager wants to know progress and when I said it's tricky to tell because of this massively blocking issue, says "put it this way, how many tests have you written?".

To me this is a horribly inaccurate measure, as the tests could be wrong, or new bugs could cause new tests to need to be written, or we may run into even further blockers, let alone the fact that the tests will need to be re-run.

He finally insisted that I tell him how many tests I've written, at which I could say "80%, but..." and he said "well there's a metric!". I tried pointing out why it's not a realistic one to measure progress though.

Are there more legitimate ways to report progress to project managers when you have big blocking issues potentially masking further problems?

  • You mentioned "I can't know if the test is right yet" and "new bugs could cause new tests to need to be written". To me, this points to a lack of planning on the PM's part. If you're not sure if a test is correct, get clarification from the PM or dev. If you're unsure of some of the specifics on how the migration should happen (which could call for more tests or invalidate tests), get specifics from the PM or dev. You should be able to write nearly all of your tests with no code written. If not, your project is sorely lacking in planning and that is on the PM, not you.
    – Sam Woods
    Commented Jul 5, 2012 at 16:26
  • To clarify, it's an automated test. So it could be buggy. But it's hard to debug when there are so many bugs in the way. So there could well be a problem with the test that will be uncovered when some of the fixes are complete. The documented tests are written - I did a detailed test plan at the start and got them signed off.
    – Mark Mayo
    Commented Jul 5, 2012 at 16:29
  • You say "I tried pointing out why it's not a realistic one to measure progress...". What response did you get?
    – user246
    Commented Jul 5, 2012 at 16:43
  • Not much response, he just wanted a number :/
    – Mark Mayo
    Commented Jul 5, 2012 at 16:55

4 Answers 4


You need to meet your project manager half-way. Using an unrealistic metric does not help either of you, and you are right to be concerned about that. At the same time, you need to avoid describing your problem in so much detail that your project manager will not understand it. Given what you wrote, I think I would tell the project manager something like, "If you want an accurate status that you can measure against your schedule, you need to break down my work into N parts (where N is preferably a very small number): design, code, debug, etc. Here is where I am against those N parts. And here are some risks to my continuing to make progress."

If you already did that, and you are not satisfied with their response, you should amend your question with that information.


The metrics can be divided so that there is reportable status. 1) List of test conditions (i.e., test cases). Do you believe that you have a complete list written down? If so there is one metric (identification of the test conditions). 2) From the list of test cases, how many are completely fleshed out as far as what needs to be done to run the test? Your blocking issue probably prevents documentation of adequate detail on many of the cases you've identified. This would be a second metric (development of test case steps). 3) Test execution would be a 3rd metric. What %, if any, have been executed and passed. If there are none that wouldn't need to be re-executed after the blocking issue is fixed, this part is 0% complete. If there are some tests which aren't impacted by the blocking issue, you "may" be able to count those as completed.

  • 1
    You may want to identify approximate percentages for how long each of these phases takes (helps to have historical data) and an estimation of how much time the total task would take (again, some historical data helps a lot). If you can mine any data about how long small vs. medium vs. large projects took in this past, that's a good place to start. Then you can make a prediction on how the current project stacks up against known projects. It's just a prediction, it's not a commitment. Commented Jul 5, 2012 at 17:29

Project managers want to track the overall progress by a % complete number. Here is a way which works (depends on your organization).

You could provide a report that goes something like this:

Total Tests Written - #count
Total Tests Executed / Tried - #count
Total Tests Passed / Found to be Working - #count
Total Tests yet to be Checked / Executed - #count
Total Tests that cannot be Checked - #count

In this way the PM can find the % that if found to be truly working and the uncertainty % to assess the potential outstanding risk.


Sometimes you will find non-technical project managers and to make the matters worse they work against metrics rather than common sense. However, his question is legitimate to some extent. Given a feature, you could always come up with N number of tests (after applying equivalence class partitioning, pairwise etc) and then you can group them as functional (positive, negative) and non-functional (internationalization, localization, security etc). Once you have covered all these areas then you can claim that the tests are complete. And then you can say that the execution is only 10% complete as there is a blocking issue and it is not worth continuing the test.

  • 1
    +1, although I would add that you may be able to continue executing some of the tests (maybe partially) even with the blocking issue to try to discover any defects as soon as possible.
    – Sam Woods
    Commented Jul 5, 2012 at 16:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.