There are a number of techniques that need to be combined for this sort of thing.
The tradeoff approach (aka tradeoff triangle) needs to be agreed and documented up front, with the key stakeholders. This is great way to discuss and decide something that they normally won't budge on.
The trade-off triangle conceptualizes the idea that resources, schedule, and features are three interconnected elements of any project, and that constraining or enhancing one or more of these elements requires trade-offs. An element not defined in the triangle is quality; however, it is assumed to be a constant that is clearly articulated in a quality bar by the project team at project inception.
For example, if the organization wants to incorporate a new set of features into the solution, the team will have to make a trade-off somewhere, whether in the schedule, the resources, or other features.
Given fixed ____________, we will choose a ___________ and adjust ___________ as necessary.
Given fixed schedule, we will choose how many resources and adjust the number of delivered features as necessary.
You can't test everything
Computer software is inherently complex, and even the simple act of entering a six character user name into a text box. Limiting ourselves to ASCII input still requires 127 x 127 x 127 x 127 x 127 x 127 = 4,195,872,914,689 test cases. If each test case takes us only 1 second to manually run, it will take 7.9 million years to test the user name field before we then moved onto the password textbox.
So if “complete” testing is out of the question, where should you start and how much testing you should do test before you can be confident that you can ship?
Determining what to test within a fixed schedule
Transfer your test cases them into a spreadsheet, adding columns for estimated execution time, and numerical ratings say from 1-5, (where higher is better) for the: priority (relative importance) of the test, the visibility of the area to users, the quality of the existing code etc.
The columns are then added together (in a column called “importance rating” to provide an overall score for the test. If you are testing a new release of an existing product you will probably want to add additional columns that identify the relative quality of the existing teat area.
With your spreadsheet in hand, you should circulate it and solicit input around the priority of the tests. You may even find it easier to submit the spreadsheet with a column for each area, say Testing, Development, Product Management and asking each area to fill in their own set of numbers. This will allow you to apply a weighting to each of the areas, say increasing the PM vote to 150% and decreasing the development vote to say 75%.
This is what it can look like, assuming you only had 10 minutes of testing.
With the ratings complete you can then add a column to the spreadsheet to sum the execution times. Then, sort the spreadsheet by the importance rating column, work out how long your schedule gives you to test and then draw a line across the sheet when the estimated execution time equals the time that you have available, and that is your level of test coverage.
Once all that expectaiton has been managed, and you are in-progress I then use the following metrics to track where we are
Active bug trend - If I had to chose only one metric, this would be it. I use this to manange expectations around the ship date and which bugs will be fixed in the current releease.
Feature burn down - If testing is happening, and the software looks good but new features aren't being added then testing is giving a false positve. This chart will tell me that.
Test exectuion progress - I use this to know how well we are getting work done.
Requirement completion - One key rule I use is that testers say when a feature is done, not the developers. This metric tells me how the product is progressing towards tested, shippable, actual completion.
Assigned work per person - I use this to know how well I am sharing work allocation between team members
Bug find rates - I use this to know how effective we are, I am looking for a flatish line here, well above zero every day.
With all thise metrics, I am communicating daily how we are progressing against the communicated plan and adjusting priorities as required.