I am working on a project to retrofit "automation" to a legacy project, it's a tricky beast, people just want to know what the progress is on it.

Is there any real way to measure automation progress, other than observing issues caught ?


For a legacy system, particularly when there isn't any lower level automation to leverage, I find it helps to start with some analysis to determine first the major functional areas of the application and how much use they receive; then repeat that analysis for sub-sections within the major areas.

This allows me to have a basic guide to approximately how much is left, and a way to ensure the most important parts of the system are covered first.

As an example, say your application is an integrated point of sale and inventory management application.

Major features would include:

  • Logon (essential, must be automated because nothing else can be done with the system without a logged on user)
  • User management (mainly used when personnel changes, can be worked around if there are problems)
  • Store/location/warehouse setup (generally set-and-forget: new stores might open once or twice in a year, and existing stores are rarely closed down completely)
  • Point of sale setup (again, mostly a set-and-forget thing)
  • Payment setup (set-and-forget)
  • On-site purchasing (will be used all the time)
  • Pre-order purchasing (will be used all the time)
  • Reporting (will be used frequently)

Login, purchasing, and pre-order would be your highest priority modules to automate, with reporting as your next priority.

Within the on-site purchase module (just to take an example) you might have such functions as:

  • Inventory updates when sales complete
  • Different product class requirements (for instance, widget A might be sold by size so you'd need a different product selection process than for widget B which has no options)
  • Product departments
  • Item types that require special handling or have related charges such as installation fees, delivery fees, etc.
  • Payment types and processing
  • Returns

And so on.

Since each application is different the user base and use profile will differ per application, so you need to do this for yourself.

Knowing what features get the most use also helps to schedule automation for new functionality - depending on how likely it is to be used, it might be necessary to prioritize automating a new feature over catching up on an existing feature.


One approach is to spend some time creating a master list of all the application functionality. The simplest approach to that may be to base the list on UI functionality. A more complicated approach is to focus on the back end, based on code, classes, methods, etc.

One you have the list - say 278 total functions then you can measure your progress in automating them, e.g. when 28 are done you are 10% through the list.

This can be done very simple with just the master list and how many are done. It is not precise however as there are many factors such as shared functionality code, different function complexity, frequency of use, data driven testing, validation verification's, etc. However it is usually better than nothing and in an Agile environment can be refined and adjusted over time.

  • Yes, a common practice of mine. q came up in MOT so posted here for others. Sep 21 '17 at 16:41

TL;DR: Code coverage

retrofit "automation" to a legacy project

What does legacy mean?

To me, legacy code is simply code without tests.

The Legacy Code Dilemma: When we change code, we should tests in place. To put tests in place, we often have to change code.

-- Michael Feathers (Working effectively with legacy code)

Michael Feathers describes some good methods how to write tests around un-testable code. Now you should refactor the classes, so you can write unit-tests them. Now you can remove most of the larger end-to-end tests again.

The retrofit word scares me as team might fall in the testing ice-cream cone trap. You will want to aim for a good balance between unit, integration and end-to-end tests. Here I think code coverage makes sense as measurement, because it shows you which branches you have not yet covered by a test. Once you have all branches covered you can start with the refactoring probably.

How do you know if your code is covered by tests? Code coverage. If you ask me I would pick branch coverage as your main metric to measure progress.

  • +1 Good answer. I'll add that "legacy" means "your current production code", a guiding phrase I heard a long time ago. Nov 26 '19 at 18:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.