Adding tests to a half-finished project

Question

Background: We are a small dev team in a larger org. We have no testers allocated to us. We have had our backs against a wall for five months developing something as quickly as possible. In that time, we've conducted manual unit tests and our (React web-based) application works well, considering.

Situation: We're on the verge of deploying to a UAT environment that will more closely mimic Production. Before we do that, management has asked me to see what we can do to get some initial testing done before we put this in the hands of users. We don't want technical issues to slow down UAT and dilute its findings. That is, we don't want UAT to catch basic technical issues we should have caught first, or to frustrate our UATesters unnecessarily.

The future: This application is going to grow rapidly as we expand it from a small subset of our client base to a much larger set. Given that we know it will grow, management is keen for the testing to get more automated as soon as possible, to save effort (cost) down the line as we add the features necessary for that expansion.

The question: We have already spent three devs x five months writing this, so there's quite a large codebase and probably a lot to refactor to get it in a maximally-testable state. I just don't know where to start. I've looked at getting Selenium and doing some UI-level testing since we can get some value there without rework. I'm also thinking of going back to management and explaining that we can't do much more than a token effort without spending a considerable amount of time during which we will likely pause development.

So! How do we begin to add some automated testing to a project that's nearly ready for deployment and which was unfortunately not built with testability as a priority?

Additional app details that may or may not be relevant to your answers:

• Built with React & Redux
• Draws data from various third party applications we use to store client information, systems information, etc.
• Some jsp thrown in where we want things done serverside for security.
• Some Java for the authentication

Answer 1

Follow Test Pyramid and apply 80/20 rule:

• 80% of tests are low-level unit tests
• 16% are API tests (services) (16% is 80% of the reminder)
• 4% are UI tests

Start with manually running unit tests. Add CI for unit test, then services and UI tests as you have time. Jenkins is commonly recommended as test runner for CI.

See recently asked:

• Tests in CI: Are API + Integration Tests the most suitable?
• Coping with an undersized QA team

You have accumulated a lot of technical debt, and you will have to pay it all, with interest. Let your management to know it. Part of debt would be to build the instrumentation pipeline and CI.

IOW: you will not be able to continue to advance (add new features) with your current speed anymore, because part of your effort will be spent in non-user-facing efforts, like CI, unit tests. You will slow down for a while (to pay down the debt) and then you will increase speed slightly (when technical debt is covered).

Unit tests and service-level tests are in your main development language.

Often, UI level test is done in JavaScript, because UI is in Angular (or another JS web GUI frameworks of the month). We (in our project) decided that because our main core language for core app is Python, and we have better skills in Python, we do most of UI testing in Python too. You need to decide where your team's skills are.

Answer 2

"We have already spent three devs x five months writing this, so there's quite a large codebase and probably a lot to refactor to get it in a maximally-testable state. I just don't know where to start. I've looked at getting Selenium and doing some UI-level testing since we can get some value there without rework. I'm also thinking of going back to management and explaining that we can't do much more than a token effort without spending a considerable amount of time during which we will likely pause development."

Within this there are several key indicators that show your problem:

"Probably a lot to refactor to get it in a maximally-testable state". Forget about "maximally-testable". Write one single simple test. iterate. This is one of the common failures in testing. Don't try to write really smart tests, just some some tests. then refactor. then iterate.

"we can't do more than a token effort". Not true. The first test you write will often add huge value, with each subsequent test adding less value. Seeing the first test as "a token effort" is a problem with how you view testing and getting started, noit a problem with tests adding value. They do. We know that.

Also, adding tests now - but not changing the way you write code will mean this is a temporary fix. Development needs to use TDD and/or BDD to change the way that development is done. Whether or not you spend time writing tests for the current trechnical debt or not, changing how you develop going forward is required.

With unit tests, you should start with quick wins so that

• The team gets used to the process
• The team sees instant feedback
• The team has something to work with when it comes time to add CI to automate the testing

Also, stay away from the advice to "automate the frequently used parts first". After all, if it's frequently used, you'll be testing it through the natural course of development as part of testing other stuff. Instead, go for areas that change a lot, or are historically prone to break often. You'll get more mileage with the time spent creating tests.

Answer 3

My friend when you find an answer for this please please publish it because many many teams around the world face this situation.

My recommendation based on my experience with a seven years running project with no automation at all

Start with unit testing and CI.

For system testing

1. Prioritize your features based on three aspects: usage, impact and how easy they are to automate.
2. Start automating the ones that are most used, have most impact and are easier to automate. In that way you'll be able to provide a lot of value with the least effort. Probably you'll have here features that have APIs behind our way to automate UIs.
3. Continue with the ones that are most used, have most impact and are harder to automate. This ones will slow down the speed of your team but these are features you don't want ever to fail. Probably a lot of UI automation here.
4. For the other ones create a backlog so you can add at least some effort each week to automate something new.

I'm my experience you'll have to do a lot of rework in point three, but it'll pay off with time.

Also, non functional testing seems to be a big deal for you because you expect this to scale rapidly so you need to spend time with load testing and stress testing.

One last thing identify all potential risks and sit down with your management so that they understand what could go wrong. Make them accountable for going live with those risks.

Let us know what works for you (y)

Answer 4

Unfortunately, this is a situation that's far too common. I've been there more times than I care to think about, sometimes with software that's not able to be unit-tested (Classic ASP web application, with everything intertwined, and no, it's not being retired any time soon).

I'd suggest you start with the approach @edwin-jaime suggests as a way to prioritize which tests to write.

For the technical aspect of turning code written without testability in mind into code that's actually testable, I've found using something like the following steps works, as long as you have and can keep management support for the effort.

1. Write bad unit tests - Initially, you're probably not going to be able to isolate dependencies and build in workable mocks or fakes. Don't worry about that so much as long as you can write something that tests the functionality of your key units of code. Ideally you'll want one unit test per potential valid outcome, but you can start with a single catch-all that covers all valid outcomes. This gets a quick-and-dirty base set of tests validating your business logic.
2. Isolate dependencies in the application code and test code - Start placing data sources behind interfaces and abstracting display code if you haven't already done this. As you make the changes, make corresponding changes to your test code, so that by the time you've got your key code units loosely coupled, your tests are also loosely-coupled and therefore much less likely to be flaky.
3. Break up multiple-responsibility units of code - once you've got your presentation, business, and data interfaces set up, you can start breaking up any code units that do more than one thing. You'll start failing tests as soon as you do this, so update your tests as you go, until you've got SOLID, DRY code.
4. Repeat any time you edit code - Once you have the key areas clean, start using the same process any time you need to be in the code for any reason. This way, you'll gradually work through your code base while you're working on bug fixes or enhancements, building a cleaner, more testable code base as you go.
5. Don't fall into the trap a second time - Make sure you code for testability and write unit tests as you go for any new code you're writing, and you won't run into this problem again.

Of course, all of this requires that your management accept that there will be a slower pace of new feature development for some time until you've built a viable set of unit tests and then higher level tests. The length of time it will take will depend on how big the code base is to start with and how messy it is.

The payoff will come with future changes where you will find out immediately if you accidentally break any code that's got unit tests around it. It will also come in the continuous integration that covers the tests and the knowledge that future testers won't need to focus so much on the core application logic and can spend more time looking into flow through the system and edge cases that can be problematic.

Answer 5

On top of other excellent answers, I would suggest for React/Redux, focus primarily on Client side unit testing with Enzyme(as a testing utility) and mocha(BDD test framework) with chai (assertion library).