How do I gain the value from CI tools as an Automation Tester?
What are the advantages to using such tools and how do they help me and the company in the development process ?
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It only takes a minute to sign up.Sign up to join this community
Let me explain in increasing levels of detail:
A place to run your tests.
A remote server that you can push code to that runs the automated tests
Continuous integration usually refers to two parts of the development process. First when users work in branches it runs the tests for their branches. Secondly is is part of a devops pipeline so that tests run remotely and then when they pass application code can be promoted.
When you make changes to either or both of your application and automation your code you need to run the test suite to be sure that nothing is broken. As tests suites grow in size and run time length, running them on a personal laptop becomes increasingly untenable. It is a constantly changing environment that is being used for many other things. To address this you have a remote server that you put the code on. This remote server then runs those tests for you and tells you if your branch passes tests. Once this is done and the ticket has also gone through the code review process it is time to merge it in to the master branch. At this point a remote server again runs all the tests in master to make sure that the merged in code hasn't broken any tests in master.
The value gained is feedback. This is critical in any agile product. It should guide the next steps.
A close cousin is also continuous delivery which basically takes the next step and says that when the code in continuous integration is merged into master and tests pass then it should be released ('delivered') in production shortly thereafter.
Regression testing should be automated (not manual) and CI is very valuable tool for automated regression testing.
So I assume you already have automated regression tests, and you are considering if you need CI to run them: Automation with CI vs automation without CI
Without CI, you need to remember to run your test. And when you forgot to run them, and when you finally run them much later and detect breakage:
So setting up CI to run your automated regression tests is extra effort, but only you can decide if that extra effort is worth the cost.
Running test manually might be easy enough, and not worth the extra effort to setup CI. But you already likely have CI to run unit test, so adding regressions tests is easy.
Value of CI is that you don't have to remember to run the test (for the price of setting up and administering CI server). This price might be too high is your team don't have admin skills and runs other (unit) tests by hand too.
CI is yet another way to increase your productivity by automating routine manual tasks, like running regression tests.
But if above benefits seems not worth the effort, it is perfectly possible to run automated regression tests without CI.
Hat tip to @FlorentB. for distinguishing between automation with vs without CI.
Continuous Integration gives you a reference environment in which to run your automated tests. It also allows you to have tests run automatically against the code base as soon as a change occurs which typically makes it easier to attribute the cause of any test fails by limiting the amount of causes.
Running automated tests in CI allows you to focus more on test functionality than orchestrating constant manual test rums. This ultimately allows you to cover more functionality and respond quicker to product changes.
CI (Continuous Integration) is a process of the continuously altering the functionality to the code so that each iteration (after all this is only called "countinuous" however this process is pretty much intermittent actually) is composed of the following steps:
So CI tools might include the tool set for each of those step as a single solution or as a suite of solutions.
You as an Automation Tester gain the most value from CI tools as from the framework that allows you to automate the steps described above. Depending on your role in the project that could be either the last phase of CI process or the 2nd and 3rd phases as well.
You must configure your build tool to inject the relevant property values to your application configuration files so that they correspond to the resources and other sepcific of your particular test environment
You must configure your deployment tool so that the components are deployed to proper hosts and make sure they build up a proper topology
You must configure your either a build tool or deployment tool (depending on your particular test architecture) so that it executes the tests against the environment you have set up on the previous phases
As a tester ,when I reach office in the morning, I get an email from CI server with overall green/ red result of regression suite as run overnight with link of detailed report where I can go & make a list(filtering out flaky test failures),simply of the things broken in last build and report it to Dev team.
As a developer , during the day (multiple times) on trying to check -in/ merging code in repo I get an email from CI when smoke suite fails (in few minutes) and my changes directly rejected as it has breaking changes which with the results of automated tests, I can fix it in next few minutes and check-in again successfully and build continues to be stable.