6

How do I gain the value from CI tools as an Automation Tester?

What are the advantages to using such tools and how do they help me and the company in the development process ?

Improve this question
7
  • 2
    Have you googled out anything before posting this question..? Do you know about Jenkins, ANT?
    – Bharat Mane
    Mar 30, 2018 at 7:08
  • I have googled about all the tools, however could not understand on how to use it in testing. What are the benefits for a QA
    – user15122
    Mar 30, 2018 at 7:18
  • I've updated the question according to your coment
    – Alexey R.
    Mar 30, 2018 at 10:37
  • 3
    @Peter Masiar, more like "What is the benefit of a nails gun compared to a hammer". A CI setup comes at a cost and it may not be worth it depending on the size/type of project.
    – Florent B.
    Mar 30, 2018 at 14:48
  • 1
    @FlorentB. - You have a very good point. Do you want to write answer using that? Automation with vs without CI?
    – Peter M. - stands for Monica
    Mar 30, 2018 at 15:11

5 Answers 5

Reset to default
3

Continuous Integration

Let me explain in increasing levels of detail:

A place to run your tests.

A remote server that you can push code to that runs the automated tests

Continuous integration usually refers to two parts of the development process. First when users work in branches it runs the tests for their branches. Secondly is is part of a devops pipeline so that tests run remotely and then when they pass application code can be promoted.

When you make changes to either or both of your application and automation your code you need to run the test suite to be sure that nothing is broken. As tests suites grow in size and run time length, running them on a personal laptop becomes increasingly untenable. It is a constantly changing environment that is being used for many other things. To address this you have a remote server that you put the code on. This remote server then runs those tests for you and tells you if your branch passes tests. Once this is done and the ticket has also gone through the code review process it is time to merge it in to the master branch. At this point a remote server again runs all the tests in master to make sure that the merged in code hasn't broken any tests in master.

The Value

The value gained is feedback. This is critical in any agile product. It should guide the next steps.

A close cousin is also continuous delivery which basically takes the next step and says that when the code in continuous integration is merged into master and tests pass then it should be released ('delivered') in production shortly thereafter.

Improve this answer
3

Regression testing should be automated (not manual) and CI is very valuable tool for automated regression testing.

So I assume you already have automated regression tests, and you are considering if you need CI to run them: Automation with CI vs automation without CI

Without CI, you need to remember to run your test. And when you forgot to run them, and when you finally run them much later and detect breakage:

  • Your code was broken for long time without you knowing about it. What is value of timely information about broken code?
  • You have to review more changes suspected of breaking the code, so finding the offending change takes longer, and this wasted time will accumulate. Would it be better invest time to setup CI, to save time later?
  • Because more time elapsed from the change which broke the build, developer who wrote the offending code might moved to other projects, or just forgot details and need to spend more time to get back up to speed on this issue. What is value of developer satisfaction?, allowing them to fix bugs promptly while they remember inner working of the offending code, instead being interrupted in the middle of another project to fix a bug in previous project? Or worse, someone else being assigned to fix the bug, who needs to learn it all, might understand requirements differently, and pull the code in different direction?
  • Running such test manually is boring. Humans are bad at doing the boring stuff. What is value of humans avoiding the boring stuff?

So setting up CI to run your automated regression tests is extra effort, but only you can decide if that extra effort is worth the cost.

Running test manually might be easy enough, and not worth the extra effort to setup CI. But you already likely have CI to run unit test, so adding regressions tests is easy.

Value of CI is that you don't have to remember to run the test (for the price of setting up and administering CI server). This price might be too high is your team don't have admin skills and runs other (unit) tests by hand too.

CI is yet another way to increase your productivity by automating routine manual tasks, like running regression tests.

But if above benefits seems not worth the effort, it is perfectly possible to run automated regression tests without CI.

Hat tip to @FlorentB. for distinguishing between automation with vs without CI.

Improve this answer
2

Continuous Integration gives you a reference environment in which to run your automated tests. It also allows you to have tests run automatically against the code base as soon as a change occurs which typically makes it easier to attribute the cause of any test fails by limiting the amount of causes.

Running automated tests in CI allows you to focus more on test functionality than orchestrating constant manual test rums. This ultimately allows you to cover more functionality and respond quicker to product changes.

Improve this answer
2

CI (Continuous Integration) is a process of the continuously altering the functionality to the code so that each iteration (after all this is only called "countinuous" however this process is pretty much intermittent actually) is composed of the following steps:

  • Submit code to the code repository
  • Build code
  • Deploy the build
  • Run the tests

So CI tools might include the tool set for each of those step as a single solution or as a suite of solutions.

You as an Automation Tester gain the most value from CI tools as from the framework that allows you to automate the steps described above. Depending on your role in the project that could be either the last phase of CI process or the 2nd and 3rd phases as well.

2nd phase

You must configure your build tool to inject the relevant property values to your application configuration files so that they correspond to the resources and other sepcific of your particular test environment

3rd phase

You must configure your deployment tool so that the components are deployed to proper hosts and make sure they build up a proper topology

4th phase

You must configure your either a build tool or deployment tool (depending on your particular test architecture) so that it executes the tests against the environment you have set up on the previous phases

Improve this answer
1

As a tester ,when I reach office in the morning, I get an email from CI server with overall green/ red result of regression suite as run overnight with link of detailed report where I can go & make a list(filtering out flaky test failures),simply of the things broken in last build and report it to Dev team.

As a developer , during the day (multiple times) on trying to check -in/ merging code in repo I get an email from CI when smoke suite fails (in few minutes) and my changes directly rejected as it has breaking changes which with the results of automated tests, I can fix it in next few minutes and check-in again successfully and build continues to be stable.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.