I'm handling a code repository that is responsible to deploy / install / upgrade agent that are running on different platform, windows, unix, linux.

The code base already has some unit test to cover the behavior of important component.

I'm now thinking how I could increase the test coverage and confidence of the setup / upgrade by introducing integration testing, and integrate with travis.

What are the available test framework that I could explore on this type of application?

===== Edit #1 =====

The application I'm looking at implementing integration testing as the follow characteristic.

  • The code base is written in ruby.
  • It is a setup / installer, the main functionality to bring the state of the machine to a desired stated. For example, we would deploy a file, set the file permission or managed the content of the file if the Shasum does not match.
    • Most of the codes are platform dependent, and create other processes to perform the job. For example chmod.
  • It calls external remote APIs, to fetch and verify configuration / data.

Is there any best practice, or code base out there that does this kind of integration testing?

  • Sorry about that!
    – Yijinsei
    Commented Jun 25, 2019 at 7:36

3 Answers 3


Travis docs explain how to setup your job for multiple OS:

language: c

os: # Will run on Linux and Mac
  - linux
  - osx

  - gcc
  - clang

      - graphviz

before_install: # You can use $TRAVIS_OS_NAME to perform OS specific actions.
  - if [ "$TRAVIS_OS_NAME" = "osx" ]; then brew update          ; fi
  - if [ "$TRAVIS_OS_NAME" = "linux" ]; then apt update; fi
  • thanks of the information on Travis. I should explained that I'm looking at ways I could run code with side effect. The code base is written in ruby, and it will perform file deployments, call other remote services for validate and retrieval of configuration and has a kafka component. Im more interested on how I could have a integration test on this kind of application
    – Yijinsei
    Commented Jun 25, 2019 at 9:03
  • 3
    I would suggest editing your question to be more specific in order people can cover each point. Commented Jun 25, 2019 at 9:11

Windows and Linux are really different.

The main question that you did not address was WHY ?

You say "It is a setup / installer, the main functionality to bring the state of the machine to a desired stated" but any such state is really different on windows vs unix.

Why do you need to run this Ruby code on both windows and non-windows machines. That is not clear to me. The description "I'm handling a code repository that is responsible to deploy / install / upgrade agent that are running on different platform, windows, unix, linux" needs more explanation. What specifically is the agent? What services does it perform and how does it do that? Installers on the two operating systems are essentially completely different with different commands and approaches. It is not a 1:1 translation. The approaches and methods themselves can be vastly different.

Any sort of infrastructure creation is going to be vastly different on Windows and Unix and you have to address them separately. It's like comparing two cars - a Ford and a BMW. They both are cars, have an engine, transport people, etc. However 'one part' does not just fit both cars. There countless reasons behind this, not least capitalism. A comparison that shows how this does works in other systems can be seen in plumbing where standards mean that different manufacturers can supply pipes because they will all be standard fittings, e.g. 1/2", 1", etc.

Conclusion: No single tool will address this other than in a large "if windows, else linux" way. So just have two programs and maintain them separately. The common thread of 'install software' is not enough to bind them perhaps.

  • Thanks for the reply. I'm not looking for a single tool. I'm curious how i could increase the confidence in the code. One way I thought of, was to spin up 2 or more VM, one windows, one *nix. And run the installation on those VM. However, I would also have to address mocking the Service / remote api call that those installation might call. If i do that, that would meant i would need to setup an environment with all the service to test the setup. I'm trying to see if there are other ways to test this kind of code base.
    – Yijinsei
    Commented Jul 2, 2019 at 6:09
  • I have additional thoughts. Please consider upvoting this (and other answers) and I'll go into more detail. Commented Jul 2, 2019 at 10:21

The integration testing, as well as any testing, as a general theory, is done against some requirements. In your case you need to have some (good) architectural / design requirements, and based on these requirements you can design the integration tests strategy.

Those requirements have to present at a minimum:

  • the list of objects / modules / units / ...;
  • the clear interfaces between said units;
  • the allowed ranges for the data;
  • etc.

Once you have the requirements, you can take into consideration how to build the test cases and which tools are more suitable for the task.

Why are requirements a pre-requisite? Simply because, without them, you cannot know / prove that a(ny) test cases succeeded or failed.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.