Seriously struggling to understand "integration" testing

Question

I am seriously struggling to wrap my head around this one, so its best to describe it. I have been tasked to setup automation at my current company and thus far I have been building out the framework robustly to handle anything I start throwing at it in terms of reporting, logging etc. my first task is the automated testing of a c# webservice (My Auto is in java).

Lets say, this service has 2 controllers foo and bar.

foo contains around 16 calls in the controller

bar contains around 14 calls in the controller

There are a large number of unit tests in this webservice already and im seriously struggling to figure out what integration tests may look like.

To my understanding the unit tests will mock everything, should i be almost duplicating those tests because the webservice is deployed on the real server and running as part of the application (utilised by the front end)?

Should I be testing the webservice calls for all sorts of input on the data?

OR, should i be writing a handful of tests to flow through the API to ensure its working correctly? Right now I have hundreds of input related tests checking for all sorts of data input which I feel is unnecessary, I am testing each call atomically for input and scenarios like deletion, adding a new, duplication etc.

I can't really grasp the concept of the integration and what is expected, at this rate I will have over 400+ tests for all ~30 calls and I feel its completely wrong.

any advice is greatly appreciated.

Answer 1

Let's start from your initial mandate. "Automated testing" can be a pretty vast land to explore. Leave the unit tests as they are for now and focus on the doubts you are having for integration tests. I presume here that you have none.

By looking at Wikipedia's definition of "integration testing":

Integration testing (sometimes called integration and testing, abbreviated I&T) is the phase in software testing in which individual software modules are combined and tested as a group. It occurs after unit testing and before validation testing. Integration testing takes as its input modules that have been unit tested, groups them in larger aggregates, applies tests defined in an integration test plan to those aggregates, and delivers as its output the integrated system ready for system testing.

You need to understand the following keywords:

1. Combined software modules
2. Tested as a group
3. Occurs after unit testing
4. Output ... ready for system testing

From this definition, applied to your context, you can understand that integration tests won't require mocking and can be combined with other webservices that make the functionnality work. So everything needs to happen when your webservice is running. You will need to generate code somewhere that can make calls to your service the same way it would happen in real life.

Even the above definition can be wrong for some people. What I usually do for my services is to get from the product owner or the people paid to know what this service is used for to tell me its purpose in the system.

Once I know that, I prepare potential positive (happy scenarios) calls to that service that would occur in real life. These calls come with their expectations that will become part of the integration test suite.

Then I fill the holes with negative call scenarios that could also happen and the service needs to react according to your expectation. This will again be part of your integration test suite.

But this is only my opinion. Before you go forward, make sure you understand what the service has to do in your system. Once you know that, it will be clear what needs to be tested.

Answer 2

Some people distinguish between

• integration (some parts are used for real, some are mocked) and
• system/end-to-end testing (where is no mocking at all).

For other people, system, integration , and end-to-end are synonyms.

Usually end-to-end testing uses just basic scenarios (Pareto principle: 20% of use cases cover 80% of functionality), and unit testing tries to cover also more of the corner cases.

For e2e, testing just basic functionality is OK. Cover ALL corner cases in unit tests. If you have more time for e2e, start adding more common corner cases.

Unit tests are more useful to detect where exactly the error is (so covering all corner cases makes more sense, and is easier with mocking).

Answer 3

If this was my piece of work, I'd be firstly focusing on the API. To test the API, look at SoapUI, JMeter (I know this is primarily used for load but you can test an API with it), Postman or even your own home baked solution in Java.

I'd firstly look at testing the parameters for a given service. E.g. if a login API, make sure you get a meaningful error for no username, invalid credentials etc. etc. As mentioned elsewhere, you'd be thinking, "thats's hundreds of combinations". Peter Masiar's point around test most common first then edge cases as time permits. In terms of your test, again using the Login API as an example, in SoapUi, I'd created a single data-driven test. I'd then put my scenarios into Excel. The data driven test calls the Login API for each row in the spreadsheet. In the Spreadsheet, you can put in your params as well a the expected message to be returned....

Test Id Test Description            Username    Password    Expected message
1       No credentials supplied.                            No user or password supplied
2       No username                             Bar Invalid credentials supplied
3       No password                 Foo                     Invalid credentials supplied
4       Wrong password              Foo         Barrr       Invalid credentials supplied
5       Valid credentials           Foo         Bar         Login Successful

The beauty of the data driven test is you minimise the number of 'tests' in your suite, but get the benefit of testing lots of combinations. Plus, when you think of a new edge case, e.g. invalid chars in username, then you just add a row to the spreadsheet.

Once you have tested a web service for positive and neagative values, then think about use cases. E.g. saveCourse and getCourse. In a single test, I'd create a course and then with the values I supplied to saveCourse, I'd attempt a getCourse.

Once you have confidence in the API and the GUI is less likely to change, then I'd look at automating the GUI....