4

I am currently working in a web-service testing project. The web-service engine works on a large set of input data and returns some data that undergo a lot of calculations and filtering. The testing approach we follow is : Create a parallel application that does the same functionality as the engine under test does based on the requirements of the web-service. We apply same input data to both our test application and application under test and compare the data that each of them returns.

Does any one follow any different approach ? How good do you think this approach is ?

Improve this question

4 Answers 4

Reset to default
4

Let me share my ideas,

  • I have worked on Web Services for OLTP Applicaton. This was Web Service fetching values from Database by execute few stored procedures.

  • Even for your case information would be persisited in DB / This web service must be some method for CRUD operations (Please add if you have more information on context of functionality of your web service)

  • One Approach is Fetching Test Data by Quering the backend system and verifying web services returns dataset correctly

  • Invoke the stored procedure, Invoke Web Service with same test data. Verify results look the same in both the cases

  • If there is any translation done (ex- For Country Name, Location if it is abbrevated and expanded in results) you may need to verify in web service output, this change is likely to be done in your code, may not be in Stored proc always

  • I'm not sure why you need to create a parallel application ? Why would you write another application, You can always check tools like SOAP UI, Codeplex - Web Service Studio for functional testing

  • You need to check timeouts, error handling (Error message & Error code returned by the Web Service), Cases when dependant calling methods are unavailable

  • SOAP UI supports data driven testing, You can check this feature. I have not developed Web Services Automation in java platform using TestNG. Other members can add if they have any learnings on Web Service Automation in Java Platform and TestNG framework.

Improve this answer
4
  • Lately we have started with the approach u have mentioned . We query the data and fill them up in a xls sheets and use macro to do the calculation that the web-service actually do . At the same time we send soap ui request to web service and get the result from it and compare the out put of web-service, got from soap ui with the output of macro calculation using soap ui pro.
    – Rakesh Prabhakaran
    Commented Jun 30, 2011 at 6:50
  • We started with vb macro to keep the test system simple. We have plans to replace the macro system if required, but for the time being we are planning to go ahead with the current system. I wish to hear more about different approaches to do such kind of testing . Thanks for your idea, it helps.
    – Rakesh Prabhakaran
    Commented Jun 30, 2011 at 6:50
  • I am also thinking of using testNG and java as you have mentioned , would like to here experiences on it
    – Rakesh Prabhakaran
    Commented Jun 30, 2011 at 6:52
  • 1
    Quoting from my .NET Experience, Java Experts Please fill in. One way is send httprequest to the webservice, another way is test stored procedure using Nunit. Compare both the results. In both cases you would need SQL queries to fetch data. Automation test case should be independant, rerunnable. The need for querying DB in current environment and fetch test data is the best practice to my knowledge. You can still code for Test Data Query, Test Case Execution and Compare results. All of them can be automated.
    – Siva
    Commented Jun 30, 2011 at 8:00
4

"The testing approach we follow is : Create a parallel application that does the same functionality as the engine under test does based on the requirements of the web-service. We apply same input data to both our test application and application under test and compare the data that each of them returns."

Hmm, I see where this tells you that your web service and your parallel application return the same data. But it doesn't indicate that either of them return the expected data. Who built the parallel application? How do you know that it works?

Look to your requirements, and let them guide you as to the tests that need to be performed, the input data required, and the expected results.

Improve this answer
5
  • 1
    +1 if this is what you are doing then the person that wrote the parallel application should just write the main application, cause obviously he's done it correctly! Then you wouldn't need to test it :)
    – stuartf
    Commented Jun 30, 2011 at 13:38
  • Stuart i dont understand what is so funny about it. Our test team created the parallel application described above which is created in xls vba macro , which fetches data given in the xls sheets and does the calculation based on the requirements of the webservice. Our test team who created the parallel application is completely independent on the development team who created the webservice. The test team also do through unit tests on the vbs code we create for the test application .
    – Rakesh Prabhakaran
    Commented Jun 30, 2011 at 16:23
  • I believe that for such a system who handles large amount of data like ours really need such a testing environment. If you find this not right way then please give more suggestions.
    – Rakesh Prabhakaran
    Commented Jun 30, 2011 at 16:23
  • Rakesh - if you need to validate large amounts of data exhaustively for some reason, your parallel application is a perfectly reasonable approach toward that. But is "large amount of data" important here? Usually when I'm testing a data delivery mechanism like a web service, I use test data. Often a fairly small amount of test data will suffice, and a parallel delivery mechanism isn't necessary.
    – Joe Strazzere
    Commented Jun 30, 2011 at 20:07
  • @RakeshPrabhakaran how do you know this parallel app your team has created works as expected?
    – dzieciou
    Commented Jan 13, 2016 at 14:48
3

A test creates some inputs, supplies them to a function, retrieves the results, and finally consults an oracle to determine whether the results are correct. Depending on the function, any of those actions may be easy or difficult. You could argue that the design you described is simply a way to distribute the oracle between the test client and the parallel application. If you need to calculate expected results somewhere, why not in parallel application?

For a QA person who wants to be a developer job someday, it may be more interesting and fun to code the oracle as a web application. The design will give the QA person the opportunity to practice their coding and design skills, and perhaps even to prove their coding abilities to a development manager.

Nonetheless, I wonder about the wisdom of what you described. Your parallel application does more than just compute a correct answer; it must also behave like a web service. For example, before your parallel application computes a correct answer, it must listen for requests, parse a request, and perhaps fetch additional data from other sources like a relational database. And after computing a correct answer, the parallel application must encode the answer in a response and send it back to the client. If you really want to test the application, you must use entirely different software for all these operations; otherwise may miss bugs because your parallel application will have the same bug as your real application.

I suspect that the alternative -- to embed the oracle in the test client -- will be simpler. Simpler software is usually easier to understand and maintain and is less likely to have bugs. Perhaps you should consider that type of design instead.

Improve this answer
2

Looking at both your question and the comments in Siva's answer, I believe that you're on a good start. I would recommend keeping your current SOAPui tests and vb macro for on the fly tests. Sometimes having a valid way of performing a manual test is worth all of the automation in the world.

That being said however, I agree with Siva that this is a process that could be easily automated. Have automation suite send request to web service (webrequest in .NET, I believe HttpUrlConnection in Java), parse the response, perform the logic that is in your current macro, and compare.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.