1

I have a few questions in regards to setting up/deciding automation frameworks against microservice architecture OR codebases that span various programming languages.

Specifically in this "scenario" a company is setting up automation against a suite of products that are interlinked from various services. Some of this may be older legacy code (C/C++/Java) and Desktop based applications, and newer services running as actual web applications and using something different (C#/JS(Node) with front-end libraries.

This presents an issue when it comes to choosing testing frameworks/libraries:

  1. Picking something that's compatible
  2. Not having members learn multiple different libraries/frameworks (Pretend all QA's have never done automation before)

On one hand, having the test codebase with the service/repo that it's testing against makes sense, but it makes you want to use similar libraries.

Another option is choosing something that sits outside and hit's API's/UI without having any coupling with the code (But you lose out on some functionality, such as using models/classes/functions from within the code perhaps). But you get the benefit of being language agnostic.

I realize this may seem to be opinionated, but I imagine there is a "best practice" with why and why not. Also I am curious how others have handled this scenario.

5
  • One important point that isn't clear is your testing mission and for what you want to leverage "automation". What are you trying to achieve? Apr 4, 2023 at 12:08
  • Note that the site is designed for one question at a time so users can vote on each answer to a specific question, etc. It also helps the questions and answers to be more focused. Apr 4, 2023 at 17:15
  • I also would like "Not having members learn multiple different libraries/frameworks " but in practice testing is different at different levels and requires that knowledge. The challenge then changes to explaining that to the business and how that cost is still worth the value added. Apr 12, 2023 at 11:57
  • IF its legacy system, the biggest question is what you already have for testing? First leverage that or in some way atleast take under consideration. It seems that part is totally missing here which is unusual. Apr 13, 2023 at 11:11
  • Or its a totally hypothetical question. Apr 13, 2023 at 11:12

4 Answers 4

4

I would approach it from these perspectives:

1. What kind of testing are you doing ?

Is it at the unit, integrated or end-to-end level? Is it white, black or grey box?. If it is at the unit or integrated level and you want to call existing routines in application code, this obviously lends itself to advantages from using the same language. If it is at the end-to-end level through a browser then it may make sense to use a different language. For example, currently Cypress.io is used a lot for browser testing so that means Javascipt, regardless of the backend language / frameworks.

2. What are you using for the backend language?

For example if you are using Python than you might consider python-selenium-webdriver to be a good choice for end-to-end, whereas if your backend is React (Javascript) then cypress.io (Javascript) may be a better choice.

3. What are the team dynamics ?

Are QA engineers embedded on application development teams and able to pair with fellow language developers, or are they segregated into a separate team where their pair is usually another QA person. The former makes a stronger case for sharing the same language for code.

4. What capability do you need from the testing tool?

For example for browser testing, cypress.io typically gives a much better experience than using selenium so the features and capability of the platform may outweigh the advantages of sharing a language.

5. How are you organized?

Are QA's paid as much as devs? Are QA's involved in all planning activities? Do QA's review application code for bugs? The more involved QA's are in application development, the more reward comes from sharing a language. Also, if QA's are second class citizens they will typically use the programming language learning opportunity to become a developer and get paid accordingly. Or they leave to get the same benefits elsewhere. This can led to a low quality automation codebase continually worked on by newbies.

6. Are different programming goals recognized ?

Application code and automation code share some quality characteristics but their priority can be quite different. Application code can have a focus of being performant and both memory and time efficient and may use extensive optimization to do so. Automation code on the other hand tends to focus more on readability (by treating the tests as the specifications) and how well application code is being exercised. It is important that these differences are recognized or the wrong foci will be used when programmers from one discipline pair with those from another

7. What structures are in place for pairing ?

Pairing can be a new or difficult practice and when performed by specialists from different areas such as dev and test, there are many organizational, culture and possibly geographic barriers to it succeeding organically. Also programmers may be introverted and not want to share by default. Leaders, coaches and guides are essential to make sure that collaboration happens in practice and not just 'because they use the same language'.

1

We developed WebTau to have an easy to use API to write e2e tests across UI, REST API, GraphQL, WebSocket, DB, CLI.

You can write tests in Groovy but tests look like a testing DSL, so you don't even need to know it is Groovy.

Or you can write tests in Java using JUnit.

We used Groovy tests to test micro-services written in Go, Python, Java and C++.

scenario("my bank balance") {
    Alice {
        http.get("/statement") {
            balance.shouldBe > 100
        }
    }

    Bob {
        http.get("/statement") {
            balance.shouldBe < 50
        }
    }
}
1

“You can do anything, but not everything.” - David Allen

Go for full blown Contract(API) testing as common denominator & low hanging fruit

Use API testing to target integration/contract testing between all the old & new subsystems to test the interlinking.Just focus on the different contracts between subsystems and you will go a long way with minimal effort.

WHY?

  • Its common denominator - Its based on microservices architecture so all systems/languages understand
  • Language Agnostic - easy to learn as single thing for QA
  • Higher Level- Integration Testing -assuming all subsystems will have "something" on unit level in their own language.It will be good idea to test various contracts set up between them.
  • Future Maintenance - As it will be language/UI agnostic , it would be easier to maintain in the long term even though implementation(UI/framework/language/backend/tooling) of subsystems drastically change but contracts between them remain same.
1

One language independent technique that I have used with great success is to produce a script configurable, event driven, test tool that would respond to an event such as a time elapsing or a message arrival with evaluate, log & optional message send. Since the system under test consisted of an input gateway an output gateway and an inter-process protocol it was only necessary to implement the 3 interfaces (plus a master controller) and any sub-system could either be emulated & substituted.

This allowed any of the sub-systems or any connected collection of subsystems up to the entire system to be tested in isolation by surrounding it with instances of the emulation. The master controller was responsible for starting, logging and shutting down cleanly. It also allowed debugging of the system by substituting one, or more, of the subsystems with the test harness and the, by then existing, behaviour definition(s).

Since a common format was selected for the response table definition and that format closely aligned with the way that system was specified it was quick and easy to define the expected responses to both valid and invalid events. Also any developers or testers that were working on components that were logically close to the ones that had already been addressed were able to re-use the existing response tables.

2
  • Curious of the details of this. Is this a custom tool you designed that uses a sort of messaging queue system similar to how some of the cloud messaging services work? (Like Azure Message Bus works but focused on testing)
    – Mercfh
    Apr 17, 2023 at 19:29
  • @Mercfh - I can only give generalised details as a) the IP belongs to the company that I was contracted to at the time & b) the specific project was classified. It did come about because I recognised that other than the modules communicating externally which I happened to be responsible for, all of the modules communications with each other could & should use the same protocol which could & should be generalised & reused. Prior to that everybody was inventing their own wheels and many were square. Apr 18, 2023 at 4:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.