1

I am testing a microservice that is grabbing logs from an Azure Log Analytics (LA) resource and returning them as a digested response or placing them into a database.

My current solution is using MS' data-collector-api to inject logs into a real LA resource, wait until my injected logs have been consumed by LA, and then call the microservice endpoint to consume the injected logs, finally verifying that the output of the microservice is in line with what is expected. The SLA on LA is about 7 minutes, but can be much longer, and the data-collector-api is still in preview with some bugs around guid values.

Is there a better way to be testing this transaction of data?

4
  • Why you are injecting logs each time when the role of microservice is just to consume the existing log ?
    – PDHide
    Dec 23 '20 at 5:08
  • 1
    Unique data is generated for each run and then removed (if possible) after the run finishes
    – st0ve
    Dec 23 '20 at 5:27
  • @stove , but the scope of microservice is in consuming the data. It would be better to have a static versatile log in azure always. And check whether ur service is able to access this and validate it cannot access it if azure is unavailable (To ensure microservice is not displaying cached information)
    – PDHide
    Dec 24 '20 at 3:33
  • 1
    It then becomes a task of scheduling another service to periodically refresh these logs though. And the. Updating the tests to point to a different test set. As I'm sure you are aware, logs in a log about it resource do not last forever (90 days retention for cheapest policy). Therefore the least brittle approach is to allow tests to produce and "consume" the test data themselves. Self sustaining test data
    – st0ve
    Dec 24 '20 at 3:59
1

From my point of view your tests are wearing too many hats. If I understood correctly, your tests are doing / checking all the following:

  1. creating logs on LA
  2. connecting to LA
  3. grabbing the logs
  4. "digest"
  5. validate

Judging by your provided microservice description, points #1, #2, #3, besides being very time consuming to perform each and every time for every test / test suite, they are also are kind of useless as they do not seem to be within the scope of the said microservice.

Possible alternatives:

  1. Contract Testing This way you can make sure that any changes in your service will not impact the reading of logs from Azure LA.
  2. Have a separate job that runs periodically to create and add a basic log to Azure LA (since you mentioned the time constraint of 90 days, then having this separate job run once a month should do the trick) and then have some specific tests that only validate the retrieval and maybe integrity of logs.

Regardless of the above approach, if the purpose of the service is to digest/interpret/transform the logs retrieved, this should be done at the service level. An approach would be for your tests to generate the random data you mentioned and save it on the disk/in memory and then the service to read it from there and validate whatever is required.

The approach of having more atomic tests should considerably speed up your testing while also allowing you to use resources more efficient and get rid of any constraints Azure LA may impose (if any).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.