3

Our team is building an AWS product which uses several sets of tools like Lambda, SNS topics, EC2, data pipelines etc. I would like to know what type of testing tools/strategies that can be used to perform integration testing on these services. For an example integration of two lambdas and how they perform when they are integrated. I did an R&D and found out that for unit testing we can use AWS toolkit plugin in Eclipse but that covers only the unit testing. I want to perform an integration testing on this. Your answers are highly appreciated.

2 Answers 2

1

Let me revive this topic so others can contribute and I'll share few cents of my recent experience.

Firstly I think the important bit is to TEST the application under test - could be an API, a process or GUI hosted or supported by any of the above AWS services. Our efforts must be more centered on achieving the quality outcomes than the cool ways to access these services using code. The reason why I say that is you'll end up spending more tine trying to solve how to code to access and use those services (lambda, EC2, SBS) than the actual testing.

Quickest alternatives like AWS CLI could be explored to use these services.

For example I had a similar situation to test some of my apps hosted by these services.

I used cypress.io and used cy.exec(<aws command>) to achieve the access and usage of aws services.

Ex: cy.exec('aws s3 ls').then((data) => {
cy.log(data)
// this prints your s3 buckets in json format 
})

That was the most simplest of examples. Use them as commands and pass your object names (S3 bucket names or topic names) dynamically if you don't want to hard code.

0

AWS Data Pipeline is a web service that you can use to automate the movement and transformation of data. With AWS Data Pipeline, you can define data-driven workflows, so that tasks can be dependent on the successful completion of previous tasks. Quickest alternatives like AWS CLI could be explored to use lambda, EC2, SBS services. You can use cypress.io and cy.exec to achieve the access and usage of aws services.

Components of AWS Data Pipeline work together to manage your data:

  1. A pipeline definition specifies the business logic of your data management. For more information, see Pipeline Definition File Syntax.
  2. A pipeline schedules and runs tasks by creating Amazon EC2 instances to perform the defined work activities.
  3. Task Runner polls for tasks and then performs those tasks. For example, Task Runner could copy log files to Amazon S3 and launch Amazon EMR clusters.

AWS Glue is a managed ETL service that you control from the AWS Management Console.ETL is a three-step process: extract data from databases or other data sources, transform the data in various ways, and load that data into a destination.

In the AWS environment, data sources include S3, Aurora, Relational Database Service, DynamoDB, and EC2. Amazon Redshift is a data warehouse and S3 can be used as a data lake. ETL testing support integrations with non-AWS data sources through graphical interfaces, and offer attractive pricing models.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.