I need to continuously monitor my website's quality, I already have some tests integrated in a CI, but I need to do some checks every x time, what the best strategies on this ? I'm using selenium, would it be a good idea to have an infinite loop that does this ?
2
-
What CI system are you using? Do you have an example of what quality you would like to assess in production?– Niels van ReijmersdalCommented Jul 21 at 11:36
-
@NielsvanReijmersdal The tests in my CI use selenium to check for various web components– Moad EnnagiCommented Jul 22 at 9:45
Add a comment
|
2 Answers
Most CI/CD tools support scheduled jobs, you could build a short pipeline that runs tests against the production environment on regular intervals (seconds, minutes, hours):
- Gitlab: https://docs.gitlab.com/ee/ci/pipelines/schedules.html
- Jenkins: https://plugins.jenkins.io/schedule-build/
Depending on what you want to check in production I would consider 'health monitoring' instead. Maybe start with:
- Response times (ping)
- Uptime
- CPU load
- Database load
Here is something you could self host: https://healthchecks.io/docs/self_hosted/
Be careful that your 'tests' don't hammer the production environment and reduce the experience of real users. Wonder if you could somehow make sure the quality is tested in a staging environment so you can trust the production functionality to just work.
Further reads:
answered Jul 21 at 11:36
Goal: To perform Continuous testing: The process of incorporating automated feedback at different stages of the software development life cycle
would it be a good idea to have an infinite loop that does this ?
Probably not, as you would be doing the same checks multiple. To exemplify:
Let's say that at 10:00 AM you have the website in a certain state, frontend at version 15 and the two services at version 32 and 58; and your automated checks take 15 minutes to execute. If by the end of the day you end up releasing 2 new versions of each service and one new version of the frontend, most of your executions will be re-executing the same checks against the same software setup: A waste of money and creation of noise with alerting.
What can you do:
1 - First, specify your automated checks. To have suites for each piece of software you release as a package. In the example above, a suite for the frontend and one suite for each service. I suggest these suites to follow Toby Clemson's Testing Strategies in a Microservice Architecture (even if you don't have microservices - a service is a service) for have a good coverage.
2 - For the execution, you can associate the execution of these suites with the build process of the packages you release. When a new version of frontend is created, you execute the suite for the frontend; when a new version of a service is created, you execute the suite for that service.
This way, you will have executed checks for each version of the software you produce, but also don't over-do it, saving resources (aka money).