10
votes
Accepted
Do you still need a Test Plan when adopting Continuous Delivery?
I don't see CI/CD as something different than non CI/CD process, tests are simply being run by a non human entity.
Do you need a test plan (or plan your tests) ? of course you do, you need to think ...
- 7,089
9
votes
Accepted
Tests in CI: Are API + Integration Tests the most suitable?
As already mentioned in your TechBeacon article, teams indeed often spend too much at the top of the test automation pyramid. In general, the pyramid is a good rule of thumb, but (as always) it ...
- 2,132
7
votes
What should a QA in software be worried about when a small configuration deployment is being pushed without testing?
It all depends on what you're configuring, and if the configuration mechanism itself was tested before.
If it's something small, like a timeout interval, number of retries, etc, then I'd agree it ...
- 1,040
7
votes
Is manual regression testing an technical debt for agile teams aiming for continuous delivery?
There are an awful lot of variables to consider here. But here are some points all the same.
Manual Testing vs Automated Testing This point alone is probably worthy of a book and is one of the most ...
- 484
7
votes
Do you still need a Test Plan when adopting Continuous Delivery?
Do you still need a Test Plan when using CI/CD?
YES you do. Because a test plan will tell you at least the thing that you did not specify in your question: how often do you plan to run the tests? ...
- 231
6
votes
Accepted
Is manual regression testing an technical debt for agile teams aiming for continuous delivery?
In my experience, when the lack of automated regression in a delivery pipeline becomes technical debt, you have exactly one option.
Treat automated regression tests as code and sprint on them the ...
- 31.5k
6
votes
Tests in CI: Are API + Integration Tests the most suitable?
Use the test pyramid as mentioned by Peter Masiar.
My contribution here is help answer the question of exactly how to decide the 20% and 80% figures that Peter mentions.
My approach to this is to ...
- 25.1k
5
votes
Tests in CI: Are API + Integration Tests the most suitable?
Consider test pyramid. Using Pareto principle (also known as 80/20 rule), you can get 80% of benefits from testing by focusing on unit test with 20% of the effort. And 80% benefit of the remainder if ...
4
votes
Accepted
Difference between Integration Testing and Continuous Integration Testing
Integration Testing
Integration testing is about taking the units of code developed with unit testing and combine them together into the desired system. It (can) also mean testing software by ...
- 25.1k
4
votes
Accepted
Non-headless Automation Testing on CI Server?
The short answer: No, it is not crazy, as long as you take the proper precautions.
The longer answer:
There's nothing inherently wrong with running UI automation from a CI server. That said, there ...
- 31.5k
4
votes
Is manual regression testing an technical debt for agile teams aiming for continuous delivery?
It depends. Part of me is concerned with the tone of the question, to be honest. I'd be concerned that automated testing is being done to the detriment of manual testing, although some types of ...
- 3,360
4
votes
Automated tests in CI: same or different repo as SUT?
This is a great question and one that many organizations face.
I recommend keeping the test code and the application code together because this will, at the highest level, help keep (or perhaps '...
- 25.1k
3
votes
Is it a good practice to run functional tests with coverage?
First of all, coverage isn't coverage. There're several coverage criteria, but most used are:
Function coverage – Has each function (or subroutine) in the program been called?
Statement ...
- 2,132
3
votes
Difference between Integration Testing and Continuous Integration Testing
In a nutshell:
Continuous integration (CI): continuously integrate (small) software changes.
Continuous delivery (CD): continuously deliver these changes.
Continuous testing (CT): continuously test ...
- 2,132
3
votes
Accepted
When to mark a build as failed?
Personally, I think this is absolutely fine. Since you said you are using feature branches, only builds on top of these branches are failing. Pull requests are only merged when the corresponding build ...
- 2,132
3
votes
When to mark a build as failed?
Perfect, fail fast(er).
When the extreme programmers came up with continuous integration the idea was when someone checked in their code a couple of times per day and if that failed their automated ...
- 32.5k
3
votes
Is manual regression testing an technical debt for agile teams aiming for continuous delivery?
Yes, I think you can group lack of test automation under technical debt.
We don't have time for test automation, so we do it manual
This sounds like "Deliberate" and "Reckless" on the Technical ...
- 32.5k
3
votes
How does Continuous Integration add value to Automation Testing?
Continuous Integration
Let me explain in increasing levels of detail:
A place to run your tests.
A remote server that you can push code to that runs the automated tests
Continuous integration usually ...
- 25.1k
3
votes
How does Continuous Integration add value to Automation Testing?
Regression testing should be automated (not manual) and CI is very valuable tool for automated regression testing.
So I assume you already have automated regression tests, and you are considering if ...
3
votes
How is compatibility of business logic tested in microservices?
By the book, microservices business logic are independent from each other - built around business capabilities. This is achieved by defining bounded contexts (check it here, here, and here).
It means ...
- 10.5k
2
votes
Running unit tests on production build?
I would only run the tests on a code change, better on each code change and not just in the night. You want to know which code change broke what test.
In your case this would mean when you branch the ...
- 32.5k
2
votes
Accepted
Is there any tool to automate test for installers (WiX or InstallShield)
You could try TestComplete - I believe we use it to test installation here. Essentially you're just testing driving another windows application: if the environment can let you check registry entries ...
- 111
2
votes
What to do with Limited Time/Testers - What is most Important?
You mentioned having a handle on testing, but wanting to know more about documentation in limited time scopes.
In that case, my advice would be to get into Jira or another bug-tracking software asap....
- 344
2
votes
How to move from test automation to devops?
Generally I would consider some of the key requirements for an entry level position to be
background as a developer
experience with OO such as Java
practical experience with automation
familiarity ...
- 25.1k
2
votes
Accepted
How do I automate front-end testing for a web app with Javascript?
Front end testing is important and is often part of User Acceptance testing.
Popular approaches uses Selenium along with a programming language - Java, Ruby, Python or C#
Update (2023)
There are now ...
- 25.1k
2
votes
Legacy Code Testing with Jenkins on PHP
You will need some steps to get there:
Have you repository system to notify Jenkins about the PR.
That will depend on your system. GitHub and Bitbucket have system to do it.
Build the PR branch.
...
- 10.5k
2
votes
Version labels in continuous integration teams
I would recommend using the versioning and release processes available in github.
We use them for both staging and production releases.
Manual Testers are given github access so they can check what ...
- 25.1k
2
votes
Version labels in continuous integration teams
Some recommendations which may be relevant (but not Jira-specific) for your case:
Your build system will change, your ticketing system will change, and your version control system will change. This ...
- 468
2
votes
What should a QA in software be worried about when a small configuration deployment is being pushed without testing?
I think your question:
What should a QA in software be worried about when a small configuration deployment is being pushed without testing?
Already contains the answer:
When a small configuration ...
- 25.1k
2
votes
What should a QA in software be worried about when a small configuration deployment is being pushed without testing?
I've seen too many times when a tiny, seemingly insignificant change resulted into a broken application.
No matter the size of a change, you've got a new version of your application - it should be ...
- 11.4k
Only top scored, non community-wiki answers of a minimum length are eligible