10 votes
Accepted

Do you still need a Test Plan when adopting Continuous Delivery?

I don't see CI/CD as something different than non CI/CD process, tests are simply being run by a non human entity. Do you need a test plan (or plan your tests) ? of course you do, you need to think ...
Rsf's user avatar
  • 7,089
9 votes
Accepted

Tests in CI: Are API + Integration Tests the most suitable?

As already mentioned in your TechBeacon article, teams indeed often spend too much at the top of the test automation pyramid. In general, the pyramid is a good rule of thumb, but (as always) it ...
beatngu13's user avatar
  • 2,132
7 votes

What should a QA in software be worried about when a small configuration deployment is being pushed without testing?

It all depends on what you're configuring, and if the configuration mechanism itself was tested before. If it's something small, like a timeout interval, number of retries, etc, then I'd agree it ...
ernie's user avatar
  • 1,040
7 votes

Is manual regression testing an technical debt for agile teams aiming for continuous delivery?

There are an awful lot of variables to consider here. But here are some points all the same. Manual Testing vs Automated Testing This point alone is probably worthy of a book and is one of the most ...
Chris Adams's user avatar
7 votes

Do you still need a Test Plan when adopting Continuous Delivery?

Do you still need a Test Plan when using CI/CD? YES you do. Because a test plan will tell you at least the thing that you did not specify in your question: how often do you plan to run the tests? ...
virolino's user avatar
  • 231
6 votes
Accepted

Is manual regression testing an technical debt for agile teams aiming for continuous delivery?

In my experience, when the lack of automated regression in a delivery pipeline becomes technical debt, you have exactly one option. Treat automated regression tests as code and sprint on them the ...
Kate Paulk's user avatar
  • 31.5k
6 votes

Tests in CI: Are API + Integration Tests the most suitable?

Use the test pyramid as mentioned by Peter Masiar. My contribution here is help answer the question of exactly how to decide the 20% and 80% figures that Peter mentions. My approach to this is to ...
Michael Durrant's user avatar
5 votes

Tests in CI: Are API + Integration Tests the most suitable?

Consider test pyramid. Using Pareto principle (also known as 80/20 rule), you can get 80% of benefits from testing by focusing on unit test with 20% of the effort. And 80% benefit of the remainder if ...
Peter M. - stands for Monica's user avatar
4 votes
Accepted

Difference between Integration Testing and Continuous Integration Testing

Integration Testing Integration testing is about taking the units of code developed with unit testing and combine them together into the desired system. It (can) also mean testing software by ...
Michael Durrant's user avatar
4 votes
Accepted

Non-headless Automation Testing on CI Server?

The short answer: No, it is not crazy, as long as you take the proper precautions. The longer answer: There's nothing inherently wrong with running UI automation from a CI server. That said, there ...
Kate Paulk's user avatar
  • 31.5k
4 votes

Is manual regression testing an technical debt for agile teams aiming for continuous delivery?

It depends. Part of me is concerned with the tone of the question, to be honest. I'd be concerned that automated testing is being done to the detriment of manual testing, although some types of ...
Kevin McKenzie's user avatar
4 votes

Automated tests in CI: same or different repo as SUT?

This is a great question and one that many organizations face. I recommend keeping the test code and the application code together because this will, at the highest level, help keep (or perhaps '...
Michael Durrant's user avatar
3 votes

Is it a good practice to run functional tests with coverage?

First of all, coverage isn't coverage. There're several coverage criteria, but most used are: Function coverage – Has each function (or subroutine) in the program been called? Statement ...
beatngu13's user avatar
  • 2,132
3 votes

Difference between Integration Testing and Continuous Integration Testing

In a nutshell: Continuous integration (CI): continuously integrate (small) software changes. Continuous delivery (CD): continuously deliver these changes. Continuous testing (CT): continuously test ...
beatngu13's user avatar
  • 2,132
3 votes
Accepted

When to mark a build as failed?

Personally, I think this is absolutely fine. Since you said you are using feature branches, only builds on top of these branches are failing. Pull requests are only merged when the corresponding build ...
beatngu13's user avatar
  • 2,132
3 votes

When to mark a build as failed?

Perfect, fail fast(er). When the extreme programmers came up with continuous integration the idea was when someone checked in their code a couple of times per day and if that failed their automated ...
Niels van Reijmersdal's user avatar
3 votes

Is manual regression testing an technical debt for agile teams aiming for continuous delivery?

Yes, I think you can group lack of test automation under technical debt. We don't have time for test automation, so we do it manual This sounds like "Deliberate" and "Reckless" on the Technical ...
Niels van Reijmersdal's user avatar
3 votes

How does Continuous Integration add value to Automation Testing?

Continuous Integration Let me explain in increasing levels of detail: A place to run your tests. A remote server that you can push code to that runs the automated tests Continuous integration usually ...
Michael Durrant's user avatar
3 votes

How does Continuous Integration add value to Automation Testing?

Regression testing should be automated (not manual) and CI is very valuable tool for automated regression testing. So I assume you already have automated regression tests, and you are considering if ...
Peter M. - stands for Monica's user avatar
3 votes

How is compatibility of business logic tested in microservices?

By the book, microservices business logic are independent from each other - built around business capabilities. This is achieved by defining bounded contexts (check it here, here, and here). It means ...
João Farias's user avatar
  • 10.5k
2 votes

Running unit tests on production build?

I would only run the tests on a code change, better on each code change and not just in the night. You want to know which code change broke what test. In your case this would mean when you branch the ...
Niels van Reijmersdal's user avatar
2 votes
Accepted

Is there any tool to automate test for installers (WiX or InstallShield)

You could try TestComplete - I believe we use it to test installation here. Essentially you're just testing driving another windows application: if the environment can let you check registry entries ...
AndyW's user avatar
  • 111
2 votes

What to do with Limited Time/Testers - What is most Important?

You mentioned having a handle on testing, but wanting to know more about documentation in limited time scopes. In that case, my advice would be to get into Jira or another bug-tracking software asap....
Melissa Fabina's user avatar
2 votes

How to move from test automation to devops?

Generally I would consider some of the key requirements for an entry level position to be background as a developer experience with OO such as Java practical experience with automation familiarity ...
Michael Durrant's user avatar
2 votes
Accepted

How do I automate front-end testing for a web app with Javascript?

Front end testing is important and is often part of User Acceptance testing. Popular approaches uses Selenium along with a programming language - Java, Ruby, Python or C# Update (2023) There are now ...
Michael Durrant's user avatar
2 votes

Legacy Code Testing with Jenkins on PHP

You will need some steps to get there: Have you repository system to notify Jenkins about the PR. That will depend on your system. GitHub and Bitbucket have system to do it. Build the PR branch. ...
João Farias's user avatar
  • 10.5k
2 votes

Version labels in continuous integration teams

I would recommend using the versioning and release processes available in github. We use them for both staging and production releases. Manual Testers are given github access so they can check what ...
Michael Durrant's user avatar
2 votes

Version labels in continuous integration teams

Some recommendations which may be relevant (but not Jira-specific) for your case: Your build system will change, your ticketing system will change, and your version control system will change. This ...
l0b0's user avatar
  • 468
2 votes

What should a QA in software be worried about when a small configuration deployment is being pushed without testing?

I think your question: What should a QA in software be worried about when a small configuration deployment is being pushed without testing? Already contains the answer: When a small configuration ...
Michael Durrant's user avatar
2 votes

What should a QA in software be worried about when a small configuration deployment is being pushed without testing?

I've seen too many times when a tiny, seemingly insignificant change resulted into a broken application. No matter the size of a change, you've got a new version of your application - it should be ...
alecxe's user avatar
  • 11.4k

Only top scored, non community-wiki answers of a minimum length are eligible