6

We have got an old system Qwerty1 - that makes a lots of things,

We also have another system Querty2 it's actually a remake of Qwerty1 (for several reasons)

Notice we still are live with Qwerty1.

Now we want to be sure for all tested input data in Qwerty2 we get the same output data in both systems.

What is this methodology called?

Improve this question
3
  • What do you think should be an appropriate name for it?
    – IAmMilinPatel
    Commented Jan 9, 2018 at 10:12
  • to be honest I have no idea really I have been asking my colleagues but we are not sure.
    – XsiSec
    Commented Jan 9, 2018 at 10:12
  • The testing part is covered, but the software development method used is called "Big redesign in the Sky", for several reasons the new product never gets delivered, as it is never feature complete as the old system still has to move forward as well. UncleBob nicely explains these situation in his free video: cleancoders.com/episode/clean-code-episode-1/show
    – Niels van Reijmersdal
    Commented Feb 6, 2018 at 13:13

8 Answers 8

Reset to default
4

We would tend to consider this a form of regression testing as you are testing that the software still behaves the same as it did before.

Regression testing is a type of software testing which verifies that software which was previously developed and tested still performs the same way after it was changed or interfaced with other software.

Qwerty1 and Qwerty2 may be separate products, but consider them just to be two different builds of the same functionality and the testing considerations are clearer.

Improve this answer
10
  • That was my thought from beginning though I have never literally understood regression testing in that way even if there is a new product that perform the same functionality as the old one. Or?
    – XsiSec
    Commented Jan 9, 2018 at 16:49
  • 1
    Any time you're checking that the behaviour of something hasn't changed - that's a regression test. Even if you rewrite the entire codebase and rename the product - if you compare it to an old version (such that a difference in behaviour is a failure), that's regression. Note that in perfect/ideal regression testing - you expect to find the same bugs as before, there should be no change to behaviour at all (but this is obviously not realistic).
    – Bilkokuya
    Commented Jan 9, 2018 at 16:53
  • 2
    I wouldn't agree with you. Regression testing is conducted on the same application to identify not desired 'mutations' and side effects. In case of OP, they are testing 2 different applications which are built for the same purpose (i.e. have the same functionality).
    – olyv
    Commented Jan 9, 2018 at 17:18
  • 2
    @olyv I'd argue it's conducted on any new version of software that adheres to the same requirements. It doesn't matter if the change was a minor patch, or an entire new product. If you wish to test a new piece of software runs identically to an old one, you perform regression testing - using the same test cases on each and considering a pass to be whatever the result of the old software was.
    – Bilkokuya
    Commented Jan 9, 2018 at 17:31
  • 1
    Why overthink it? You want to verify that some new code has the same behavior as some old code = regression testing.
    – Alex
    Commented Jan 10, 2018 at 0:23
4

It sounds like you're talking about Comparative Testing (also known as Product Testing)

...the process of measuring the properties or performance of products.

Improve this answer
1
  • I added a critera to the question, I am not sure if that was clarified enough before.
    – XsiSec
    Commented Jan 9, 2018 at 11:47
3

I would go with Benchmark test. http://glossary.istqb.org/search/benchmark%20test

(1) A standard against which measurements or comparisons can be made.

(2) A test that is used to compare components or systems to each other or to a standard as in (1).

Improve this answer
4
  • I added a critera to the question, I am not sure if that was clarified enough before.
    – XsiSec
    Commented Jan 9, 2018 at 11:47
  • @DanielElmnas I still assume 'benchamrk' is appropriate term to apply. Because this is only about comparison of two systems
    – olyv
    Commented Jan 9, 2018 at 12:02
  • The original question has been edited such that my answer is now incorrect. The original Q talked about performance and the reason Qwerty2 had not been released was because it was slower that Qwerty1. The original question was more about performance rather than functionality.
    – Chris Adams
    Commented Jan 10, 2018 at 10:53
  • still, 'benchmark' doesn't strictly specify 'only quantitive measurements'. It's only about comparing
    – olyv
    Commented Jan 10, 2018 at 14:54
2

I agree with Olyv (and have 'up voted' his answer).

Qwerty1 is your benchmark. Qwerty is failing because it's not as fast as Qwerty2.

You ought to log the results for each test side-by-side so you can time how performance changes over time.

Improve this answer
2
  • I added a critera to the question, I am not sure if that was clarified enough before.
    – XsiSec
    Commented Jan 9, 2018 at 11:47
  • Yep, it was clear. I was just reiterating based on your Q and Olyv's answer. I'd still call it benchmarking as Qwerty2 is a remake of Qwerty1. So, Qwerty1 is the benchmark perf that Qwerty2 is striving to better or at least match.
    – Chris Adams
    Commented Jan 9, 2018 at 12:01
2

It seems to me that you are refactoring

Looking to improve the performance of the code and the implementation of the design without changing the functionality.

Improve this answer
1
  • Yes it sounds more like that, but is that a still a software test methodology? I did read following thread softwareengineering.stackexchange.com/questions/233222/… though this feels more like you maintenance the existing system in our we made a totally brand one that have same functionality more easygoing and better performance.
    – XsiSec
    Commented Jan 9, 2018 at 11:55
1

This could be considered a form of Canary testing, which is typically reserved for a DevOps context of slowly switching traffic from the old system to the new system, and monitoring IO for discrepancies.

Improve this answer
1
  • It feels like few things we does it's Canary testing but less than the half it feels like.
    – XsiSec
    Commented Jan 10, 2018 at 8:39
0

The old system "Qwerty 1" will be the reference for testing new system "Qwerty 2", so the appropriate methodology "Comparison Testing",but it is not enough ,you will use this with other methodologies and techniques in SW testing to ensure the new one works fine with no(incorrect or missing function,interface errors,performance errors,business errors)

Improve this answer
-1

While not a traditional UI/website A/B test, I don't think it'd be wrong or misleading to say you're doing a blackbox A/B test and comparing the output, performance, whatever, between the two implementations.

I like the term benchmark, but not benchmark test. Clearly, Qwerty1 is the benchmark and the results of running it are the benchmark test using the ISQTB definition, but the actual compare/contrast of Qwerty1 to Qwerty2 is not a benchmark test.

I also dislike the ISQTB definition of benchmark test, mainly since in my experience, when people discuss benchmarking, it's often performance testing related, when it seems that your testing isn't focused on performance testing.

Improve this answer
2
  • I don't agree that is somehow any kind of Blackbox testing its wrong in my opinion really. softwaretestingfundamentals.com/black-box-testing
    – XsiSec
    Commented Jan 10, 2018 at 8:37
  • Sure, this could be white, grey, whatever. I mentioned blackbox since that's all that's required if the goal is "to be sure for all tested indata in Qwerty2 we get the same outputdata in the both systems." In other words, all you need to do is try various inputs and check the outputs, the underlying details aren't important.
    – ernie
    Commented Jan 10, 2018 at 15:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.