Is it still needed to write acceptance tests that target verifying
those business rules implementation?
Yes, it is absolutely necessary.
Your unit tests cover the business rules in isolation. Your acceptance tests verify that the application properly implements those business rules from a customer perspective.
There are many cases where the unit tests ...
Yes that is the ideal situation for Unit tests
To look at a different situation - if you are writing software that will not be changed in the future then perhaps you could consider skipping the tests. I have yet to work with such software of course :) Also TDD and BDD proponents would argue that even in those cases you should still use those techniques. ...
Yes, it is. Because the unit tests are not only for the current version of the code. They are especially for all future versions and if somebody changes the code and does not know what s/he did, the test can fail and tell him that he did something wrong. So you can't say that this will never happen.
Also unit tests do not only test code, they are ...
Well, first of all - congrats buddy, glad to know you work for customer / company with understanding that involving QA at early stages is definitely worth it!
I'll reveal below several related points from my own experience of managing QA team (up to 10 FTE) involved in testing of fairly big project, where reqs page count was about 1000 pages and more, and ...
The short answer - Usually they don't. Developers write unit tests and ideally run them every time the software builds.
The longer version - Unit tests are intended to test a single unit of code. This means that they are not testing through the application user interface, but are running directly against the single unit they are testing and using stub/mock ...
Yes, the idea of writing the right form of unit-tests is that it keeps the cost of change low. If you make a lot of changes they are here to help you go faster.
The biggest mistake most people make is too test implementation instead of behavior, which increases the cost of change, because you build change detectors. The tests should help you refactor the ...
A key strategy for me is to convince the business of what needs to be tested where, otherwise... they'll end up directing testing everything through the UI... So the two key main points I am making to my business are the need for test automation that performs well in terms of two key factors:
Success in both speed and reliability for ...
There is no concept of overlapping testcases in different test levels,
Both are completely isolated
Just because API or Component works fine you cannot guarantee the whole system or integrated system works fine.
Imagine all your unit tests passing but user not able to use the UI or API workflow, Imagine all your UI working due to cached information but the ...
How much testing is sufficient? Meaning, what's the minimum effort required to test reliability?
When looking at the question of "how much testing?" you have to consider "how lucky do you feel?" You could do no testing at all if you feel really lucky, or if the consequences to being wrong are extremely low. You could test everything for a really long time ...
You're absolutely right to feel that your time in the early stages is best spent understanding the problem.
Mapping the use cases and data/control flow of the requirement is one good way of building up that understanding, and in doing so, you are also testing those requirements as you will discover questions that you want to ask, gaps and ambiguities that ...
The TDD cycle is more a development cycle for a developer, to quote James Shore, The Art of Agile, Test-Driven Development chapter:
Programmers new to TDD are often surprised at how small each increment
can be. Although you might think that only beginners need to work in
small steps, my experience is the reverse: the more TDD experience you
have, the ...
Ideally I would like to write the test based on a file that approximates what my script would be actually processing, rather a much much smaller sample size.
If it were me, I would test functional correctness and scale independently. Fault isolation is easier that way. Your code/test/debug cycle can go more quickly too.
The thinking here is that ...
Full disclosure: we rely heavily on FitNesse, run many hours worth of FitNesse tests daily in our CI buildbot daily, and I personally wrote both FitNesse tests (using Fit wiki format in browser) and WebDriver tests in Python.
FitNesse is for end-to-end testing (browser automation) and Selenium/WebDriver in much superior replacement, especially if you ...
Nose is no longer maintained:
Nose has been in maintenance mode for the past several years and will
likely cease without a new person/team to take over maintainership.
New projects should consider using Nose2, py.test, or just plain
Even though unittest from the Python standard library is a very mature test framework with built-...
Estimate working load, how many test cases are there to be migrated from UI tests?
Estimate how much time you can spare in each sprint and discuss with your team leader and your fellow members regarding you want to spare some time in each sprint to work on test case migration.
Prioritize automated UI tests, there is a chance a portion of them are no longer ...
Mutation testing (or mutation analysis or program mutation) is used to
design new software tests and evaluate the quality of existing
software tests. Mutation testing involves modifying a program in small
https://pitest.org/ for java
https://pypi.org/project/mutmut/ for python
Yes, Unit-testing is only software related. No testing the implementation of a dashboard cannot be considered Unit-testing. Unit-tests typically test relatively small programming functions. A dashboard is the result of a lot of programming classes which consist of even more smaller programming methods (a.k.a. functions).
Unit tests are typically automated ...
Yes. I call them Contract Tests. One easy way to understand them:
Start with tests for a specific implementation of the interface. For example, consider ArrayList implements List. You write tests for ArrayList. One test could be this:
list = new ArrayList()
assert_equals 0, list.size()
Notice that ...
There are two (actually both are very similar) techniques in order to reveal the tests which tend to be "false/positive" ones – Error Seeding and Mutation Testing.
The both principles are based on introducing the errors in the application's program code, mainly in the places where it will have the most dramatical effect for application.
It can be for ...
Unit test are far superior in detecting root cause of the bug, compared to e2e tests. Maybe I am lucky, but our developers are religious about adding unit tests - because they experienced that it helps them to detect errors sooner, and allow for refactoring.
When e2e test reports a problem, you have to investigate what went wrong. Unit test tells you ...
It starts with explaining developers to take their discipline serious. They should follow the programmers oath.
I will produce, with each release, a quick, sure, and repeatable proof
that every element of the code works as it should.
I like how UncleBob compares developing software to double-book-keeping in accounting and disciplines like hand-washing ...
If there is one thing I like about good test names is that when it fails on a build-server I can understand what is broken just from reading its name. If the name of the test does not describe what it does it means you need to read the test code to see what is going on.
When an automated test fails, I want to understand what I just broke — without
Yes. Because those bits that do not change still need testing. The rest that does change, well, that's just a part of the process, you simply need to change things once in a while. Writing tests makes you think about the whole problem, which is alone a good practice to do. Without tests, problems might remain in the product until later, which might be then ...
I assume you are asking how to test code that is time-dependent.
One strategy is to refactor the code under test so that you control the definition of the current time. For example, in Java, instead of calling System.currentTimeMillis(), you might call a method on a "time provider" interface. The default to implementation of that interface would call ...
Yes you should have scenarios which verify the unit tested business rules.
good unit tests mock and stub the datastore. Acceptance tests make sure it is configured and works correctly.
unit tests usually test logic but don't allow for usability and accessibility
unit tests don't test whether the application works on the vast array of devices ...
I think you just need to re-raise the assertion error after printing out log messages:
self.assertEqual(len(self.verificationErrors), 0) # no errors
except AssertionError as e:
for message in self.verificationErrors:
raise # < HERE
Also, if you want to check for severe console messages after every single ...
I believe that the strongest instrument for filtering such a-low-quality tests is a code review. You can power-up your code review with mutational testing techniques which will help you to detect the tests which probably do not introduce any value to the overall project scope.
Here is the example of mutational testing framework where you can pick some ...
On my mind comes at least three thing:
If unit test are developed by developer with focus on testing simple functionality: then you as tester (this highly depends on experience of developer with for example Test Driven Development or experience with testing his own work) can assume that basic function are working (review of unit tests is nice thing, to get ...
Carefully name tests reflecting requirement in the form of input data & expected result.
I have been in similar situation but for integration UI tests, where it really helped us when we started carefully naming the tests with single specific requirement although sometimes names were very long.
We made sure that test name should unambiguously reflect ...