11
Your application uses an API to interact with the database. It is possible to write your API in such a way that it presents correct results to the application and yet still uses the database in the wrong way.
For example, imagine a database with an EMPLOYEE table and a MANAGER table. The tables are alike -- e.g. each contains a first name, last name, ...
10
No. The items you mention scale very differently and there are far too many factors and resources that will get used.
For instance if I time requests on a local server using an application I'll find things like
1 user = 2 second response average (time per request)
10 users = 2.5 second average
100 users = 2.5 second average
1000 users = 20 second average
...
10
Does this actually catch, before production, many of the "surprise"
problems we might anticipate? Or is there a more fundamental flaw in
the approach that will cause deleterious changes to pass testing and
affect production?
You are wise to have a test system that you can use for catching performance issues, but your "scaling" approach is flawed.
...
9
See if your test framework gives you a way to parameterize tests. Many test frameworks allow you to supply the values using a "data provider" method or object or class that you write yourself. If you have a framework like that, see if you can use its data provider mechanism to supply the values.
The usual mechanism is that your code fetches the values from ...
7
There's two major directions you can take here.
If your tests are structured such that one test covers a full user scenario
(By this I mean that a single test covers something like log on as user X, navigate to ordering, select quantity A of product B, quantity C of product D, check out, pay with card details Y and check that all the amounts add up ...
7
Wouldn't any data inconsistencies expose themselves in the application
itself?
Maybe, maybe not.
I've seen cases where applications lost some data after you've logged off. So while the UI looked fine, the database was actually incorrect after logoff.
In addition, are you sure that every single element in the database is being displayed in the UI ...
7
It depends on how your process is set up. It makes sense to have read-only access to the Production environment for investigation.
Does QA have own copy of the system for integration testing (separate from development system)? In this QA system, QA should be able to make any changes. But even here, it would be smart to get proposed changes to be reviewed ...
6
As UI tests are slowish I think its important to be able to run them in parallel. In order to be able to run them in parallel, your data-management becomes even more complex, since if the tests use the same data-source they could change data into conflicting states and make tests randomly fail, that why the tests should all run in isolation.
Currently, I am ...
answered Feb 21 '15 at 11:40
Niels van Reijmersdal
31.6k4 gold badges52 silver badges118 bronze badges
6
I don't want to repeat great answers other have provided, but I would like to share with one more lesson I learned about using database in tests.
Combining feedback you have from both database assertions and UI assertions, often in one test, can be very useful for test case design, test execution performance and defect root cause isolation:
If you spot an ...
6
Selenium itself is only a framework to drive web-browsers, but you can combine it with some code to check and or manipulate the database the application under test uses.
If you use Selenium with a programming language then you can use the same language in combination with a database library to access and change the database. Supported databases depend on ...
answered Apr 10 '15 at 9:59
Niels van Reijmersdal
31.6k4 gold badges52 silver badges118 bronze badges
6
Yes this is a useful strategy in converting from one system to another as a one time event (it might span a couple of iterations but the idea is very short term).
You could effectively view this as BDD with 'half (the tests) already done'. You have the tests against data that works. You create a new back-end version and see if it does the same.
As I ...
5
In addition to user246's excellent example, some other cases where you'd want to validate the database storage in your scenario would include:
You have a bulk update/insert function where it's impractical to validate the results via the front end, such as importing new user records from a CSV file. While you can go in through the front end and check that ...
5
Selenium Webdriver is a web test automation framework and primarily used for functional testing of front-end part of the application (that piece, that the user sees in his/her browser).
Since this framework is open source and can be relatively easily extended, someone could have already implemented some additional libraries for interacting with the database....
5
This is a really interesting question. I don't think we're entirely happy with our approach, and we discuss it regularly, but these are our constraints and what we've done so far (framework has been in use for about 3 years).
1) We reload a skeleton database at the start of each test run. This has a minimal set of data, as little as possible - a user, ...
4
It's hard to provide input on your specific processes, but here's how I generally deal with Selenium automation. Maybe it can help you out.
One separate database which contains all the data for the tests: data-driven testing. This means you'll likely have to compose Views (combining tables as necessary) which serve as the data source for test methods.
...
4
I have always loved using Scooter Software - Beyond Compare (limited to 2 links) to visually compare between folder, files etc. Group of them or single.
This should help with an easy to use Visual Compare.
Daniel has created a great BC File Extension Forum Link | How To on GitHub.
It uses simple export with command line to do compare SQLite DB.
I ...
4
CRUD stands for Create, Read, Update and Delete. (Retrieve may occasionally be substituted for Read.)
Different users may have different CRUD cycles based upon the requirements of the system. A customer, for instance, might have the ability to create an account, retrieve it upon return to a website, update billing information, or delete it if necessary. An ...
4
Presumably the database is just a piece of your overall system, and your goal is to determine whether database changes break things or slow the system down.
Using a downsized database is a reasonable way to check whether database changes break things.
I'm not sure it makes sense to use a downsized database for load testing. Whether this makes sense ...
4
This depends on what type of testing you implement:
If it is a component testing (or if your goal is to test the particular end-point) you should isolate your component from the possible impact of other components. So having no options to mock the data I would use direct database access.
If it is an integration testing then I would use endpoints to obtains ...
3
Based on your response to my comment, you're not actually looking to test database operations but an application that employs the operations. That makes things a little simpler.
If you have access to the data store - the simplest method you can use is to perform an operation with your application and treat the data store as an oracle for verification ...
3
Selenium WebDriver itself cannot be used to test databases, but the language you are using to code your WebDriver tests (eg. C# or Java, etc.) can.
3
Reposting from my blog notes
Database Testing involves the following Activities for a OLTP System
Verify Table Schema, Column names as per Design Document
Verify Column Length and DataType
Verify Unicode Support (Storing Chinese/Japanese Characters) - NVARCHAR Datatype
Verify Indexes on Table (Clustered, Non-Clustered), Triggers for Auditing
Verify Primary ...
3
Test environment will NOT scale proportionally.
But what you can do is:
make test environment as similar to PROD as you possibly can (with possibly less servers, that's what we do: have about half of the servers in PROD)
replicate most of the complexity you can (fallback servers as in PROD)
replicate the load from PROD in a reasonable way (we collect few ...
3
Selenium is used for automating user-browser interaction with the HTML and Javascript on websites.
It does not have any functionality to test databases other than indirectly through web applications and HTML pages.
Per http://www.seleniumhq.org/
"Selenium automates browsers. That's it! What you do with that power is entirely up to you. Primarily, it is for ...
3
Check out the answer to this question over at StackOverflow.
(From the accepted answer) NUnit has few advantages over MS-Test
Suite attribute - can aggregate tests and execute them separately (useful for large projects with fast and slow tests for example)
Readable Assert method, e.g. Assert.AreEqual(expected, actual) vs Assert.That(actual, Is.EqualTo(...
3
With any kind of framework or database upgrade like this, after I'd covered system sanity and critical path regression, I would go through the release notes looking for any changes likely to impact my application and test around each of those.
I'd be particularly careful with these areas, in this approximate order of priority:
Breaking changes
Deprecated ...
3
I'm a little out of date using sqllite but data types and stored procedures come to mind as feature gaps between sqllite and something like PostgreSQL. Beyond features, configuration settings (ex: threads in sqllite) could cause further irritation while trying to speed up your tests if used incorrectly.
More generally though, you should understand that ...
3
Database shared between tests is a common obstacle.
One way ("orthodox") is to insist that unit test should NOT test database (just exercise the tested library), and mock the DB interface.
Another way ("synthetic") is to use real database, even if such test are not strictly a unit test, but low-level integration tests. Then either:
test might interact ...
3
Back-end and back-end testing per se is a complex mechanism. Basically anything that communicates with the server or database is back-end. Another definition can be anything under the hood is back-end. Server side is back-end, database level changes again back-end. So back-end testing isn't just database testing although db testing to a certain extent is a ...
3
Comparing the old system to the new system given identical inputs and expecting identical outputs is a legitimate test. There are problems, however:
a) When the legacy system is retired, the Source of Truth is no longer available and comparisons are impossible. You'll have to be very comfortable that the systems work similarly enough to satisfy business ...
Only top voted, non community-wiki answers of a minimum length are eligible