Should finding specific data to test with be included in the test steps

Question

When testing our website, I usually have to find a customer order with which I can test. I do this by running queries against various databases. Once I find the order that will work, I begin the test. My question is whether or not the "search" for the customer order should actually be included in my test steps.

Answer 1

You should aim to create the data yourself

although whether you are doing manual or automated testing is a key factor and the following mostly applies to automated testing.

One of the principles you should be looking to follow when performing automated testing is

setup - execute - teardown

Basically you want the system and most notably the data store to be empty of any transactional data. The only data that should be considered to stay between tests is seed reference data, e.g. valid zipcodes, auto model types, etc.

So in many cases you will need to create the customer and all the associated records that are needed for the test. This is a big change from 'let me just query for a record'. It also implies that you should be using a version of the system that has a mostly empty test database suitable for such testing. This is different from the development database that will usually need a fair mount of data to be present in order to use and manually test the UI.

When first getting into professional testing this can be a big hurdle / change. It also tends to differentiate between manual testing and writing automated tests.

Answer 2

This would be your test precondition, of which if successful would mean Step 1 you are ready to go with the test. For example, precondition "query database to find X order, navigate to order". Step1 - Using order from precondition, press button Y

If the precondition can be achieved then the test case can be run. If the precondition cannot be achieved then the test case is "blocked" rather than "failed". ie, you cannot fail it as you have not got it into a state to start it.

You might well raise one or more bugs on precondition steps if relevant.

Answer 3

While documenting tests or rather reporting them to your dev team, its suppose to have sufficient details that will help the team trace, understand and fix the issue at hand.

Every data you use for testing is important test artifact. Mentioning that in your test cases will tell the developer what data used used for your test that caused the failure. Using this they can check the conditions and logic they implemented and correct it to make sure that the issue won't be repeated.

Yes, in certain cases it may be a part of pre-requisites or pre-condition. You may not say how you found that piece of data (Although it may be easier for the developer to analyze if you did), but you should mention what data you used.