How can I automate a test scenario of account expiry or account at different time transition?

Question

I have an application where trial credentials expires after a week and if user tries to login after that, they see only few things enabled.This is one of the example. Similarly, verifying expiry of premium and gold membership.

One of the solution which probably most of you will suggest is updating the timestamp in DB. Can someone suggest me something other than this?

Answer 1

You have several choices, although none are perfect. In the order I prefer to use them:

1. Database/data manipulation - unless you have a ridiculously complex data structure (been there) or no access to the database in question (been there, too), this is the simplest, safest way to manipulate expiry/deadlines.
2. Setup/configuration manipulation - configure your membership types to expire within some short timeframe. A sequence I've used for manual testing is to create the data on day 1, using a set up that will expire within 24 hours of creation. On day 2, I can test expiry.
3. Mocking tools - a mocking tool such as Microsoft shims that will intercept any calls to now() and return the date of your choosing can be used if you have the programming skill to integrate these into your automation (not every automator does, particularly if their primary experience is through record/playback tools).
4. Application API hooks - this method usually requires the assistance of the developers, and may not be available. In addition, if the hooks are only present in a special build, you won't be testing against the same code used by your customers.
5. Modifying system time - This has a number of problems because most operating systems check against a canonical time source and will either force the system time to the canonical value or fail to run, fail to communicate, and sometimes simply fail when the system time is too far out of sync with the canonical time.
6. Modifying system time zone - this method can work if your application uses an unsophisticated method of checking time.

Answer 2

I am going to coin this user246's First Law of Testing: everyone who tests software will eventually need to test something with a deadline. (I'm sure several hundred thousand other testers have coined the same law.) Although the specifics vary from one problem to the next, there are two essential solutions: either change the clock or change the deadline.

Changing the clock

The most obvious way to change the clock is to literally change the operating system's definition of the current time. This is risky because there are probably other systems running on the same machine that will behave incorrectly if you change the system time.

The risk is lower if you run your software inside of a disposable virtual machine, i.e. a VM that you plan to destroy when the test is over. However, some VMs (e.g the Xen hypervisors in Amazon EC2) synchronize the VM clock with the host's clock no matter what you do, so this option may not be available to you.

Of course, unless you are testing an operating system, changing the operating system's time is just a way to change an application's definition of time. Sometimes a developer will have the foresight to provide a way to override the application's definition of time. Using this feature probably requires that the tester be able to restart the application.

Changing the deadline

If you can't change the clock, you may be able to change the deadline. Sometimes a developer has the foresight to provide a way to override deadlines. There may be a way to override the deadline when it is calculated; for example, if the trial membership is supposed to last 7 days, there may be a way to force the trial membership to last 5 minutes instead. If you can't override the deadline before it is calculated, there may be a way to change the deadline after it is calculated via a database query or a special web form or whatever.

Your developers may know of a way to do this

Most of the techniques for testing a deadline require special testability features. These features are not necessarily easy to write, and you won't get them by default. If it's hard for you to test deadlines, it may be worth asking your developers for help or advice.

Answer 3

You can pass the "now" time into the function which verifies the expiry (and obviously also the function which creates the credentials). The test function calls it with a hard-coded time, and the production function calls it with the actual current time. For example (untested):

def credentials_valid(credentials, now):
    return credentials.expiry > now

def test_should_return_false_for_expired_credentials(self):
    now = datetime(year=2000, month=1, date=1)
    future = now + timedelta(second=1)
    credentials = create_credentials(now)
    self.assertFalse(credentials_valid(credentials, now))

Answer 4

If you are running the tests in a VM, with an offline snapshot of the DB also in a VM, there is no reason not to advance the VM system date/time as necessary. You could even do the same without a VM but with standalone isolated hardware and the actual system date/time.

This has the advantage of mimicking the real world situation better than modifying backwards the users account creation date/time as the database would be unmodified.

Depending on your test scripting environment and your OS your test scripts should be able to issue a system time command to set the time - as pointed out in the comments if the DB polls the time periodically you should allow long enough for the system to conduct a poll before continuing, usually at least 1.5 times the poll rate unless the system API has a time refresh call available, even then you should allow a little time to pass after issuing the refresh.