What are some good test cases in order to do License Key authentication while software is being installed?

Question

I want to learn about:
What could be the test cases to ensure the test coverage while any proprietary software is being installed using a license key. I wish to avoid (a) re-installation (b) multiple instances of installation - using the same license key.

Answer 1

I am assuming that you want to test the installer code that verifies the license key. To test that there are a few things that would be required.

a) THOROUGH understanding of the validation routines involved.

As these systems are designed to enforce copy protection, they are usually very, very complex and maths heavy. If you want to understand how to test it, you need to understand the implementation.

b) Unit test thoroughly, in isolation.

Take the key verification code, isolate it and using the knowledge from step a), unit test it to death. You would need to use fuzzing attacks as well.

c) Integrate and test the install

Once you are happy with the thoroughness of your unit tests, integrate it and then test a few scenarios. I would create a semi-autmated solution that involves restoring a virtual machine, then trying to install. Then repeat, and use a lot of machines to scale-out your execution.

d) Test negative first, then positive

One way to cheat the number of re-installs would be to test all your negative scenarios first, then test your positive ones, as you, in theory, shouldn't need to re-install as your install would have failed. But personally, I wouldn't do this I would do (c) instead.

Answer 2

These are few test case, that can help you:

1. install successfully according to the instructions.
2. check the disk space after installation;
3. check if the installed software is working or not;
4. change conditions such as directing to the non-existing folder to check if installation continues;
5. remove some necessary files in the installation package to check if the installation continues;
6. cancel the installation when installation is progressing to check if the installation stops and the installed files are cleaned up;

7. uninstall the application normally

8. cancel the uninstallation half way when it is in progress; then again try to uninstall normally (uninstallation should be successful)

9. reinstall the application (after successful uninstallation).

10. uninstall the application (after successful reinstallation).

11. Check if the website is created in the selected or expected structure by the installer.

12. Check if the installer updates the correct version of the .net Frame work version correctly as expected(.Net Application )

13. Check if the Credentials are given as expected.

14. Check if the Windows services are Installed as expected.

Generic Installation Test Cases

1. Support of the different platforms and configurations needed?
2. Does Installation in “clean state” working?
3. Does the installer is able to calculate needed disk space?
4. Does the installer capturing the baseline free space before launching the installer?
5. Does the amount of space the installer claims it needs for the various type of installation is actually taken up or is there any discrepancy?
6. How much disk space is used by the installer if the installation is quit midway (this would help gathering information on temp files that’s deleted on quitting).
7. Does the installation recover in case an error is met during the installation?
8. Does the installer able to Repair any corrupt installation?

9. If the application installed properly fro each type of installation (for typical, custom and complete)?

10. Does Installation over network working?

11. If any file association is made during installation, upon uninstallation, does the association is removed and the base file association is returned to the files?

12. Does running the installer, followed by launching the program to run some tests, and then running the uninstaller, also return your machine to the base state?

13. Does uninstallation leave any registry entry, data files in the system?

14. If there exists a version of the app to be installed already on the machine, does the installer identify that?

15. Does the installer identify if some needed components (such as, MSDE etc.) are already installed on the system?

16. Does running two instances of the installer should prompt a message to the user that an installation setup is already running?

17. If the user logged in, doesn’t have write permission for the machine, how installation reacts to this?

18. What happens if the installer tries to install to a directory where there is no write access?

19. Is the installation path configurable/non configurable?

20. Check to ensure that when installing the product, it should provide a browse button which enables the user to install at any folder, and it should provide by default folder (For ex: C:\program files)

21. Is the registering and un-registering the components (dlls) on installation and uninstallation is occurring properly?

22. Are all the files installed in the respective folders and path?

23. Whether all the files/registry values/services are installed properly.

24. Check whether the shortcuts are installed properly and also the PATH (any other ENV variables) is updated properly.It can be CURRENT USERS profile or All Users profile based on your requirements).

25. Does the installation support, 'UnInstall', 'Modify', 'ReInstall' options?? If yes, does it work?

26. If the installer is supporting upgrade feature, does it preserve all the necessary settings (mostly user preferences)?

27. Check for the user privileges before starting installation. (In most of the cases, installer requires ADMIN privileges).

28. Check the uninstaller entry in add-remove programs. (Check for display string, Display icon and Support information etc.).

29. Does running the installer, and then running the uninstaller, return the machine to the base state?

30. Reinstallation should, apart from identifying previous versions, also should give an option to Remove and Repair.

31. Check to ensure that license key is properly stored in Windows Registry library.

32. Check to ensure that if an evaluation version is installed, then a proper message should be displayed when the date of period is expired for evaluation version with proper error message.

33. Check to ensure that, if Windows Services are installed then it should install in the Services folder of windows directory.

34. Check to ensure that if any product is installed and it is dependent on some other product, then it should give proper message as "The Product is not installed and it should exit". Check for dependencies.

35. If the product to be installed uses any third party dll and if it is already installed by some other product confirm that the current installation doesn’t un-register/tamper it and uses the existing one.

36. If the dll is already there in the system how does the installation work?

37. And also while uninstalling, check should be made as to whether that the shared dll is left without affecting other product.

38. “Usability” consideration of the installer.

Answer 3

I don't know a way to ensure much in the way of test coverage without re-installation. I guess you could install once and pretend that's good enough.

Why do you wish to avoid re-installation and multiple instances?

Answer 4

If it is an internet connected application, having a ssl cert validation and using its private key to be used as a salt for using license key could be a good idea. SSL certs give you a very promising theft-prevention towards license.

Also, using hardware tokens could be a good idea. So, even if you copy the license key to your computer, your hardware key would be really very hard to replicate.

A link worth a read: http://discuss.joelonsoftware.com/default.asp?biz.5.82298.34