Anyone who has worked with Windows knows that every so often Microsoft has a tendency to release updates that have the potential of breaking things in production env. hence why testing is necessary. Does anyone have experience testing Win10 updates (Monthly Patch Release and Semi-Anual Feature Updates)? if so, what is your testing approach? what are the usual things that you test/check for as far as OS and Desktop Application Functionality (I know with Desktop apps it varies and depends on the company's standard software), are there any tools that you use during your testing process (if so what are they)? And finally, are there any books or documentation on how to test Operating Systems?
Add a comment
|
1 Answer
I don't work under MS-Windows much anymore. But Linux can have similar issues between releases (like Ubuntu 16.04 vs Ubuntu 18.04).
I tend to have all my tests automated and create as full coverage as I can. I also compile with many warning flags turned on and transformed to errors if they ever appear. This part is called static analysis and I'm amazed at how many programmers skip on that part. It has improved so much that a ton of bugs can be fixed even before you run your software for the first time.
Then I use a VPS to install a new version and compile and run my tests and my software on. I only need to know that my software works, so I don't actually test anything on the OS itself. However, one big thing that impacts you between major upgrades are settings which are not always easy to test. Especially if you rely on other libraries or software which can change drastically.
But in most cases, if all the automated tests pass, I'm generally in very good shape. Ad hoc testing is also useful, especially if you have people who are expected to spend time testing your software anyway.
And of course, apply fixes for anything that breaks. For Windows, it's often in link with settings in the registry (if your software uses it) and software dependencies (3rd party libraries, i.e. Qt). I worked on an embedded software that modified many registry entries to tweak various settings (i.e. prevent the system screensaver) and when moving from one version to another, those usually stopped working. For such, I don't have any good way of testing other than verifying each value you change and see that it has the same effect(s) on the new OS. That is likely to remain a manual job... To simplify that kind of testing, each parameter should be documented so it can easily be verified when you go to the next release.
