I like using modern.ie (by Microsoft) because they carry all IE browsers from IE6 to IE11 and it's free.
You basically choose which version of IE you want and it comes prepackaged with its own virtual machine so you don't have to install anything. Just run and test. The downside is the virtual machines are time limited and expire after a certain amount of ...
Does this actually catch, before production, many of the "surprise"
problems we might anticipate? Or is there a more fundamental flaw in
the approach that will cause deleterious changes to pass testing and
You are wise to have a test system that you can use for catching performance issues, but your "scaling" approach is flawed.
No. The items you mention scale very differently and there are far too many factors and resources that will get used.
For instance if I time requests on a local server using an application I'll find things like
1 user = 2 second response average (time per request)
10 users = 2.5 second average
100 users = 2.5 second average
1000 users = 20 second average
This depends on what kind of limitations you are experiencing. You can come up with virtualization. This will let you run several environments in a single host.You can also approach with containerization where you use the same host but isolate your instances with effective segregation of computational resources. Both of these options require some non-trivial ...
If your current Agile practices and approaches are:
Some projects are Agile where continuous feedback from client will be coming, and the developers need to work on the existing item if it's in testing phase too because the deadline of a project will be too short and the client won't extend the deadline.
Then you are not agile
Although Agile does ...
Let’s consider the following situation:
In this case, some ...
One more important thing people forget communication tools, a tester should always ask questions, so we need some communication tools like Hipchat, Skype, slack, whatsapp...etc or any client email tool.
Clipboard Manager is very useful tool for multiple copy history.
Presumably the database is just a piece of your overall system, and your goal is to determine whether database changes break things or slow the system down.
Using a downsized database is a reasonable way to check whether database changes break things.
I'm not sure it makes sense to use a downsized database for load testing. Whether this makes sense ...
Generally you should think of tests as doing
Given X, When Y happens, Then expect Z
This actually occurs at two (or more) levels:
Entire test suite
Certain files, constants, database commands, etc. need to be run before the entire test suite. Often there is no 'tear-down' in this area.
Each test should also have the ...
The only difference I have found in practical terms is colour replication. It's not Bad, just not exact (in my experience).
On the whole, it's not a problem, but if you are making a site meet AAA standard for accessibility, it can be irritating at the times.
A few suggestions:
If you can mock or fake the responses of the integrated apps, you could avoid the need to test live. It does require more work to set this up, but you then have a non-production environment you can use and you will not find yourself inadvertently load testing the integrated apps.
If you must test in production, make sure you know the ...
I prefer to use separate tools for deployment/configuration and running tests, because a tool that focuses on one thing will do better than a tool that tries to do everything.
There are innumerable tools for automating deployments, where a deployment includes installing binaries and configuration files. Some popular ones include Puppet and Ansible.
There are several tools available over the web but my favourite one is Octopus Deploy, because after a relative simple configuration it can deploy your build, setting up automatically your testing environment.
The configuration can be done through a user friendly interface and, it's important at least for me, it's free for a limited number of "tentacles" (...
Test environment will NOT scale proportionally.
But what you can do is:
make test environment as similar to PROD as you possibly can (with possibly less servers, that's what we do: have about half of the servers in PROD)
replicate most of the complexity you can (fallback servers as in PROD)
replicate the load from PROD in a reasonable way (we collect few ...
white box testing can vary from what helping hands mentioned above, to functional or non-functional testing done at the code level.
Usually (but not always) white box testing allows you better flexibility than testing through external interfaces (UI or other) enabling more test coverage.
I am not sure about the official definition but the way ...
The problem with production data is that that the cases are extensive but not exhaustive. Many cases that the system should take into account may not have shown up in the productive system, and if QA is a copy of Production, when they do they will not have been tested.
The test cases should be part of the functional specification and the test environment ...
While it's great to have completely isolated environments, that also has issues (e.g. getting good data sets).
In my experience, companies are going to do what makes sense for their problems. My current employer has 4 environments, with prod and an integration environment that are customer facing. Our dev and staging infrastructures generally have their ...
I firmly believe in having a completely separate QA environment (at least the very first environment where the QAs are going to test the application); nothing shared
I understand that how difficult it is to work in shared environments. I have gone through this:
You always have to remember what all links, configuration setups; you have to change ...
If you have Live environment only I'm afraid you're limited to testing in "dead" times like overnight or weekend. Also make sure your tests have proper setUp and tearDown logic in place so your test would leave the system in the original state (nothing added, nothing removed)
For DevOps (I understand it as running on machines with lower hardware ...
if it makes sense to write some environment-specific tests ?
Absolutely! This is also frequently the job of smoke tests which basically assert that the latest deployment was successfull with very simple tests such as 'can I reach the page or do I get a 404'
Testing the tests is an essential part of any quality automation system.
You should generally test ...
If it were me, I would use a bash or python script that reads from a properties file and reads/executes each query from the file and prints the query string and its return value (your link says Jenkins has email notifications, which is good for this)
Then show the DBA how to edit the properties file to add/edit/remove queries.
I would think it would be ...
In the 5 years since it was asked I also seen the following useful tools gain prominence:
Docker Even for manual testing, using docker, aws and containerization generally is a great way to set up and test other systems.
CloudApp On a mac* and windows this is my go to share tool. I copy my screenshot or video to it and bam I get a cloud link to use....
For applications that exclusively run above the operating system layer, VMs should be sufficient test environments.
If you are doing testing for stuff like full disk encryption (exceptions acknowledged), you should consider limitations such as not being able to test workflows like pre-boot authentication (which would happen below the operating system layer)....
Maybe setting up a VM of your environment is a solution you can look into. So as part of running your tests, you spin up a VM (which has everything where it should be) then run your suite of tests within it.
Virtualization software. You can use tools like VMWare to setup virtual machines which you then run from your existing machine. You might be on a Mac, a Linux box or a Windows machine, and from it run WinXP, Win7, Win8, Win XP /w SP1, etc. IIRC even MS has a Virtual Server which you can script to turn machines on and off, which means one script to run all ...
Whitebox is a keyword used for defining a test category that is opposite to blackbox testing.
The difference between the two is really straightforward:
whitebox testing is performed with a knowledge of what there is behind the scenes. Talking about software testing it basically means you can access the source code.
blackboox testing is performed against an ...
Ideally the best thing is to have UAT or Staging environment which would be exact replica of the production to test your changes there
If your application has "dead" time i.e. in the night or during weekends it should be possible to test it on production environment during that period.
Load testing on scaled down environments is the "last resort". You'll ...
Ideally, testing environments should be independent of each other; otherwise, coupled testing environments would introduce extra time and effort to isolate a variable.
What you described in your question is similar to having a resource farm.
Having a resource farm has its own advantages and disadvantages:
A resource farm offers cost saving: instead of ...