In my company, we use a separate test environment where we copy production data daily. This environment is periodically used to detect issues like the ones you have encountered.
The vast majority of our testing is carried out with synthesized, non-production data. Some of this is produced by hand, but most is produced by scripts we build. We periodically ...
I like using modern.ie (by Microsoft) because they carry all IE browsers from IE6 to IE11 and it's free.
You basically choose which version of IE you want and it comes prepackaged with its own virtual machine so you don't have to install anything. Just run and test. The downside is the virtual machines are time limited and expire after a certain amount of ...
No. The items you mention scale very differently and there are far too many factors and resources that will get used.
For instance if I time requests on a local server using an application I'll find things like
1 user = 2 second response average (time per request)
10 users = 2.5 second average
100 users = 2.5 second average
1000 users = 20 second average
Does this actually catch, before production, many of the "surprise"
problems we might anticipate? Or is there a more fundamental flaw in
the approach that will cause deleterious changes to pass testing and
You are wise to have a test system that you can use for catching performance issues, but your "scaling" approach is flawed.
Jing (Screens capturing tools)
Small little tool that let's you record a short video of the screen. I personally use this when taking screenshots or attempting to explain an issue becomes difficult. The next best thing is to record the problem. It also saves the clip on the cloud so you don't have to worry about finding a location to host the video.
This depends on what kind of limitations you are experiencing. You can come up with virtualization. This will let you run several environments in a single host.You can also approach with containerization where you use the same host but isolate your instances with effective segregation of computational resources. Both of these options require some non-trivial ...
If your current Agile practices and approaches are:
Some projects are Agile where continuous feedback from client will be coming, and the developers need to work on the existing item if it's in testing phase too because the deadline of a project will be too short and the client won't extend the deadline.
Then you are not agile
Although Agile does ...
There are several ways you can work with this situation.
No-work - let the tests for the new feature fail until the feature is migrated to the staging environments. This can be an option if the tests don't block builds or aren't reporting directly to management.
"smart" detection - as Dale suggested, use some coding to detect whether the feature is ...
One more important thing people forget communication tools, a tester should always ask questions, so we need some communication tools like Hipchat, Skype, slack, whatsapp...etc or any client email tool.
Clipboard Manager is very useful tool for multiple copy history.
Let’s consider the following situation:
In this case, some ...
Generally you should think of tests as doing
Given X, When Y happens, Then expect Z
This actually occurs at two (or more) levels:
Entire test suite
Certain files, constants, database commands, etc. need to be run before the entire test suite. Often there is no 'tear-down' in this area.
Each test should also have the ...
Presumably the database is just a piece of your overall system, and your goal is to determine whether database changes break things or slow the system down.
Using a downsized database is a reasonable way to check whether database changes break things.
I'm not sure it makes sense to use a downsized database for load testing. Whether this makes sense ...
The only difference I have found in practical terms is colour replication. It's not Bad, just not exact (in my experience).
On the whole, it's not a problem, but if you are making a site meet AAA standard for accessibility, it can be irritating at the times.
A few suggestions:
If you can mock or fake the responses of the integrated apps, you could avoid the need to test live. It does require more work to set this up, but you then have a non-production environment you can use and you will not find yourself inadvertently load testing the integrated apps.
If you must test in production, make sure you know the ...
There are several tools available over the web but my favourite one is Octopus Deploy, because after a relative simple configuration it can deploy your build, setting up automatically your testing environment.
The configuration can be done through a user friendly interface and, it's important at least for me, it's free for a limited number of "tentacles" (...
white box testing can vary from what helping hands mentioned above, to functional or non-functional testing done at the code level.
Usually (but not always) white box testing allows you better flexibility than testing through external interfaces (UI or other) enabling more test coverage.
I am not sure about the official definition but the way ...
The problem with production data is that that the cases are extensive but not exhaustive. Many cases that the system should take into account may not have shown up in the productive system, and if QA is a copy of Production, when they do they will not have been tested.
The test cases should be part of the functional specification and the test environment ...
I've done this in many places myself, where we copied over production data because it was so very useful in troubleshooting issues that were only apparent in Customer data and it also provided additional test scenarios and data structures that we did not then need to always recreate. It's extremely useful, and if your Development Team is balking at this you ...
One of the most important tools I use all the time is virtual machines. I have a library of different test environments saved and if I need to test something in any one of those environments I just fire up that VM. The ability to snapshot the virtual machine means that I can corrupt the environment and very easily come back to a pristine state.
I would hate ...
I prefer to use separate tools for deployment/configuration and running tests, because a tool that focuses on one thing will do better than a tool that tries to do everything.
There are innumerable tools for automating deployments, where a deployment includes installing binaries and configuration files. Some popular ones include Puppet and Ansible.
Test environment will NOT scale proportionally.
But what you can do is:
make test environment as similar to PROD as you possibly can (with possibly less servers, that's what we do: have about half of the servers in PROD)
replicate most of the complexity you can (fallback servers as in PROD)
replicate the load from PROD in a reasonable way (we collect few ...
While it's great to have completely isolated environments, that also has issues (e.g. getting good data sets).
In my experience, companies are going to do what makes sense for their problems. My current employer has 4 environments, with prod and an integration environment that are customer facing. Our dev and staging infrastructures generally have their ...
I firmly believe in having a completely separate QA environment (at least the very first environment where the QAs are going to test the application); nothing shared
I understand that how difficult it is to work in shared environments. I have gone through this:
You always have to remember what all links, configuration setups; you have to change ...
If you have Live environment only I'm afraid you're limited to testing in "dead" times like overnight or weekend. Also make sure your tests have proper setUp and tearDown logic in place so your test would leave the system in the original state (nothing added, nothing removed)
For DevOps (I understand it as running on machines with lower hardware ...
In the 5 years since it was asked I also seen the following useful tools gain prominence:
Docker Even for manual testing, using docker, aws and containerization generally is a great way to set up and test other systems.
CloudApp On a mac* and windows this is my go to share tool. I copy my screenshot or video to it and bam I get a cloud link to use....
One alternative I have seen is to have the company set up a "test" environment within their domain, then give specific personnel within your company access to it. That setup provides two benefits: it provides a pre-deployment test environment to evaluate new versions directly against production data in a protected environment and also provides a platform to ...
For applications that exclusively run above the operating system layer, VMs should be sufficient test environments.
If you are doing testing for stuff like full disk encryption (exceptions acknowledged), you should consider limitations such as not being able to test workflows like pre-boot authentication (which would happen below the operating system layer)....
Perhaps you can either automatically detect whether the feature is installed, or manually create some kind of test configuration file appropriate to each environment. Then use that information to deselect tests that are not appropriate to the configuration.