6

As in many IT jobs we work with two environments:

  • Integration: for QA
  • Development: for developers.

Is there a general rule of thumb on what to do when for a bug the developer says it is ok in Development environment but it cannot be validated in Integration environment?

Reasons for this happening might be that the services cannot be duplicated or because of third party involvement e.g. Payment.

Any ideas?

  • 3
    can you use full terms instead of "INT"? – ECiurleo Dec 16 '15 at 15:26
  • 2
    there you have it. – Pixie Dec 16 '15 at 15:29
  • 3
    It can always be validated in the integration environment. Perhaps more setup needs to be performed, perhaps a payment needs to be executed and then backed out. You need to decide if the validation is worthwhile or not. If not, just push it to Production and be ready to back it out quickly. – Joe Strazzere Dec 16 '15 at 15:49
7

Push to make everything testable in the integration environment. The location you are testing in should mimic the production environment as closely as possible. Most payment providers should also have a testing interface, even credit-card validation has fake numbers to use.

If the devs can confirm they fixed it, use their environment, maybe even pair test with a developer so you feel confident to put it into production.

If something is only testable in the production environment I would get some extra developer eyes and do a risk analyses to see if we have covered everything. Afterwards making an educated guess if we can gamble to put it into production.

  • I agree 100% with the first paragraph but it is not always possible. For the rest I do not want to use DEV in any case as it would not be something that I can have as standard procedure in the future i.e. use DEV again and again. Finally I was called as QA to stop using Devs as testers and put in place a QA strategy, so using extra dev eyes is good but again cannot consider it proper QA logic. Thank you. – Pixie Dec 16 '15 at 15:37
  • The ideal world is a plain copy of production to test on, though it's often forbidden for simple privacy laws. – gazzz0x2z Dec 16 '15 at 15:39
  • 4
    I once had a manager and he said, we will make it testable/possible and we did. This sometimes even took drastically changes to the software and or changing libraries or external vendors. But our goal was to automatically test everything, certainly the defects we found. To prevent defects in the future. My advice is to make it testable, this will be worth it on the long run, question yourself how many years and generations developers and testers this product will have and take action accordingly. :) easier said then done, but at-least keep it in mind when making new features. – Niels van Reijmersdal Dec 16 '15 at 15:41
  • i agree but i was looking for more palable answers. thanks again Niels! – Pixie Dec 16 '15 at 15:45
  • 2
    Niels, I have some experience and I agree 100% and every company I've worked with struggles with this. You have to push extra hard to make this happen. You want devs and qa to agree on approaches like mocking, stubbing, fake services, test data, etc. so that the same tools and procedures and processes are being used by dev and qa. All the more reason to have them working hand-in-hand as features are designed and implemented. – Michael Durrant Dec 17 '15 at 12:40
2

This is a very common problem and you will experience it everywhere you go. Every now and then the developers come up saying that they could not replicate a bug on their pc or the development environment. AND As you asked for a rule of thumb, well a simple rule of thumb is to make a platform independent software platform independent. I agree with Niels van Reijmersdal in that Testing Environment should be similar to Production Environment and if something is working in Development Environment but not in Testing Environment you need to ask your development team to look for the dependency issue. Maybe the Development Environment is different than the Testing Environment and there is something on that platform which is causing something to not work.

For example, I always have developers and designers walk up to me saying that a certain web page design looks perfectly fine when they view it in responsive view using the browser tools in Chrome and FireFox. When I see the web page in actual device I find that the design is breaking so I report it and ask them to check whether the animation of JS or whatever it is they used really compatible with mobile devices. Yes it is not possible to check compatibility on every device in the world but we can make a list of supportable environments and check in them. And we need to make sure that the software is compatible in those environments and not in just one specific environment.

So again the rule of thumb is that the software is suppose to work in production environment and for that it is suppose to work in the identical test environment.

On a funnier note you can always tell your developers that we wont be delivering your system to the client or every user of the software will not be using it via your system. ;) :p

Happy Testing :)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.