Should testers be looking at code in the development environment?

Question

I am a software tester, working on a project which has multiple different environments:

• dev: developers are doing work here. This code is known to be under development.
• test: this code has been released by the developers, and is expected to be ready to test
• prod: this code has been released by the testers, and is expected to be ready for use by customers

The question is: as a tester, should I be looking at / testing the 'dev' version of the product?

On the one hand, I can see this being helpful, in that if the developers are doing something wrong, or have misinterpreted a requirement, I might be able to catch it earlier in the development process, when it would be easier to fix the problem with less effort.

On the other hand, I can see this being harmful, in that I am looking at the product in a state when it is known not to be ready for me. I could end up filing bug tickets on something which the developers already know is an issue, and intended to fix before releasing it to me. This could both result in my filing more bug tickets than necessary, and causing slowdowns or errors by the developers by unnecessarily redirecting their efforts, or otherwise interfering with their process.

So I am conflicted as to whether doing this is good or not. If anyone has any experience with having done this in the past, I'd appreciate your input on whether it ended up being a net negative, net positive, or if there are any things to watch out for when doing so.

Answer 1

“Testing is the process of evaluating a product by learning about it through exploration and experimentation, which includes: questioning, study, modeling, observation and inference, output checking, etc.”

If you can help people to learn about important aspects of the product earlier and less costly, you should do it.

All your concerns

I am looking at the product in a state when it is known not to be ready for me

I could end up filing bug tickets on something which the developers already know is an issue

Can be tackled with pair testing, after all

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

You can try to both help developers define the automated check that will drive next development step and then quickly explore the results together.

The results can be interesting:

• You learn more about the structure of your product;

• Problems are tackled earlier, before faulty code is shared with other people;

• You can more easily influence to structure of the code;

• And a secret.... You won't need to fill bug reports... just help people to detect problems and fix them... nobody cares about bug reports... (Besides seriously regulated environments)

Answer 2

Yes

The most value that can added to tests happens when the tests are written close to, or with the application code.

There has been a movement in recent years to 'shift-left' in testing so that tests become part of the application being built.

A key principle here is that testing does NOT equal quality. Traditionally testing looked at the final product but this led to a lot of problems that all showed that testing after the item is made is good for verification but does NOT, in of itself, improve quality.

Often the practices of quality are copied from other industries such as manufacturing. Again this copies the model of testing as verification at the end of the process.

This is why the practices of BDD and TDD have grown greatly over the past few years. They attempt to remove the distinction between 'build' and 'test' with 'build testable code with tests built in'. Whether it's TDD or BDD and whether tests are written before (ideally) or just after the app code, the main idea is that you have tests today for the code you write today.

Continuing the focus on value I have found that conversations before development starts can add massive value. I have experienced a 10 minute conversation about functionality that changed the following month of implementation work and I've experienced this multiple times.

Answer 3

You would have to work with your team on this. Do keep in mind that if you have access and look at the code, you would be shifting toward white-box testing techniques. Developing tests against the same requirements or description of intended functionality that the developers used to build their implementation would be a black-box testing technique. Both have their uses.

Even if you do shift into white-box testing techniques, there are also different times to get involved. Pairing with a developer when the code is being written is different than examining as part of a code review, perhaps with other developers.

You'll have to talk it over with your team to figure out the costs and benefits of being involved at different times and what the involvement looks like. Generally speaking, though, making sure that someone with a testing mindset is involved early and throughout the development process is a good thing to start asking questions about how the work will be tested and looking for edge cases that may create difficulty in implementation and test. It can also be a good opportunity to teach the developers good test techniques so they can be more effective at testing their own work (manually or through writing meaningful automated tests) before any formal testing activities.

If you do get involved early, you don't necessarily need to write and file bug tickets. Sometimes, a conversation is all that is needed. If you're using an electronic tool to track the development work, using that tool to ask questions or make observations is sufficient as well, without the need to go through the process of writing up new tickets.

Answer 4

For organizations that work on end-to-end testing, looking at the code within the development environment is quite a usual practice. And since I have been associated with the QA industry for a long time, I believe testers involved in the development process to check code allow early fixing of the issues that might hamper the deployment stage of the product.

Oftentimes, the issues left unchecked when reaching the deployment stage require a massive investment of time and money. On the other hand, organizations that have developers and testers in sync for testing the product from its earliest stages for every module and feature developed have more streamlined process.

Answer 5

I have been associated with an outsourced software testing company for more than nine years and have seen a shift in QA strategies.QA teams have started being involved in the early phases of development before the whole feature is ready for testing.

The efficient implementation of QA in the development/early phase depends on: clear communication between the Dev and QA teams and the availability of product modules developed by the Dev team that can be tested independently.

The benefits of early testing are:

• Identifying and fixing issues in the early phases
• shorter testing cycles
• fewer defects on the test env
• and better product quality in less time.

Additionally, as mentioned in the above answers, filing of the defects is not necessary during early phase. However, if we still want to track it, we can track them as one-liner defects in a spreadsheet or brief JIRA tickets instead of detailed defects.