1

I come from a background that strongly emphasizes automated testing baked into the build process. You commit, the code is checked out by the build system, compiled, and tests run against it. If it fails, the artifacts are discarded and cannot even be sent for manual testing. After all, what's the point if even the automated tests are failing? Those are the easy stuff!

When I moved on to my next gig, you may recall I didn't have that same luxury. There was little to no emphasis on automated testing, and was largely considered a waste of time. I actually had one business manager say "Can you imagine how busy your developers would be if they had to go fixing all those bugs? We'd never get anything done!" You can't make this kind of stuff up.

Well now I find myself in a happy medium. There's no tests around this code either, and there's no automated build engine, but that's okay. The folks here are very happy to look at new techniques, incorporate them when you can, and preventing bugs is largely considered a Good Thing. Great. Naturally, I asked if there were any projects that used JUnit (it's a Java gig) and was told no, there's not. But, if something needs to be verified to be correct, it is done at system startup. Here's a code example of how it might work.

public class DateTimeUtility {

    /* returns true if 'late' is later in time than 'early' */
    public boolean isAfter(Calendar early, Calendar late) {
        return late.compareTo(early) > 0;
    }

    /* verifies the integrity of the isAfter method */
    static {
        Calendar early = new GregorianCalendar(1999, 1, 1);
        Calendar late = new GregorianCalendar(2000, 1, 1);
        if(!isAfter(early,late)) throw new Error("DateTimeUtility.isAfter is broken");
    }

}

It doesn't have the fancy features that a unit testing framework would have, but it does appear to be quite useful. Now, date time utilities aren't exactly something that break often, but it's just an example. Clearly something more complicated can be substituted.

The pros I see is that it would be literally impossible to have a failing test in any environment. If you have a bug with one of these tests around it, it simply won't work. (I suppose technically you could catch the Error, but that's a particularly bad practice to be getting into, considering the Error contract.)

The cons would be that you end up with a larger artifact (as the tests are bundled into the artifacts you'd be sending to production) and, depending on the thoroughness of the testing, could have a significant increase to the boot time of your application.

It's not a widespread practice among the team, but it does show up on occasion. Of course, you'd never know it was there unless you looked for it, because needless to say any problems with it don't last long. The first time you load it into your dev environment it explodes spectacularly.

Is this a practice that was a fad at one point? Maybe it goes by a special buzzword? Has anyone run into this and might be able to provide some insight into it, its pros and cons, and other special considerations I'm not seeing? Is it a flat out bad idea and I should actively attempt to migrate these blocks to a bona-fide unit test?

4
  • Are those tests always stored in the components they test? Also, do they tests multiple components? I.e. how dense is dependency diagram between those tests and components they tests? My worry is that it might be hard to maintain those tests when you keep them in the same place as production code. One way to tell that would be to consider how hard would be to move those tests out of components they test.
    – dzieciou
    Nov 24, 2012 at 8:17
  • "if something needs to be verified to be correct, it is done at system startup." Do you verify only single components or also correctness of deployment/configuration? E.g. whether dynamic components/services are found at startup?
    – dzieciou
    Nov 24, 2012 at 8:22
  • Why leave company 1? :) Dec 2, 2012 at 1:51
  • There was a higher rate of crime than my wife and I were prepared for. When we got pregnant we decided to move somewhere a bit safer, and with less racism. Crime and racism in the city, not the job. Hahaha...
    – corsiKa
    Dec 2, 2012 at 1:52

1 Answer 1

3

Our company runs Linux on our production machines. Rather than using the default distribution, we install only those packages necessary to run our software, because each additional, superfluous packages increases the risk that something will go wrong, e.g. a security hole or a resource problem. Similarly, you could argue that running tests in your production environment introduces unnecessary risk. The test in your question looks innocent enough, but as the tests become more complicated, it is possible that a test may introduce bugs in the production system.

1
  • 1
    I agree. In production or test environment, it would make more sense to have rather deployment tests that verifies the system is up and running than tests of single components. The latter should be run before even integration tests.
    – dzieciou
    Nov 24, 2012 at 8:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.