How to choose between shell script vs compiled, from testing perspective

Question

There is a small program I had to create for some basic data conversion/gateway functionality (network download from one place, text manipulation, network upload to some other place). Because this piece of software is responsible for providing continuous data flow between multiple subsystems, it could become single point of failure in the chain, so it would be prudent to do thorough testing on it.

I have originally started to work on this program as a shell script, because it seemed well suited as the functionality could be implemented in a compact way using common command-line programs.

Realizing the importance of testing made me think though: "Wouldn't it be a better solution to reimplement it as a compiled program?"

It is very well possible that this might result in a more complex solution, as it might not be possible or practical to use readily available libraries for everything. But my feeling is that this would provide better quality code in the end, as it would require less compromise regarding unit testing, especially in an environment with integrated testing tools. Granted, I don't have much experience in writing tests for scripts, so I may be biased.

This is not about script vs compiled in a general sense, of course ones mileage may vary and there are pros and cons on either side. This might also be true for testing tools, as I am sure there are ways to test a script in a thorough way. The one thing that we can safely say though, is that somehow as complexity increases, the probability of a project implemented as a shell script seem to decrease, and I leave it at that.

So in your experience, would it be a sound decision to steer away from using shell scripts even for moderately complex tasks, just because seemingly it might not be as efficiently testable? Or on the contrary, would it worth to study the arts of testing scripts even if I had the luxury of being able to choose not to use scripts in the first place?

Answer 1

Here are some cons for shell scripting that would make me take my decision towards "compiled" case:

• Shell scripts are more dependent on the execution context. So this impacts the portability. It might happen that the required tool which is used in the script is not installed at your new place. When you write your script in programming language you can easily package all you need to a single executable file
• It is harder to write well-structured code in shell scrips which also means it is really hard to extend and maintain such code.
• It is harder to debug shell scripts
• Shell script logic cannot be well parallelized
• Shell scripting has poor IDE support
• Shell scripting has poor error handling mechanisms
• Since there is no compilation phase you will know about some bugs only when your script will crash.
• Not for test scripts but anyway (shell scripts are more sensitive to malicious impact)

Here are some pros for shell scripting:

• Shell script takes less resources (but it is not always true)
• Shell script does not require compilation phase so that you can just go to where your script is deployed and fix the issue by editing file
• Shell script does not require any additional software to interpret the commands