How do you test your scripts?

Question

I'm looking for ways to test my scripts and some patterns you use/recommend. For instance, how would you test a script such as this one:

#!/usr/bin/env bash
build="$1"

# check that build number is passed
[[ -z $build ]] && { echo "missing argument: build"; exit 1; }

cd ./visual || exit 1

curl "https://" | grep -o 'https://[^"]*' > artifacts.txt

awk -v TOKEN="$TOKEN" '/current/ {print $0 "?token=" TOKEN;}' artifacts.txt \
| xargs -P4 -l1 wget \
  --content-disposition \
  --force-directories \
  --no-host-directories \
  --cut-dirs=5

# let's remove our token from the filename
shopt -s globstar
for file in **/*; do
  mv $file ${file/\?*/}
done

rm artifacts.txt

How do I check I have been cd to the correct directory?

Currently I stub cd to check how it's call.

  cd() { echo "cd $*"; exit; }  # mock
  export -f cd
  run my-script.bash "1234"

  [[ "${lines[0]}" == "cd ./visual" ]]

But then the script continue to run everything after that.

How to test a pipe?

curl "https://" | grep -o 'https://[^"]*' > artifacts.txt

or

awk -v TOKEN="$TOKEN" '/current/ {print $0 "?token=" TOKEN;}' artifacts.txt \
| xargs -P4 -l1 wget \
  --content-disposition \
  --force-directories \
  --no-host-directories \
  --cut-dirs=5

What would you test and how? Is it possible to test part of the pipe?

How would you check value of an env var

awk -v TOKEN="$TOKEN" '/current/ {print $0 "?token=" TOKEN;}' artifacts.txt \

How would you chek that we use globstar?

shopt -s globstar
for file in **/*; do
  mv $file ${file/\?*/}
done

related: github issue

Answer 1

Script is a program, so you test it as any program.

But testing such fundamental constructs as pipe, env var, or command cd is inherently hard to test from the shell script using it, and you just assume it works, or develop a test suite using underlying language (C) to simulate shell activities and comparing the results.

In a programming language you just assume that assignment or comparison "works". You test (using basic commands of your programming language) construct build using these basic commands. If you cannot trust your basic commands, you cannot test, because you have no means to express your checks.

Edit: to test if cd worked, you can save output of pwd, like pwd > pwd.txt and later read and parse it to see if its content is the right directory. To test pipes, you can save output to a file and feed it to the next step, to see it you will get identical results. Of course much harder would be to test the "staggering effect" of processing a pipe output from something like tail -f filename.

But you are using higher level language to test lower-level concepts. For a dedicated attacker, it would be possible (even if not trivial) to create tools to fool your tests (provide correct output for test, and do something else).

In fact, when I was first time learning Unix, I subverted ls this way to hijack it and confuse my fellow students. Course leader realized what I did and went along, because my fake ls was not too much sophisticated and was easy to see it is a fake.

In my experience, if you start writing your code with goal of being testable, it results in (sometimes distinctly) different design, where unit test can access processing steps. But all is just competent programming craft, no magic.

Answer 2

If you have a complex shell script and think it's worth the effort, you can use a container/virtualization technology. Let the script run in there, export the shell log to a text file and make it your golden master. You can additionally diff the resulting directory/file tree or other relevant stuff with tools such as diffoscope. This way, you have an automated regression test you can easily execute and even integrate in your CI/CD pipeline.