If all you want is pass-or-fail status on a series of scripts by comparing their output to a known-good output, you can easily setup GNU-Make to run the tests for you. In your Makefile:

# Mark the test rule as not producing a matching file as output
.PHONY: test

# Run a series of tests tests
test:
    ./bin/program1 | diff - ./tests/expected_output1.json
    ./bin/program2 | diff - ./tests/expected_output2.json

Once the expected output is it place, make test should pass or fail depending on whether the programs are currently giving exactly the output they were expected too or not.

This assumes you've generated the expected output by hand. If the test output updates frequently you could automate that too:

# Update test expected output
update_tests:
    ./bin/program1 > ./tests/expected_output1.json
    ./bin/program2 > ./tests/expected_output2.json

Don't forget to add this rule to the .PHONY: set. Once everything is setup, running make update_tests will regenerate the expected output, which you will probably want to commit to your repository. If the nature of your tests ever changes, run that part again and commit it to the repo.

Better yet, if the testes are really so simple you could cheat and write both the tests and the updater in one pass:

# Swap out the test action depending on a user-overridable variable
UPDATE_EXPECTED := false
ifeq ($(UPDATE_EXPECTED),false)
    TEST_ACTION = | diff -
else
    TEST_ACTION = >
endif

test:
    ./bin/program1 $(TEST_ACTION) ./tests/expected_output1.json
    ./bin/program2 $(TEST_ACTION) ./tests/expected_output2.json

Running make test will test as usual by substituting the pipe to diff. On the other hand when you want to update the test output you can run make UPDATE_EXPECTED=true test to rewrite the output files instead.

If all you want is pass-or-fail status on a series of scripts by comparing their output to a known-good output, you can easily setup GNU-Make to run the tests for you. Yes, to write a Makefile is to dabble in dark arts — but this sort of thing is exactly what make does and it will handle a ton of scenarios out of the box that most test frameworks would just be duplicating anyway.

Running tests

In your Makefile:

# Mark the test rule as not producing a matching file as output
.PHONY: test

# Run a series of tests
test:
    ./bin/program1 | diff - ./tests/expected_output1.json
    ./bin/program2 | diff - ./tests/expected_output2.json
    ./bin/program3 | diff - ./tests/expected_output3.json

Once the expected output is it place, make test should pass or fail depending on whether the programs are currently giving exactly the output they were expected too or not. (Note that if you use the .ONESHELL: option, also be sure to set the shell to fail on any failed exit code using .SHELLFLAGS = -e.)

Optional tests

If some of your tests are optional and you want to still run them but see the results of the failure without flunking the test run, you can use the - prefix. In the example above to mark program2 as allowed to fail, simply use:

test:
    ./bin/program1 | diff - ./tests/expected_output1.json
    -./bin/program2 | diff - ./tests/expected_output2.json
    ./bin/program3 | diff - ./tests/expected_output3.json

(Note this won't work with .ONESHELL: as in that configuration only the first line of the recipe can be prefixed, but you can add ||: to the end of the line to achieve the same thing with POSIX shell syntax.)

Optionally at run time you can choose to ignore errors and run all tests instead of aborting on the first failed one by using make -i test.

Generating expected test output

The above assumes you've generated the expected output by hand and saved it in the appropriate place. If the test output updates frequently you could automate that part too:

# Update test expected output
update_tests:
    ./bin/program1 > ./tests/expected_output1.json
    ./bin/program2 > ./tests/expected_output2.json
    ./bin/program3 > ./tests/expected_output3.json

Don't forget to list this target in the .PHONY: set. Once everything is setup, running make update_tests will regenerate the expected output, which you will probably want to commit to your repository. If the nature of your tests ever changes, run this target rule again and commit the changes to your repo.

DRY (Don't Repeat Yourself)

Better yet, if the tests are really as simple as the example above you could cheat and write both the tests and the expected output updater in one pass:

# Swap out the test action depending on a user overridable variable
UPDATE_EXPECTED := false
ifeq ($(UPDATE_EXPECTED),false)
    TEST_ACTION = | diff -
else
    TEST_ACTION = >
endif

test:
    ./bin/program1 $(TEST_ACTION) ./tests/expected_output1.json
    ./bin/program2 $(TEST_ACTION) ./tests/expected_output2.json
    ./bin/program2 $(TEST_ACTION) ./tests/expected_output3.json

Running make test will test as usual by substituting the action with "pipe to diff and compare with file". On the other hand when you want to update the test output you can run make UPDATE_EXPECTED=true test to make the action be a simple redirect that rewrites the output files instead.

# Mark the test rule as not a matching file as output
.PHONY: test

# Run tests
test:
    ./bin/program1 | diff - ./tests/expected_output1.json
    ./bin/program2 | diff - ./tests/expected_output2.json

Once the expected output is it place, make test should pass or fail depending on whether the programs are currently giving exactly the output they were expected too or not.

This assumes you've generated the expected output by hand. If the test output updates frequently you could automate that too:

# Update test expected output
update_tests:
    ./bin/program1 > ./tests/expected_output1.json
    ./bin/program2 > ./tests/expected_output2.json

Don't forget to add this rule to the .PHONY: set. Once everything is setup, running make update_tests will regenerate the expected output, which you will probably want to commit to your repository. If the nature of your tests ever changes, run that part again and commit it to the repo.

Better yet, you could cheat and write both the tests and the updater in one pass:

# Swap out the test action depending on a user-overridable variable
UPDATE_EXPECTED := false
ifeq ($(UPDATE_EXPECTED),false)
    TEST_ACTION = | diff -
else
    TEST_ACTION = >
endif

test:
    ./bin/program1 $(TEST_ACTION) ./tests/expected_output1.json
    ./bin/program2 $(TEST_ACTION) ./tests/expected_output2.json

Running make test will test as usual by substituting the pipe to diff. On the other hand when you go to update the test output you can run make UPDATE_EXPECTED=true test to rewrite the output files instead.

If all you want is pass-or-fail status on a series of scripts by comparing their output to a known-good output, you can easily setup GNU-Make to run the tests for you. In your Makefile:

# Mark the test rule as not producing a matching file as output
.PHONY: test

# Run a series of tests tests
test:
    ./bin/program1 | diff - ./tests/expected_output1.json
    ./bin/program2 | diff - ./tests/expected_output2.json

Once the expected output is it place, make test should pass or fail depending on whether the programs are currently giving exactly the output they were expected too or not.

This assumes you've generated the expected output by hand. If the test output updates frequently you could automate that too:

# Update test expected output
update_tests:
    ./bin/program1 > ./tests/expected_output1.json
    ./bin/program2 > ./tests/expected_output2.json

Don't forget to add this rule to the .PHONY: set. Once everything is setup, running make update_tests will regenerate the expected output, which you will probably want to commit to your repository. If the nature of your tests ever changes, run that part again and commit it to the repo.

Better yet, if the testes are really so simple you could cheat and write both the tests and the updater in one pass:

# Swap out the test action depending on a user-overridable variable
UPDATE_EXPECTED := false
ifeq ($(UPDATE_EXPECTED),false)
    TEST_ACTION = | diff -
else
    TEST_ACTION = >
endif

test:
    ./bin/program1 $(TEST_ACTION) ./tests/expected_output1.json
    ./bin/program2 $(TEST_ACTION) ./tests/expected_output2.json

Running make test will test as usual by substituting the pipe to diff. On the other hand when you want to update the test output you can run make UPDATE_EXPECTED=true test to rewrite the output files instead.

# Mark the test rule as not a matching file as output
.PHONY: test

# Run tests
test:
    ./bin/program1 | diff - ./tests/expected_output1.json
    ./bin/program2 | diff - ./tests/expected_output2.json

Once the expected output is it place, make test should pass or fail depending on whether the programs are currently giving exactly the output they were expected too or not.

This assumes you've generated the expected output by hand. If the test output updates frequently you could automate that too:

# Update test expected output
update_tests:
    ./bin/program1 > ./tests/expected_output1.json
    ./bin/program2 > ./tests/expected_output2.json

Don't forget to add this rule to the .PHONY: set. Once everything is setup, running make update_tests will regenerate the expected output, which you will probably want to commit to your repository. If the nature of your tests ever changes, run that part again and commit it to the repo.

Better yet, you could cheat and write both the tests and the updater in one pass:

# Swap out the test action depending on a user-overridable variable
UPDATE_EXPECTED := false
ifeq ($(UPDATE_EXPECTED),false)
    TEST_ACTION = | diff -
else
    TEST_ACTION = >
endif

test:
    ./bin/program1 $(TEST_ACTION) ./tests/expected_output1.json
    ./bin/program2 $(TEST_ACTION) ./tests/expected_output2.json

Running make test will test as usual by substituting the pipe to diff. On the other hand when you go to update the test output you can run make UPDATE_EXPECTED test to rewrite the output files instead.

# Mark the test rule as not a matching file as output
.PHONY: test

# Run tests
test:
    ./bin/program1 | diff - ./tests/expected_output1.json
    ./bin/program2 | diff - ./tests/expected_output2.json

Once the expected output is it place, make test should pass or fail depending on whether the programs are currently giving exactly the output they were expected too or not.

This assumes you've generated the expected output by hand. If the test output updates frequently you could automate that too:

# Update test expected output
update_tests:
    ./bin/program1 > ./tests/expected_output1.json
    ./bin/program2 > ./tests/expected_output2.json

Don't forget to add this rule to the .PHONY: set. Once everything is setup, running make update_tests will regenerate the expected output, which you will probably want to commit to your repository. If the nature of your tests ever changes, run that part again and commit it to the repo.

Better yet, you could cheat and write both the tests and the updater in one pass:

# Swap out the test action depending on a user-overridable variable
UPDATE_EXPECTED := false
ifeq ($(UPDATE_EXPECTED),false)
    TEST_ACTION = | diff -
else
    TEST_ACTION = >
endif

test:
    ./bin/program1 $(TEST_ACTION) ./tests/expected_output1.json
    ./bin/program2 $(TEST_ACTION) ./tests/expected_output2.json

Running make test will test as usual by substituting the pipe to diff. On the other hand when you go to update the test output you can run make UPDATE_EXPECTED=true test to rewrite the output files instead.