Return to Revisions

# Mark the test rule as not a matching file as output
.PHONY: test

# Run tests
test:
    ./bin/program1 | diff - ./tests/expected_output1.json
    ./bin/program2 | diff - ./tests/expected_output2.json

Once the expected output is it place, make test should pass or fail depending on whether the programs are currently giving exactly the output they were expected too or not.

This assumes you've generated the expected output by hand. If the test output updates frequently you could automate that too:

# Update test expected output
update_tests:
    ./bin/program1 > ./tests/expected_output1.json
    ./bin/program2 > ./tests/expected_output2.json

Don't forget to add this rule to the .PHONY: set. Once everything is setup, running make update_tests will regenerate the expected output, which you will probably want to commit to your repository. If the nature of your tests ever changes, run that part again and commit it to the repo.

Better yet, you could cheat and write both the tests and the updater in one pass:

# Swap out the test action depending on a user-overridable variable
UPDATE_EXPECTED := false
ifeq ($(UPDATE_EXPECTED),false)
    TEST_ACTION = | diff -
else
    TEST_ACTION = >
endif

test:
    ./bin/program1 $(TEST_ACTION) ./tests/expected_output1.json
    ./bin/program2 $(TEST_ACTION) ./tests/expected_output2.json

Running make test will test as usual by substituting the pipe to diff. On the other hand when you go to update the test output you can run make UPDATE_EXPECTED=true test to rewrite the output files instead.