I'm hoping the IVR system code is split up in several layers of code that have different responsibilities:
- detection of typed number by pitch of the audio, which serves as input for;
- the logic of all the possible call flows that decide which audio file id's must be played back
- the part that actually plays back an audio file with a certain audio file id
Layers 1 and 3 should be generic and able to work for any logic implementation of an IVR system. I think your primary interest in this case is testing layer 2. I would investigate if you are able to test layer 2 by itself via unit or integration tests, possibly using mocks for layers 1 and 3.
In this way, you avoid the complexity of the need to set up an actual phone call, dial numbers, etc. Also, it will be much easier to check that the correct audio file is played back by simply asserting that the mocked layer 3 got the command to play back the expected audio file. Alternatively in an end-to-end scenario you would have to compare similarities in audio files and that seems far from easy.
If it is even possible to use this approach, using a Gherkin based test framework (i.e. Cucumber, Behave) will allow you to write the tests exactly how an end user would experience them, which might help when not so technical users are also involved with these tests.