I have to test a messaging service and would like to know if there are any proven techniques developed specifically to test messaging services. I know that for example JMeter has JMS support. That would cover load/performance testing. However, I'm not looking for tools but for concepts and techniques.
I assume you consider a technique to be proven if it has been shown to find bugs. I also assume you are interested in messaging in the backend system sense, not in the instant messaging sense.
A network partition test is a test in which network connections between nodes in a distributed system are temporarily severed. Aphyr has a series of blog posts describing network partition tests of distributed systems. You may be particularly interested in the post on RabbitMQ, and the Kafka posts here and here.