How do you stress test a video streaming app?

Question

I have a a streaming application that I am going to test. I don't really know how to do this. Do I need a special plugin (if so, why)?

Why is it difficult to record a scenario for streaming a video, and you need a special plugin instead? Why is it difficult to simulate with just recording?

Examples are HLS or other streaming technologies.

Answer 1

I believe that you can start with How to Load Test HTTP Live Media Streaming (HLS) with JMeter guide followed by what dzieciou suggests if there still will be open questions.

Answer 2

To realistically load test such protocols, you need to either build your own plugin or use existing one.

Indeed proceeding without a plugin will lead to:

Unrealistic simulation

Indeed, players have a complex behavior in order to reduce pause times (Lag) during the play.

Few examples:

• They select the playlist depending on the bandwidth
• They initially buffer few seconds before starting to play
• They download data while video is being played ...

Another problem is that your test won’t be able to limit the bandwidth for some of your users based on their network type.

Time consuming development

Creating an « imperfect simulation » will take you some time and skill as you need to:

• Master Regular Expression Extractor
• Ensure you handle correctly pauses
• Select correctly the playlist depending on bandwidth
• Reproducing live behavior is very complex and requires a lot of scripting
• Reproducing Player behavior and the way it interacts with your servers requires that you code in Java multi-threading as a player downloads while it plays and reloads playlist for Live Streaming

Limited scalability

Stream manipulate important volumes of data per users. So whatever you do it won't scale OOTB.

Difficult analysis and reporting

You need specific metrics for Video that JMeter does not provide OOTB:

• Buffer fill
• Lag time
• Play time
• Lag ratio
• Download time

You can read this for more details:

• https://www.ubik-ingenierie.com/blog/videostreaming-plugin-with-live-and-abr-streaming/
• https://ubikloadpack.com/video-streaming-performance.php

You can use either a JMeter SAAS or your own Cloud server to overcome possible network issues from your injector and use Apache JMeter + this commercial plugin which simulates realistically the Players behaviour for HLS, MPEG-DASH and MS Smooth.

Disclaimer : We are behind the development of this solution

Answer 3

My company has been working to develop an open source plugin for JMeter to help you do exactly this. It's the HLS plugin for JMeter, which is already on its second version.

In this post we wrote all about how it works and some things to keep in mind before you start testing:

https://abstracta.us/blog/performance-testing/how-to-run-video-streaming-performance-tests-on-hls/