In this post, we’re going to look at stale playlists in the context of live streaming, what they are and what can cause them, as well as other errors that may be reported by tools like HLS Report and mediastreamvalidator.
Some of the causes of errors when live streaming are caused by things like intermittent network problems or encoder issues that we can’t necessarily do anything about. These things happen from time to time. However, there are things that we can do and we’ll look at some of these in this article.
So what is a stale playlist and what are some of the issues that can cause it? A stale playlist is one that hasn’t been updated with new media segments for a period of time, which is unusual in a live stream. During a live stream, the client must periodically reload the playlist. (This is in contrast to a VOD playlist, which the client only has to download once.) This behaviour is described in section 6.3.4 of the HLS RFC. The playlist is constantly changing as new media segments added and old ones are removed. The time between reloads is determined primarily by the target duration, so you would expect the playlist to change every few seconds. If it doesn’t, then this can indicate a problem with the live stream. Some possible causes are:
- The streaming server stops updating the playlist because of an issue with the encoder or a network problem
- CDN caching where the client retrieves an older version of the playlist because the cache settings are wrong
If the playlist doesn’t update, this will result in the stream appearing to be frozen.
Let’s look at some of the more common problems that can cause stale playlists.
Media Entry URL does not match previous playlist
If you use mediastreamvalidator to check your playlists, you may have come across this error before:
- Media Entry URL does not match previous playlist for MEDIA-SEQUENCE
To understand the error and why it happens, let’s recap how a live playlist gets updated. Let’s say the client loads the playlist for the first time and gets this:
#EXTM3U
#EXT-X-VERSION:4
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:1
#EXTINF:10.000000,
fileSequence1.ts
#EXTINF:10.000000,
fileSequence2.ts
#EXTINF:10.000000,
fileSequence3.ts
#EXTINF:10.000000,
fileSequence4.ts
#EXTINF:5.208333,
fileSequence5.tsThere are 5 media segments in the playlist and the media sequence property (#EXT-X-MEDIA-SEQUENCE) is initialised to 1. Each segment in the playlist is assigned a sequence number. For example, the first segment (fileSequence1.ts) is assigned number 1, the next one (fileSequence2.ts) is assigned number 2, and so on.
When the client reloads the playlist, it receives the following:
#EXTM3U
#EXT-X-VERSION:4
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:2
#EXTINF:10.000000,
fileSequence2.ts
#EXTINF:10.000000,
fileSequence3.ts
#EXTINF:10.000000,
fileSequence4.ts
#EXTINF:5.208333,
fileSequence5.ts
#EXTINF:10.000000,
fileSequence6.tsThe playlist has changed. One segment has been removed, one has been added, and the media sequence number is set to 2. As before, each media segment in the playlist is assigned a sequence number. The first segment (fileSequence2.ts) is assigned 2, the next one is assigned 3, and so on. Each media segment (apart from the last one) has the same sequence number as before in the previous playlist.
However, if the value of the #EXT-X-MEDIA-SEQUENCE tag in the playlist was 1 then the first segment would be assigned the value 1, but in the previous playlist the sequence number of this segment was 2, not 1. The RFC says that “after reloading a Media Playlist, the client SHOULD verify that each Media Segment in it has the same URI (and byte range, if specified) as the Media Segment with the same Media Sequence Number in the previous Media Playlist.” In this instance, the sequence number of the media segment isn’t the same and this is what causes the error.
The next time you come across this error, you should check how your playlist is being generated because it’s an indication that something is wrong because the media sequence number should increase when the playlist is reloaded.
Next up, let’s take a look at caching.
Caching
A client may on occasion receive a stale playlist because it has been cached. Caching allows resources to be stored locally (or in an intermediate cache) for faster retrieval the next time the resource is requested. The amount of time a resource can be cached for depends on the values of certain headers in the response from the server. HTTP specifies a number of headers related to caching. One such header is the Cache-Control header. This header specifies how long the response can be cached for. If the resource is requested again after this time, a fresh copy should be retrieved from the origin server. Here’s an example: Cache-Control: max-age=300, public. This means that the response can be cached for 5 minutes (300 seconds) and shared by anyone.
So how long can a live playlist be cached? We know that the client will reload the playlist periodically based on the value of the target duration. Let’s assume the target duration is 6 seconds. This will result in the client reloading the playlist approximately every 6 seconds. The RFC also states that “if the client reloads a playlist file and finds that it has not changed, then it MUST wait for a period of one-half the target duration before retrying”. This means that the playlist can only be cached for 3 seconds.
Tip: If you are gzip encoding your playlists, make sure you set the Vary: Accept-Encoding header in the response. This instructs caching proxies to store separate versions of the playlist based on the client’s Accept-Encoding header. This ensures that clients that support gzip compression will receive the compressed version of the playlist and clients that don’t will receive the uncompressed version.
Personally, I wouldn’t bother setting the cache control header to anything other than no-cache. This doesn’t mean that the playlist shouldn’t be cached; it means that every time the playlist is requested, the origin server should be checked to see if the resource has changed. This should more or less guarantee that the client will always receive the latest version of the playlist. Live playlists are typically quite small so requesting them from the origin server each time shouldn’t really impact performance.
If you aren’t explicit about how long the playlist should be cached for, you’re leaving it up the browser or other caching proxies, such as a CDN, to decide how to long to cache the playlist for. Make sure you set the Cache-Control header in the server response for requests for the playlist to ensure that clients don’t receive stale versions.
Illegal change in tag
The final error we’ll look at it is an illegal change in a tag. Specifically, the target duration (#EXT-X-TARGETDURATION) tag. The target duration of a live playlist must not change. The value of this header should be set to the maximum duration of any given media segment in the playlist at any point during the broadcast. I’ve seen this error before when using ffmpeg to “live” stream pre-recorded video. It can happen because the keyframe interval is not constant, which causes the media segments to vary in duration, which is reflected in the playlist when it is generated. The target duration is also used by the client to determine how often it should reload the playlist. If it changes, then this can affect how often the client retrieves the playlist, which in turn could result in the stream stalling. To avoid this problem, make sure your encoding process creates media segments that are more or less the same duration.