How to Add Subtitles to a Live HLS Stream

In this post I’ll describe how to add subtitles to a live HLS stream.

Subtitles can be added to a live video stream by creating a live subtitle playlist. Before I delve into the details, let’s recap how the playlist for a live video stream works. A live playlist contains a fixed number of entries. Entries are added and removed accordingly as time progresses. Let’s look at an example:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:2
#EXT-X-TARGETDURATION:10
#EXTINF:10,
fileSequence2.ts
#EXTINF:10,
fileSequence3.ts
#EXTINF:10,
fileSequence4.ts
#EXTINF:10,
fileSequence5.ts
#EXTINF:10,
fileSequence6.ts

In this instance there are 5 entries in the playlist, each segment being 10 seconds in duration. Every 10 seconds or so, the first segment is removed and a new one is added to the end of the playlist. The target duration tells the client how often it should go back to the server to retrieve new content. The client will continue to fetch the playlist until the stream ends. This occurs when the #EXT-X-ENDLIST tag is added to the playlist. (There’s more detail in the book.)

A live subtitle playlist behaves in much the same way. For example, at a given point in time the subtitle playlist may look something like this:

#EXTM3U
#EXT-X-TARGETDURATION:10
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:1
#EXTINF:10,
subtitleSegment1.webvtt
#EXTINF:10,
subtitleSegment2.webvtt
#EXTINF:10,
subtitleSegment3.webvtt

I’m only showing 3 segments here to be concise; a real playlist may have more entries. After an additional 10 seconds, it will look like this:

#EXTM3U
#EXT-X-TARGETDURATION:10
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:2
#EXTINF:10,
subtitleSegment2.webvtt
#EXTINF:10,
subtitleSegment3.webvtt
#EXTINF:10,
subtitleSegment4.webvtt

Just like with a live playlist, one segment has been removed and a new one (highlighted in bold) has been added to the end of the playlist. This will continue for the duration of the live broadcast. If you look at your web server’s access log, you should see multiple requests for the subtitle playlist. (The process of creating subtitles in real-time is beyond the scope of this article so I won’t be covering it here.)

Note that the target duration of a live subtitle playlist must be the same as the other streams, which in this instance is 10 seconds.

How does all this hang together? Create a master playlist and add references to the subtitle and live video stream playlist(s). For example:

#EXTM3U
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",DEFAULT=NO,FORCED=NO,URI="subtitles.m3u8",LANGUAGE="en"
#EXT-X-STREAM-INF:BANDWIDTH=1118592,CODECS="mp4a.40.2, avc1.64001f",RESOLUTION=640x360,SUBTITLES="subs"
prog_index.m3u8

This is the playlist that should be used by clients to access the live stream. The subtitles will be displayed (if selected) while the video plays. And that’s it!

(To learn how to add subtitles to an HLS video stream, take a look at the sample chapter from the book.)

2 thoughts on “How to Add Subtitles to a Live HLS Stream

  1. Ape Machine

    Thank you for the information. I have a question, is there any way I could get some help on doing this with FFMPEG? Basically I want to be able to programmatically update the subtitle track to carry along timed metadata with my video feed, which is a live feed streamed to nginx as HLS.

    Reply
  2. machinaeX philip

    thank you for the info and the book!

    i bought the book but am actually looking for more information about live subtitling in hls live streams. so this post sparked my interest.

    at the current moment i am not yet able to stream subtitles live into a live stream and i am a little bit lost about the why:

    i send a rtmp stream from OBS to this wonderful nginx-streaming server: https://github.com/codeworksio/docker-streaming-server and can successfully grab the created stream / playlist in the browser using videojs as the client library.

    But: for it is a live stream the client does not seem to know/get any timecode information and therefore does not display any timeline. it does though display the captions/subtitles i prepared yet never seems to get to the point to also display them. whilst file streaming, the same subtitles did show and i could also live update the subtitles playlist file.

    in the livestream: subtitles are offered yet never displayed.

    Does this have to do with the lack of a timecode in the stream or do i miss something else?

    Reply

Leave a Reply to Ape Machine Cancel reply