Tencent EdgeOne Blog

Tutorial

How to Dynamically Rewrite M3U8 Files with Tencent EdgeOne Edge Functions?

Tencent EdgeOne - Product Manager

Jul 10, 2024

At present, major mainstream manufacturers have launched their own edge serverless services, such as CloudFlare Workers, Vercel Edge Runtime, etc. Tencent EdgeOne Edge Function provides a serverless code execution environment deployed on edge nodes. You only need to write business function code and set trigger rules to run the code elastically and securely on edge nodes close to users.

1. What is M3U8 Files?

M3U8 is a file format that is used to store multimedia playlists. It is based on the M3U format, with the "8" signifying that it uses UTF-8 character encoding. M3U8 files are primarily used in HTTP Live Streaming (HLS), a popular adaptive streaming protocol developed by Apple for delivering video and audio content over the internet.

An M3U8 file contains a list of media file URLs, which can be a mix of audio, video, or both. It can also include metadata and other information, such as the duration of each media segment and the sequence of segments. In the context of HLS, M3U8 files are used to create adaptive streaming playlists, allowing the video player to switch between different quality levels or bitrates based on the viewer's network conditions and device capabilities. In this article, we provide a detailed introduction to the relevant content of M3U8.

2. How to Rewrite M3U8 Files with Edge Functions?

In actual use, developers may need to rewrite and process M3U8 files. Here are some common scenarios:

Customized Playlists: Developers may need to choose different media streams based on the user's network conditions, device performance, or geographic location.
Content Security and Access Control: To protect copyrighted content, developers may need to process M3U8 files to implement encryption, add access tokens, and other functions.
Advertising Insertion: Inserting ads into streaming content is a common business model. Developers can modify the M3U8 file to insert ad segments at specific points during video playback.

Rewriting and processing M3U8 files can help developers implement richer and more flexible streaming media application scenarios.

Thanks to the programmability of EdgeOne edge functions, developers can process M3U8 media files at the edge nodes, dynamically modifying and injecting content.

Now, let's take a look at three scenarios to understand how to use edge functions to dynamically rewrite M3U8 files.

2.1 Add URL Authentication

Rewrite the M3U8 file content and add URL authentication parameters for each TS file request, mainly to protect the security of video content. Here are some possible scenarios:

Paid Content: For example, some video websites may offer paid movies, TV shows, or live streaming services. These websites can use URL authentication to ensure that only paid users can access the content;
Member Content: Some websites may provide exclusive content for members. These websites can use URL authentication to restrict access to these contents only for members;

We can implement dynamic URL authentication parameter addition in edge functions to achieve the purpose of protecting content.

Assume the original M3U8 file is:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:10.000000,
8k60_-bulgaria_8k_hdr_60p_fuhd692899210.ts
#EXTINF:10.000000,
8k60_-bulgaria_8k_hdr_60p_fuhd692899211.ts
#EXTINF:10.000000,
8k60_-bulgaria_8k_hdr_60p_fuhd692899212.ts
...
#EXTINF:9.360422,
8k60_-bulgaria_8k_hdr_60p_fuhd6928992133.ts
#EXT-X-ENDLIST

The client's HTTP request to obtain the M3U8 file carries a Type A parameter auth_key, in the form of: http://localhost:8080/xxx.m3u8?auth_key=1698752518-0-00001-ff87dbc0598...

This request triggers the edge function, which retrieves key information such as timestamp, rand, and uid from the auth_key parameter of the M3U8 HTTP request. Based on these parameters, it calculates the Type A authentication parameter auth_key for each TS request and modifies the URL of each TS request in the M3U8 file:

...
async function rewriteLine({ line, querySign }) {
  if (/^\s*$/.test(line)) {
    return line;
  }

  if (line.charAt(0) === "#") {
    if (line.startsWith("#EXT-X-MAP")) {
      const key = await createSign(querySign, line);

      line = line.replace(/URI="([^"]+)"/, (matched, p1) => {
        return p1 ? matched.replace(p1, `${p1}?key=${key}`) : matched;
      });
    }
    return line;
  }

  const key = await createSign(querySign, line);

  return `${line}?${AUTH_KEY_NAME}=${key}`;
}

async function createSign(querySign, line) {
  const { basePath, ts, rand, uid } = querySign;
  const pathname = `${basePath}/${line}`;

  const md5hash = await md5([pathname, ts, rand, uid, SECRET_KEY].join("-"));
  const key = [ts, rand, uid, md5hash].join("-");

  return key;
}
...

After the above processing, the TS requests in the original M3U8 file have been rewritten:

The client player parses the M3U8 file and initiates a TS request with the Type A authentication parameter auth_key, in the form of: http://localhost:8080/xxx.ts?auth_key=1698752518-0-00001-88cbab261a...

The HTTP request to obtain the TS resource will still trigger the execution of the edge function, which will verify the auth_key at this time:

...
async function checkTypeA(urlInfo) {
  const sign = urlInfo.searchParams.get(AUTH_KEY_NAME) || "";
  const elements = sign.split("-");

  if (elements.length !== 4) {
    return {
      flag: false,
      message: "Invalid Sign Format",
    };
  }

  const [ts, rand, uid, md5hash] = elements;
  if (!ts || !rand || !uid || !md5hash) {
    return {
      flag: false,
      message: "Invalid Sign Format",
    };
  }

  if (!isNumber(ts)) {
    return {
      flag: false,
      message: "Sign Expired",
    };
  }

  const now = Math.floor(Date.now() / 1000);
  if (now > Number(ts)) {
    return {
      flag: false,
      message: "Sign Expired",
    };
  }

  const hash = await md5(
    [urlInfo.pathname, ts, rand, uid, SECRET_KEY].join("-")
  );
  if (hash !== md5hash) {
    return {
      flag: false,
      message: "Verify Sign Failed",
    };
  }
  return {
    flag: true,
    message: "success",
  };
}
...

If the auth_key carried by the TS URL is verified successfully, the resource will be returned normally, and the client can play the video properly, as shown in the figure:

If the auth_key carried by the TS URL is invalid or not carried, the edge function will return a 403, and the video will not play properly:

2.2 Inserting Ads

Inserting ads in the M3U8 file by adding TS files is a common method of delivering online video ads. This method allows seamless insertion of ads into the video stream. The main application scenarios are:

Online video platforms: For example, YouTube, Netflix, etc., can insert ads at the beginning, middle, or end of the video;
Live streaming platforms: For example, Twitch, Douyu, etc., can insert ads at specific moments during the live stream;

This method has the following advantages compared to other ad insertion methods:

Flexibility: Ads can be inserted at any time as needed, such as at the beginning, middle, or end of the video;
Compatibility: This method does not rely on a specific player; as long as the device supports HLS, it can play ads;
Personalization: Customized ads can be delivered based on user behavior, interests, or geographical location, etc.;

We can use this idea to dynamically modify the TS file list in the M3U8 file on the edge nodes.

Assume the original M3U8 file is:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:10.000000,
8k60_-bulgaria_8k_hdr_60p_fuhd692899210.ts
#EXTINF:10.000000,
8k60_-bulgaria_8k_hdr_60p_fuhd692899211.ts
#EXTINF:10.000000,
8k60_-bulgaria_8k_hdr_60p_fuhd692899212.ts
...
#EXTINF:9.360422,
8k60_-bulgaria_8k_hdr_60p_fuhd6928992133.ts
#EXT-X-ENDLIST

Using edge functions, inject the pre-prepared ad TS segments:

const AD_TS = `#EXTINF:10.000000,
http://m3u8-1251557890.cos.ap-guangzhou.myqcloud.com/apple_watch_apple_watch_hermes184006680.ts
#EXTINF:10.000000,
http://m3u8-1251557890.cos.ap-guangzhou.myqcloud.com/apple_watch_apple_watch_hermes184006681.ts
#EXTINF:10.000000,
http://m3u8-1251557890.cos.ap-guangzhou.myqcloud.com/apple_watch_apple_watch_hermes184006682.ts
#EXT-X-DISCONTINUITY`;

...
async function handleM3U8(request) {
  ...
  const response = await fetchOrigin(request);
  if (response.status !== 200) {
    return response;
  }

  const originalM3U8 = await response.text();

  const modifiedM3U8 = originalM3U8.replace(
    "#EXT-X-MEDIA-SEQUENCE:0",
    `#EXT-X-MEDIA-SEQUENCE:0\n${AD_TS}`
  );

  return new Response(modifiedM3U8, response);
}
...

After the above processing, the ad TS segments are inserted into the original M3U8 file:

When the client player gets the rewritten M3U8 file, it will play the ad first:

After the ad is played, it seamlessly switches and plays the video content:

The ad content is relatively fixed within a certain period, so there is no need to rewrite the M3U8 file repeatedly. You can use Cache for reasonable caching.

2.3 Geo-targeting

Delivering ads based on geographic location can more effectively achieve advertising goals, improve advertising effectiveness, and provide personalized experiences. Based on Scenario 2, we can further modify the code, using the edge function Request API's request.eo.geo parameter to obtain the client's geographic location, inject different ad content, and achieve refined ad delivery.

const AD_TS_1 = `...AD_TS_1...`;
const AD_TS_2 = `...AD_TS_2...`;

const ADS = {
  CN: AD_TS_1,
  US: AD_TS_2,
};

...
async function handleM3U8(request) {
  ...
  const response = await fetchOrigin(request);
  if (response.status !== 200) {
    return response;
  }

  const originalM3U8 = await response.text();
  const alpha2code = request.eo.geo.countryCodeAlpha2;
  const AD_TS = ADS?.[alpha2code];
  const modifiedM3u8 = originalM3U8;
  
  if (AD_TS) {
    modifiedM3U8 = originalM3U8.replace(
      "#EXT-X-MEDIA-SEQUENCE:0",
      `#EXT-X-MEDIA-SEQUENCE:0\n${AD_TS}`
    );
  }

  return new Response(modifiedM3U8, response);
}
...

Similar to customizing ad TS segments based on request.eo.geo, developers can customize other parts of the TS in the video. Due to the length of the article, this is not demonstrated here.

3. Conclusion

Using Tencent EdgeOne Edge Function to rewrite M3U8 files on edge nodes allows for convenient and flexible streaming media processing while ensuring CDN acceleration. It can dynamically adjust media streams according to the actual situation of users, thereby optimizing business processes and reducing maintenance costs. We have now launched a free trial for the first month. Click here or contact us for more information.

Develop

Edge Function

Web