Demuxed

Video: CMAF and DASH-IF Live ingest protocol

Posted on by Russell Trafford-Jones

Of course, without live ingest of content into the cloud, there is no live streaming so why would we leave such an important piece of the puzzle to an unsupported protocol like RTMP which has no official support for newer codecs. Whilst there are plenty of legacy workflows that still successfully use RTMP, there are clear benefits to be had from a modern ingest format.

Rufael Mekuria from Unified Streaming, introduces us to DASH-IF’s CMAF-based live ingest protocol which promises to solve many of these issues. Based on the ISO BMFF container format which underpins MPEG DASH. Whilst CMAF isn’t intrinsically low-latency, it’s able to got to much lower latencies than standard HLS and LHLS.

This work to create a standard live-ingest protocol was born out of an analysis, Rufael explains, of which part of the content delivery chain were most ripe for standardisation. It was felt that live ingest was an obvious choice partly because of the decaying RTMP protocol which was being sloppy replaced by individual companies doing their own thing, but also because there everyone contributing, in the same way, is of a general benefit to the industry. It’s not typically, at the protocol level, an area where individual vendors differentiate to the detriment of interoperability and we’ve already seen the, then, success of RMTP being used inter-operably between vendor equipment.

MPEG DASH and HLS can be delivered in a pull method as well as pushed, but not the latter is not specified. There are other aspects of how people have ‘rolled their own’ which benefit from standardisation too such as timed metadata like ad triggers. Rufael, explaining that the proposed ingest protocol is a version of CMAF plus HTTP POST where no manifest is defined, shows us the way push and pull streaming would work. As this is a standardisation project, Rufael takes us through the timeline of development and publication of the standard which is now available.

As we live in the modern world, ingest security has been considered and it comes with TLS and authentication with more details covered in the talk. Ad insertion such as SCTE 35 is defined using binary mode and Rufael shows slides to demonstrate. Similarly in terms of ABR, we look at how switching sets work. Switching sets are sets of tracks that contain different representations of the same content that a player can seamlessly switch between.

Watch now!
Speaker

Rufael Mekuria Rufael Mekuria
Head of Research & Standardisation,
Unified Streaming

Video: AV1 at Netflix

Posted on by Russell Trafford-Jones

Netflix have continually been pushing forward video compression and analysis because their assets are played so many times that every bit saved is real money saved. VMAF is a great example of Netflix’s desire to push the state of the art forward. Developed by Netflix and two universities, this new objective metric allowed them to better evaluate the quality of videos using computer analysis and has continued to be the foundation of their work since.

One use of VMAF has been to verify the results of Netflix’s Per-Shot Encoding method which alters encoding parameters for each shot of the film rather than using a fixed set of parameters for the whole film. The Broadcast Knowledge has featured talks on their previous technique, per-title encoding (among others).

AV1, however must be the most famous innovation that Neflix is behind. A founding member of the Alliance for Open Media (AoM), Netflix saw a need a for a better codec and by making an open one, which also played to the needs of other internet giants such as Google, was a good way to create a vibrant community around it driving submissions to the codec itself but also, it is hoped, in the implementation and adoption.

In this two-part talk, LiWei Guo starts off by explaining the ways in which AV1 will be used by Netflix. Since this talk took place, Netflix has started streaming in AV1 to Android clients. LiWei points out that AV1 supports 10-bit video as standard – a notable difference from other codecs like AVC and HEVC. This allows Netflix to use 10-bit without worrying about decoder compatibility and he shows examples of skies and water which are significantly by the use of 10-bit.

Another feature of AV1 is the Film Grain synthesis which seeks to improve encoding efficiency by removing the random film grain of the source during the encode process then inserting a similar random noise on top to recreate the same look and feel. As anything random can’t be predicted, noise such of this is very wasteful for a codec to try and encode, therefore it’s not <a surprise that this can result in as much as a 30% reduction in bitrate. Before concluding, LiWei briefly explains per-shot encoding then shows data showing the overall improvements.

Andrey Norkin, also from Netflix explains their work with Intel on the SVT-AV1 software video encoder which leverages Intel’s SVT technology, a framework optimised for Xeon chips for video encoding and analysis. Netflix’s motivations are to further increase adoption by delivering a data centre-ready, optimised encoder and to create an AV1 encoder they can use to support their own internal research activities (did someone say AV2?). SVT allows for parallelisation, important for any computer nowadays with so many cores available.

Finishing up, Andrey points us to the Github repository, lets us know the development statement (as of November 2019) and looks at the speed increases that have taken off, comparing SVT-AV1 against the reference libaom encoder.

Watch now!
Speakers

Andrey Norkin Andrey Norkin
Senior Research Scientist,
Netflix
LiWei Guo LiWei Guo
Senior Software Engineer,
Netflix

Video: Virtues of Recycling in Multi-rate Encoding

Posted on by Russell Trafford-Jones

Recycling may be good for the environment, but it turns out it’s good for bit rate too. Remembering that MPEG (and similar) video compression includes splitting the picture into blocks, decomposing them into basic patterns and also estimating their motion, this talk wonders whether calculations made on the blocks and the motion of these blocks done on the SD picture can be re-used on the HD picture and then again on the UHD picture. If so, this would surely reduce the computation needed.

“The content is perceptually identical,” explains Alex Giladi from Comcast, “…the only difference is how many pixels it occupies.” as he highlights the apparent wastefulness of ABR encoding where the same video is taken in multiple resolutions and encoded independently. The technique starts by analysing the lowest resolution video for motion and re-using the calculations at a higher resolution. Naturally there are aspects which can’t be captured in the lower resolutions, but also there are sensitivities to the bitrate so Alex explains the refinement options which have been developed to adapt to those.

As the talk wraps up, Alex presents the results found which show that the quality is not degraded and there is a better than 2x speed increase. Finally we look at a real-life flow of encoding.

Watch now!
Speakers

Alex Giladi Alex Giladi
Distinguished Architect,
Comcast

Video: Three Roads to Jerusalem

Posted on by Russell Trafford-Jones

With his usual entertaining vigour, Will Law explains the differences to the three approaches to low-latency streaming: DASH, LHLS and LL-HLS from Apple. Likening them partly to religions that all get you to the same end, we see how they differ and some of the reasons for that.

Please note: Since this video was recorded, Apple has released a new draft of LL-HLS. As described in this great article from Mux, the update’s changes are

  • “Delivering shorter sub-segments of the video stream (Apple call these parts) more frequently (every 0.3 – 0.5s)
  • Using HTTP/2 PUSH to deliver these smaller parts, pushed in response to a blocking playlist request
  • Blocking playlist requests, eliminating the current speculative manifest request polling behaviour in HLS
  • Smaller, delta rendition playlists, which reduces playlist size, which is important since playlists are requested more frequently
  • Faster rendition switching, enabled by rendition reports, which allows clients to see what is happening in another playlist without requesting it in its entirety”[0]

Read the full article for the details and implications, some of which address some points made in the talk.

Furthermore, THEOplayer have released this talk explaining the changes and discussing implementation.

Anyone who saw last year’s Chunky Monkey video, will recognise Will’s near-Oscar-winning animation style as he sets the scene explaining the contenders to the low-latency streaming crown.

We then look at a bullet list of features across each of the three low latency technologies (note Apple’s recent update) which leads on to a discussion on chunked transfer delivery and the challenges of line-rate delivery. A simple view of the universe would say that the ideal way to have a live stream, encoded at a constant bitrate, would be to stream it constantly at that bitrate to the receiver. Whilst this is, indeed, the best way to go, when we stream we’re also keeping one eye on whether we need to change the bitrate. If we get more bandwidth available it might be best to upgrade to a better quality and if we suddenly have contested, slow wifi, it might be time for an emergency drop down to the lowest bitrate stream.

When you are delivered a stream as individual files, you can measure how long they take to download to estimate your available bandwidth. If a file can be downloaded at 1Gbps, then it should always arrive at 1Gbps. Therefore if it arrives at less than 1Gbps we know that there is a bandwidth restriction and can make adjustments. Will explains that for streams delivered with chunked transfer or in real time such as in LL-HLS, this estimation no longer works as the files simply are never available at 1Gbps. He then explains some of the work that has been undertaken to develop more nuanced ways of estimating available bandwidth. It’s well worth noting that the smaller the files you transfer, the less accurate the bandwidth estimation as TCP takes time to speed up to line rate so small 320ms-length video segments are not ideal for maximising throughput.

Continuing to look at the differences, we next look at request rates with DASH at 20 requests per second compared to LL-HLS at 720. This leads naturally to an analysis of the benefits of HTTP/2 PUSH technology used in LL-HLS and the savings that can offer. Will explores the implications, and some of the problems, with last year’s version of the LL-HLS spec, some of which have been mitigated since.

The talk concludes with some work Akamai has done to try and establish a single, common workflow with examples and a GitHub repository. Will shows how this works and the limitations of the approach and finishes with a look at the commonalities in approaches.

[0] From “Low Latency HLS 2: Judgment Day” https://mux.com/blog/low-latency-hls-part-2/

Watch now!
Speakers

Will Law Will Law
Chief Architect,
Akamai