AV1’s no longer the slow codec it was when it was released. Real-time encodes and decodes are now practical with open-source software implementations called rav1e for encoding and dav1d for decoding. We’ve also seen in previous talks the SVT-AV1 provides real-time encoding and WebRTC now has a real-time version with the AV1 codec.
In this talk, rav1e contributor Vibhoothi explains more about these projects and how the ARM chipset helps speed up encoding. The Dav1d started project started in 2018 with the intention of being a fast, cross-platform AV1 encoder with a small binary which Vibhoothi says is exactly what we have in 2021. Dav1d is the complementary decoder project. AV1 decoding is found in many places now including in Android Q, in Microsoft’s media extension for it, VLC supports AV1 on linux and macOS thanks to dav1d, AV1 is supported in all major browsers, on NVIDIA and AMD GPUs plus Intel Tiger Lake CPUs. Netflix even use dav1d to stream AV1 onto some mobile devices. Overall, then, we see that AV1 has ‘arrived’ in the sense that it’s in common and increasing use.
The ARM CPU architecture underpins nearly all smartphones and most tablets so ARM is found in a vast number of devices. It’s only relatively recently that ARM has made it into mainstream servers. One big milestone has been the release of Neoverse which is an ARM chip for infrastructure. AWS now offer ARM instances that have a 40% higher performance but a 20% reduced cost. These have been snapped up by Netflix but also by a plethora of non-media companies. Recently Apple has made waves with their introduction of the M1 ARM-based chip for desktops which has benchmarks far in excess of the previous x86 offering which shows that the future for ARM-based implementations of the rav1e encoder and dav1d decoder are bright.
Vibhoothi outlines how dav1d works better on ARM then x86 with improved threading support including hand-written asm optimisations and support for 10-bit assembly. rav1e has wide support in VLC, GStreamer, FFmpeg, libavif and others.
The talk finishes with a range of benchmarks showing how better-than-real-time encoding and decoding is possible and how the number of threads relates to the throughput. Vibhoothi’s final thoughts focus on what’s still missing in the ARM implementations.
A bumper video here with 7 short talks from VideoLAN, Will Law and Hulu among others, all exploring the state of MPEG DASH today, the latest developments and the hot topics such as low latency, ad insertion, bandwidth prediction and one red-letter feature of DASH – multi-DRM.
The first 10 minutes sets the scene introducing the DASH Industry Forum (DASH IF) and explaining who takes part and what it does. Thomas Stockhammer, who is chair of the Interoperability Working Group explains that DASH IF is made of companies, headline members including Google, Ericsson, Comcast and Thomas’ employer Qualcomm who are working to promote the adoption of MPEG-DASH by working to improve the specification, advise on how to put it into practice in real life, promote interoperability, and being a liaison point for other standards bodies. The remaining talks in this video exemplify the work which is being done by the group to push the technology forward.
Meeting Live Broadcast Requirements – the latest on DASH low latency!
Akamai’s Will Law takes to the mic next to look at the continuing push to make low-latency streaming available as a mainstream option for services to use. Will Law has spoken about about low latency at Demuxed 2019 when he discussed the three main file-based to deliver low latency DASH, LHLS and LL-HLS as well as his famous ‘Chunky Monkey’ talk where he explains how CMAF, an implementation of MPEG-DASH, works in light-hearted detail.
In today’s talk, Will sets out what ‘low latency’ is and revises how CMAF allows latencies of below 10 seconds to be achieved. A lot of people focus on the duration of the chunks in reducing latency and while it’s true that it’s hard to get low latency with 10-second chunk sizes, Will puts much more emphasis on the player buffer rather than the chunk size themselves in producing a low-latency stream. This is because even when you have very small chunk sizes, choosing when to start playing (immediately or waiting for the next chunk) can be an important part of keeping the latency down between live and your playback position. A common technique to manage that latency is to slightly increase and decrease playback speed in order to manage the gap without, hopefully, without the viewer noticing.
Chunk-based streaming protocols like HLS make Adaptive Bitrate (ABR) relatively easy whereby the player monitors the download of each chunk. If the, say, 5-second chunk arrives within 0.25 seconds, it knows it could safely choose a higher-bitrate chunk next time. If, however, the chunk arrives in 4.8 seconds, it can choose to the next chunk to be lower-bitrate so as to receive the chunk with more headroom. With CMAF this is not easy to do since the segments all arrive in near real-time since the transferred files represent very small sections and are sent as soon as they are created. This problem is addressed in a later talk in this talk.
To finish off, Will talks about ‘Resync Elements’ which are a way of signalling mid-chunk IDRs. These help players find all the points which they can join a stream or switch bitrate which is important when some are not at the start of chunks. For live streams, these are noted in the manifest file which Will walks through on screen.
Ad Insertion in Live Content:Pre-, Mid- and Post-rolling
Whilst not always a hit with viewers, ads are important to many services in terms of generating the revenue needed to continue delivering content to viewers. In order to provide targeted ads, to ensure they are available and to ensure that there is a record of which ads were played when, the ad-serving infrastructure is complex. Hulu’s Zachary Cava walks us through the parts of the infrastructure that are defined within DASH such as exchanging information on ‘Ad Decision Parameters’ and ad metadata.
In chunked streams, ads are inserted at chunk boundaries. This presents challenges in terms of making sure that certain parameters are maintained during this swap which is given the general name of ‘Content Splice Conditioning.’ This conditioning can align the first segment aligned with the period start time, for example. Zachary lays out the three options provided for this splice conditioning before finishing his talk covering prepared content recommendations, ad metadata and tracking.
Bandwidth Prediction for Multi-bitrate Streaming at Low Latency
Next up is Comcast’s Ali C. Begen who follows on from Will Law’s talk to cover bandwidth prediction when operating at low-latency. As an example of the problem, let’s look at HTTP/1.1 which allows us to download a file before it’s finished being written. This allows us to receive a 10-second chunk as it’s being written which means we’ll receive it at the same rate the live video is being encoded. As a consequence, the time each chunk takes to arrive will be the same as the real-time chunk duration (in this example, 10 seconds.) When you are dealing with already-written chunks, your download time will be dependent on your bandwidth and therefore the time can be an indicator of whether your player should increase or decrease the bitrate of the stream it’s pulling. Getting back this indicator for low-latency streams is what Ali presents in this talk.
Based on this paper Ali co-authored with Christian Timmerer, he explains a way of looking at the idle time between consecutive chunks and using a sliding window to generate a bandwidth prediction.
Implementing DASH low latency in FFmpeg
Open-source developer Jean-Baptiste Kempf who is well known for his work on VLC discusses his work writing an MPEG-DASH implementation for FFmpeg called the DASH-LL. He explains how it works and who to use it with examples. You can copy and paste the examples from the pdf of his talk.
Managing multi-DRM with DASH
The final talk, ahead of Q&A is from NAGRA discussing the use of DRM within MPEG-DASH. MPEG-DASH uses Common Encryption (CENC) which allows the DASH protocol to use more than one DRM scheme and is typically seen to allow the use of ‘FairPlay’, ‘Widevine’ and ‘PlayReady’ encryption schemes on a single stream dependent on the OS of the receiver. There is complexity in having a single server which can talk to and negotiate signing licences with multiple DRM services which is the difficulty that Lauren Piron discusses in this final talk before the Q&A led by Ericsson’s VP of international standards, Per Fröjdh.
From VideoLAN’s Video Dev Days 2018, Romain Bouqueau from GPAC and FireKast discusses MPEG DASH and how this helps us with low-latency streaming. Romain starts by looking at latency in Adaptive Streaming Workflow and then covers 4 false assumptions in streaming. We are walked through all the different points that latency can occur and we see how to reduce each before looking at the drawbacks.
From VideoLAN’s Video Dev Days event 2018, this talk discusses the latest updates to x265, a free software library and application for encoding video streams into the H.265/MPEG-H HEVC compression format, released under GNU GPL.
Pradeep Ramachandran, Principal Engineer at Multicore takes us through:
Views and opinions expressed on this website are those of the author(s) and do not necessarily reflect those of SMPTE or SMPTE Members.
This website is presented for informational purposes only. Any reference to specific companies, products or services does not represent promotion, recommendation, or endorsement by SMPTE