AV1 seems to be shaking off its reputation for slow encoding, now only 2x slower than HEVC. How practical, then is it to put AV1 into a real-time codec aiming for sub-second latency? This is exactly what the Alliance for Open Media are working on as parts of AV1 are perfectly suited for the use case.
Dr Alex from CoSMo Software took the podium at the Alliance for Open Media Research Symposium to lay out the whys and wherefores of updating WebRTC to deliver AV1. He started by outlining the different requirements of real-time vs VoD. With non-live content, encoding time is often unrestricted allowing for complex encoding methods to achieve lower bitrates. Even live CMAF streams aiming to achieve a relatively low 3-second latency have time enough for much more complex encoding than real-time. Encoding, ingest, storage and delivery can all be separated into different parts of the workflow for VoD, whereas real-time is forced to collapse logical blocks down as much as possible. Unsurprisingly, Dr Alex outlines latency as the most important driver in the WebRTC use case.
When streaming, ABR isn’t quite as simple as with chunked formats. The different bit rate streams need to be generated at the encoder to save any transcoding delays. There are two ways of delivering these streams. One is to deliver them as separate streams, the other is to deliver only one, layered stream. The latter method is known as Scalable Video Coding (SVC) which sends a base layer of a low-resolution version of the video which can be decoded on its own. Within that stream, is also the information which builds on top of that video to create a higher-resolution version of the same stream. You can have multiple layers and hence provide information for 3, 4 or more streams.
Managing which streams get to the decoder is done through an SFU (Selective Forwarding Unit) which is a server to which WebRTC clients connect to receive just the stream, or parts of a stream, they need for their current bandwidth capability. It’s important to remember that compared to video conferencing solutions based on WebRTC, that streaming using WebRTC scales linearly. Whilst it’s difficult to hold a meeting with 50 people in a room, it’s possible to optimise what video is sent to everyone by only showing the last 5 speakers in full resolution, the others as thumbnails. Such optimisations are not available for video distribution, rather SFUs and media servers need to be scaled and cascaded. This should be simple, but testing can be difficult but it’s necessary to ensure quality and network resilience at scale.
Cisco have already demonstrated the first real-time AV1-based WebRTC system, though without SVC support. Work is ongoing to deliver improvements to RTP encapsulation of AV1 in WebRTC. For instance, providing Decoding Target Information which embeds information about frames without needing to decode the video itself. This information explains how important each frame is and how it relates to the other video. Such metadata can be used by the SFU or the decoder to understand which frames to drop and send/decode.
Live Streaming is an important part of not only online viewing, but increasingly of broadcast in general. It’s well documented that live programming is key to keeping linear broadcast’s tradition of ‘everyone watching at once’ which has been diluted – for both pros and cons – by non-linear viewing in recent years.
This panel, as part of IBC’s Content Everywhere, looks at the drivers behind live streaming, how it’s evolving and its future. Bringing together ultra-low-latency platform nanocosmos with managed service provider M2A Media and video player specialists Visual On, Editor of The Broadcast Knowledge, Russell Trafford-Jones starts the conversation asking what gamification is and how this plays in to live streaming.
nanocosmos’s Oliver Lietz explains how gamification is an increasing trend in terms of not only monetising existing content but is a genre in and of itself providing content which is either entirely a game or has a significant interactive element. With such services, it’s clear that latency needs to be almost zero so his company’s ability to deliver one-second latency is why he has experience in these projects.
We hear also from VisualOn’s Michael Jones who explains the low-latency service they were involved in delivering. Here, low-latency CMAF was used in conjunction with local synced-screen technology to ensure that not only was latency low, but second screen devices were not showing video any earlier/later than the main screen. The panel then discussed the importance of latency compared to synchronised viewing and where ultra-low latency was unnecessary.
Valentijn Siebrands from M2A talks about the ability to use live streaming and production in the cloud to deliver lower-cost sports events but also deliver new types of programming. Valentijn then takes us into the topic of analytics, underlining the importance of streaming analytics which reveal the health of your platform/infrastructure as much as the analytics which are most usually talked about; those which tell you the quality of experience your viewers are having and their activities on your app.
The talk concludes with a look to the future, talking about the key evolving technologies of the moment and how they will help us move forward between now and IBC’s Content Everywhere Hub in 2021.
Nobody wants to find out about a big play or major news event on Twitter before they see it in their video stream, so reducing latency is crucial for OTT services’ success. Likewise, ultra-low latency is crucial for interactive streaming applications. Depending on your use case, a few seconds of latency might be fine, or you might need to try to hit that sub-second target.
Learn which technologies and solutions are best for your business, and make sure your viewers get their video on time, every time. In this webinar, you’ll learn the following:
Why it’s important to evaluate and improve latency end-to-end, including software and services, encoder, platform, and player
How to decide which technology and solution is best for your use case (e.g. CMAF, HLS/DASH, WebRTC, Websocket)
How chunked CMAF offers a standards-based approach that allows latency to be decoupled from segment duration
How chunked CMAF leverages existing CDN HTTP capacity to provide low-latency solutions at high scale
How WebRTC can be used to deliver live video sub-second latency at scale, and provide rich, interactive experiences for live streaming applications
How a single misconfigured component can undo any other effort to achieve low latency
How integrated solutions create new business opportunities for low latency interactive use cases
How to achieve low latency across all platforms and devices
VP of Product Strategy,
Moderator: Eric Schumacher-Rasmussen
Streaming Media discusses moving away from Flash and the alternatives. Viewers want better viewing experiences than ever, meaning CDNs have to stay one step ahead. This prompts a move to HTML5 delivery, an emphasis on live streaming, and a commitment to lower latency than ever before – with all this on any device.
This webinar will bring you up to date on the cutting edge of content delivery, with a focus on the following topics:
Why enterprise streaming professionals need to develop a strategy for transitioning away from Flash
From Flash-free to plugin-free: live encoding and playback on any device
Interactive live streaming: Live video and audio has turned away from broadcast towards two-way or multi-directional communication
Why an agnostic content delivery strategy is a must for live streaming inside the enterprise
Scalable distribution of ultra-low-latency live streams worldwide
The importance of utilizing solutions that can analyze the quality of the live streaming viewing experience
THURSDAY, MARCH 22 – 16:00 GMT / 11 AM PT / 2 PM ET / 8 PM CET
Eric Schumacher-Rasmussen, VP/Editor, Streaming Media
Jared Hawkins, VP Product, MediaPlatform
Oliver Lietz, CEO, Nanocosmos
Nivedita Nouvel, VP Marketing, Broadpeak
Tom Racca, CEO, Ramp
Subscribe to get daily updates
Views and opinions expressed on this website are those of the author(s) and do not necessarily reflect those of SMPTE or SMPTE Members.
This website is presented for informational purposes only. Any reference to specific companies, products or services does not represent promotion, recommendation, or endorsement by SMPTE