Video: Low Latency, Real-Time Streaming & WebRTC

Can any stream be too low-latency? For some matching broadcast latency, is all they need. But for others, particularly for gaming, gambling or more interactive services, sub-second is a must and they are happy to swap out parts of their technology stack to make that happen. WebRTC is often seen as the best choice for anyone wanting to go achieve an almost instant stream. Started by Google in 2011 for video conferencing applications, WebRTC hit a 1.0 release in 2018 and has been adopted by a number of companies catering to the broadcast market.

WebRTC stands out among the plethora of streaming protocols since it is an actual stream of data and not a series of files transferred just in time. Traditionally buffers have been heavily used in streaming because it was so hard to get data to the player when the mainstream internet was starting out in the 90s and as the mobile internet was establishing itself 10 years later. Whilst those buffers are very helpful in dealing with delayed data, they are a big set back in delivering a low-latency stream. With WebRTC, there is very little buffering, so when using the protocol you have to understand that you may not get all your data delivered and if packets are missing glitches will be seen. This is one significant difference since MPEG DASH and HLS will either show you a blank screen or a perfect rendition of the file chunk that was sent thanks to TCP. This is an example of the compromises of going to sub-second latency; there are no second chances to get the packet again. And whilst this compromise may be a great exchange for an auction site or betting service, for other streaming services, it may be better to use CMAF with 3-second latency.

In this talk, Limelight Networks Video Architect Andrew Crowe introduces WebRTC and explains how it can be deployed. He starts by talking about the video codecs it contains. VP9 has recently been added to the options and for a long time, it was a VP8 technology. Andrew explains how the codecs it carries does have a knock-on effect on its compatibility with browsers. UDP is the underlying technology to all low-latency technologies since the bureaucracy of TCP/IP gets in the way of real-time media streams. Andrew also explains how security pervades WebRTC from its use of DTLS (which is like HTTPS/TLS for UDP) to secure RTP and SRTCP.

The last part of the talk discusses the architectures that CDN LimeLight uses to enable large-scale WebRTC streams including the need to get through firewalls. Andrew discusses how some features of the technology suit small-scale events, but can’t be used with thousands of viewers. He also discusses how adaptive bitrate streams can be delivered, although not within WebRTC itself, there is enough information to implement ABR in addition to the standard stream.

Watch now!
Speakers

Andrew Crowe Andrew Crowe
Video Architect,
Limelight Networks

Video: Demystifying Video Delivery Protocols

Let’s face it, there are a lot of streaming protocols out there both for contribution and distribution. Internet ingest in RTMP is being displaced by RIST and SRT, whilst low-latency players such as CMAF and LL-HLS are vying for position as they try to oust HLS and DASH in existing services streaming to the viewer.

This panel, hosted by Jason Thibeault from the Streaming Video Alliance, talks about all these protocols and attempts to put each in context, both in the broadcast chain and in terms of its features. Two of the main contribution technologies are RIST and SRT which are both UDP-based protocols which implement a method of recovering lost packets whereby packets which are lost are re-requested from the sender. This results in a very high resilience to packet loss – ideal for internet deployments.

First, we hear about SRT from Maxim Sharabayko. He lists some of the 350 members of the SRT Alliance, a group of companies who are delivering SRT in their products and collaborating to ensure interoperability. Maxim explains that, based on the UDT protocol, it’s able to do live streaming for contribution as well as optimised file transfer. He also explains that it’s free for commercial use and can be found on github. SRT has been featured a number of times on The Broadcast Knowledge. For a deeper dive into SRT, have a look at videos such as this one, or the ones under the SRT tag.

Next Kieran Kunhya explains that RIST was a response to an industry request to have a vendor-neutral protocol for reliable delivery over the internet or other dedicated links. Not only does vendor-neutrality help remove reticence for users or vendors to adopt the technology, but interoperability is also a key benefit. Kieran calls out hitless switching across multiple ISPs and cellular. bonding as important features of RIST. For a summary of all of RIST’s features, read this article. For videos with a deeper dive, have a look at the RIST tag here on The Broadcast Knowledge.

Demystifying Video Delivery Protocols from Streaming Video Alliance on Vimeo.

Barry Owen represents WebRTC in this webinar, though Wowza deal with many protocols in their products. WebRTC’s big advantage is sub-second delivery which is not possible with either CMAF or LL-HLS. Whilst it’s heavily used for video conferencing, for which it was invented, there are a number of companies in the streaming space using this for delivery to the user because of it’s almost instantaneous delivery speed. Whilst a perfect rendition of the video isn’t guaranteed, unlike CMAF and LL-HLS, for auctions, gambling and interactive services, latency is always king. For contribution, Barry explains, the flexibility of being able to contribute from a browser can be enough to make this a compelling technology although it does bring with it quality/profile/codec restrictions.

Josh Pressnell and Ali C Begen talk about the protocols which are for delivery to the user. Josh explains how smoothstreaming has excited to leave the ground to DASH, CMAF and HLS. They discuss the lack of a true CENC – Common Encryption – mechanism leading to duplication of assets. Similarly, the discussion moves to the fact that many streaming services have to have duplicate assets due to target device support.

Looking ahead, the panel is buoyed by the promise of QUIC. There is concern that QUIC, the Google-invented protocol for HTTP delivery over UDP, is both under standardisation proceedings in the IETF and is also being modified by Google separately and at the same time. But the prospect of a UDP-style mode and the higher efficiency seems to instil hope across all the participants of the panel.

Watch now to hear all the details!
Speakers

Ali C. Begen Ali C. Begen
Technical Consultant, Comcast
Kieran Kunhya Kieran Kunhya
Founder & CEO, Open Broadcast Systems
Director, RIST Forum
Barry Owen Barry Owen
VP, Solutions Engineering
Wowza Media Systems
Joshua Pressnell Josh Pressnell
CTO,
Penthera Technologies
Maxim Sharabayko Maxim Sharabayko
Senior Software Developer,
Haivision
Jason Thibeault Moderator: Jason Thibeault
Executive Director,
Streaming Video Alliance

Video: Non-standard Codecs with Standard WebRTC

WebRTC is used on a massive scale thanks to Facebook messenger and Google, but when it comes to video streaming services, some find its open source codec VP8 too restrictive. WebRTC is actively evolving to adapt and become codec agnostic though this work is ongoing. In the meantime, Comcast is here to show us there is a way to inject the codec of your choice into WebRTC.

Finding that many of their video capture devices, CCTV cameras and the like, had hardware AVC encoders, Bryan Meissner explains Comcast didn’t feel it had much of a choice in codec, therefore they looked for a way to make WebRTC to carry AVC.

While forcing an unsupported codec into a protocol wasn’t ideal, they were able to leave much of WebRTC unchanged. The RTP and Data channels were established as normal and peering continued to work as ever. With control of both the send and receive side, the team found they could pick out the data from the WebRTC stack ahead of the normal decoder and feed that into Exoplayer using its API. This allowed playback on Android devices. Bryan goes on to explain the approach for iOS and web browsers. As WebRTC is ‘baked in’ to browsers, there really are very few ways to change the signal flow.

At the end of the day, Comcast made this work and used it in production or many years, Jeff Cardillo explains as he wraps up this video. But he also takes time to talk through some of the problems. Having to bypass parts of a program with parts of another library does increase complexity. Not only does the code become more complex but the code becomes platform specific, you need control over the source and keeping the individual parts synchronously up to date can be a balancing act.

Jeff finishes this talk from Demuxed SF 2019 by elaborating on the mobile and browser tradeoffs at play.

Watch now!
Speakers

Bryan Meissner Bryan Meissner
Sr. Director Software Development and Engineering,
Comcast
Jeff Cardillo Jeff Cardillo
Principal Software Engineer,
Comcast Interactive Media

Video: Low Latency Streaming

There are two phases to reducing streaming latency. One is to optimise the system you already have, the other is to move to a new protocol. This talk looks at both approaches achieving parity with traditional broadcast media through optimisation and ‘better than’ by using CMAF.

In this video from the Northern Waves 2019 conference, Koen van Benschop from Deutsche Telekom examines the large and low-cost latency savings you can achieve by optimising your current HLS delivery. With the original chunk sizes recommended by Apple being 10 seconds, there are still many services out there which are starting from a very high latency so there are savings to be had.

Koen explains how the total latency is made up by looking at the decode, encode, packaging and other latencies. We quickly see that the player buffer is one of the largest, the second being the encode latency. We explore the pros and cons of reducing these and see that the overall latency can fall to or even below traditional broadcast latency depending, of course, on which type (and which country’s) you are comparing it too.

While optimising HLS/DASH gets you down to a few seconds, there’s a strong desire for some services to beat that. Whilst the broadcasters themselves may be reticent to do this, not wanting to deliver online services quicker than their over-the-air offerings, online sports services such as DAZN can make latency a USP and deliver better value to fans. After all, DAZN and similar services benefit from low-second latency as it helps bring them in line with social media which can have very low latency when it comes to key events such as goals and points being scored in live matches.

Stefan Arbanowski from Fraunhofer leads us through CMAF covering what it is, the upcoming second edition and how it works. He covers its ability to use .m3u8 (from HLS) and .mpd (from DASH) playlist/manifest files and that it works both with fMP4 and ISO BMFF. One benefit from DASH is it’s Common Encryption standard. Using this it can work with PlayReady DRM, Fairplay and others.

Stefan then takes a moment to consider WebRTC. Given it proposes latency of less than one second, it can sound like a much better idea. Stefan outlines concerns he has about the ability to scale above 200,000 users. He then turns his attention back to CMAF and outlines how the stream is composed and how the player logic works in order to successfully play at low latency.

Watch now!
Speakers

Koen van Benschop Koen van Benschop
Senior Manager TV Headend and DRM,
Deutsche Telekom
Stefan Arbanowski Stefan Arbanowski
Director Future Applications and Media,
Fraunhofer FOKUS