Video: I know X, what does WebRTC get me?

WebRTC is now a W3C standard providing sub-second peer-to-peer video and audio streaming with NAT traversal. Widely used for video conferencing, its sub-second latency has also been the focus of video streaming companies such as Millicast and Limelight (to name but two) who aim to deliver this otherwise peer-to-peer technology to thousands or millions of people in under a second enabling interactive video, gamefied streams, auctions and ultra-low-latency sports.

Addressing directly people using other streaming protocols, Pion creator Sean DuBois spoke at SF Video Tech about what WebRTC brings over and above protocols like RTMP, SRT and RIST. At the heart of it, WebRTC, like SRT and RIST, creates a connection over which it can send a variety of data. Whilst we expect media to be sent, actually, file transfer can be easily achieved – let’s not forget the whole of SRT is build upon UDT which is specifically a file delivery utility. Where file transfer can be achieved, so can real-time data & metadata transfer.

Sean quickly summarises WebRTC as a Protocol between (typically) browsers, an peer-to-peer secure connection over which multiple audio & video streams can flow. In common with RIST and other recent protocols, it’s based on many pre-existing
technologies such as SRTP, DTLS, ICE and SDP to deliver signalling, connection management, encryption and communication.

 

 

The list of improvements over RTMP is very long. They’re spelt out concisely in the video so we will highlight just a few here. Importantly, low-latency is key. RTMP was low-latency for its time, but not by today’s standards. Google’s Stadia can boast 125ms video latency for a keypress, explains Sean. DTLS and SRTP are essential for security but are well understood, trusted methods of securing your data. DTLS is pretty much exactly the same as the TLS which secures your bank transfers, just moved into UDP instead of TCP. However, WebRTC can work by exchanging ‘fingerprints’ (DTLS-SRTP) instead of the full trusted certificate infrastructure that underpins TLS on the web. Removing the requirement for certs is a big boost for flexibility and agility as long as you are confident you can exchange fingerprints securely ahead of time.

NAT traversal is also a big boon where, even with both endpoints behind a firewall, endpoints can always find a way to communicate although this does mean that ICE servers are needed to facilitate connectivity. Within broadcasting, however, it’s more likely that you’ll have control of one end so this is less needed. Sean highlights the ability to send multiple quality levels within the same stream using the ‘simulcast’ ability of WebRTC.

Sean then looks at SRT and RIST. Both of these are low-latency streaming protocols which can, both, also provide sub-second streaming for good connections with a relatively low RTT. Sean highlights the lack of SRT and RIST to negotiate the codec in use and their optional security. Being focused more on delivering contribution feeds, they tend to have a more static configuration often created after a programme of testing to ensure the quality will be acceptable to the broadcaster/streaming provider.

To finish, Sean highlights a whole series of interesting, innovative uses of WebRTC from informal group streaming to drones to shared online games to file transfers and more.

Watch now!
Speaker

Sean DuBois Sean DuBois
Developer, Apple
Creator of Pion WebRTC

From WebRTC to RTMP

Continuing our look at the most popular videos of 2020, in common with the previous post on SRT, today we look at replacing RTMP for ingest. This time, WebRTC is demonstrated as an option. With sub-second latency, WebRTC is a compelling replacement for RTMP.
 

 
Read what we said about it the first time in the original article, but you’ll see that Nick Chadwick from Mux takes us through the how RTMP works and where the gaps are as it’s phased out. He steps through the alternatives showing how even the low-latency delivery formats don’t fit the bill for contribution and shows how WebRTC can be a sub-second solution.

RIST and SRT saw significant and continued growth in use throughout 2020 as delivery formats and appear to be more commonly used than WebRTC, though that’s not to say that WebRTC isn’t continuing to grow within the broadcast community. SRT and RIST are both designed for contribution in that they actively manage packet loss, allow any codecs to be used and provide for other data to be sent, too. Overall, this tends to give them the edge, particularly for hardware products. But WebRTC’s wide availability on computers can be a bonus in some circumstances. Have a listen and come to your own conclusion.

Watch now!
Speaker

Nick Chadwick Nick Chadwick
Software Engineer,
Mux

Video: Real-time AV1 in WebRTC

AV1 seems to be shaking off its reputation for slow encoding, now only 2x slower than HEVC. How practical, then is it to put AV1 into a real-time codec aiming for sub-second latency? This is exactly what the Alliance for Open Media are working on as parts of AV1 are perfectly suited for the use case.

Dr Alex from CoSMo Software took the podium at the Alliance for Open Media Research Symposium to lay out the whys and wherefores of updating WebRTC to deliver AV1. He started by outlining the different requirements of real-time vs VoD. With non-live content, encoding time is often unrestricted allowing for complex encoding methods to achieve lower bitrates. Even live CMAF streams aiming to achieve a relatively low 3-second latency have time enough for much more complex encoding than real-time. Encoding, ingest, storage and delivery can all be separated into different parts of the workflow for VoD, whereas real-time is forced to collapse logical blocks down as much as possible. Unsurprisingly, Dr Alex outlines latency as the most important driver in the WebRTC use case.

When streaming, ABR isn’t quite as simple as with chunked formats. The different bit rate streams need to be generated at the encoder to save any transcoding delays. There are two ways of delivering these streams. One is to deliver them as separate streams, the other is to deliver only one, layered stream. The latter method is known as Scalable Video Coding (SVC) which sends a base layer of a low-resolution version of the video which can be decoded on its own. Within that stream, is also the information which builds on top of that video to create a higher-resolution version of the same stream. You can have multiple layers and hence provide information for 3, 4 or more streams.

Managing which streams get to the decoder is done through an SFU (Selective Forwarding Unit) which is a server to which WebRTC clients connect to receive just the stream, or parts of a stream, they need for their current bandwidth capability. It’s important to remember that compared to video conferencing solutions based on WebRTC, that streaming using WebRTC scales linearly. Whilst it’s difficult to hold a meeting with 50 people in a room, it’s possible to optimise what video is sent to everyone by only showing the last 5 speakers in full resolution, the others as thumbnails. Such optimisations are not available for video distribution, rather SFUs and media servers need to be scaled and cascaded. This should be simple, but testing can be difficult but it’s necessary to ensure quality and network resilience at scale.

Cisco have already demonstrated the first real-time AV1-based WebRTC system, though without SVC support. Work is ongoing to deliver improvements to RTP encapsulation of AV1 in WebRTC. For instance, providing Decoding Target Information which embeds information about frames without needing to decode the video itself. This information explains how important each frame is and how it relates to the other video. Such metadata can be used by the SFU or the decoder to understand which frames to drop and send/decode.

Watch now!
Download the slides
Speaker

Alex Gouaillard Dr Alex Gouaillard
Video Codec Working Group – Real-time subgroup, Allience for Open Media
Founder, Directory & CEO, CoSMo Software Consulting Pte. Ltd.
Co-founder & CTO, Millicast

Video: Low Latency, Real-Time Streaming & WebRTC

Can any stream be too low-latency? For some matching broadcast latency, is all they need. But for others, particularly for gaming, gambling or more interactive services, sub-second is a must and they are happy to swap out parts of their technology stack to make that happen. WebRTC is often seen as the best choice for anyone wanting to go achieve an almost instant stream. Started by Google in 2011 for video conferencing applications, WebRTC hit a 1.0 release in 2018 and has been adopted by a number of companies catering to the broadcast market.

WebRTC stands out among the plethora of streaming protocols since it is an actual stream of data and not a series of files transferred just in time. Traditionally buffers have been heavily used in streaming because it was so hard to get data to the player when the mainstream internet was starting out in the 90s and as the mobile internet was establishing itself 10 years later. Whilst those buffers are very helpful in dealing with delayed data, they are a big set back in delivering a low-latency stream. With WebRTC, there is very little buffering, so when using the protocol you have to understand that you may not get all your data delivered and if packets are missing glitches will be seen. This is one significant difference since MPEG DASH and HLS will either show you a blank screen or a perfect rendition of the file chunk that was sent thanks to TCP. This is an example of the compromises of going to sub-second latency; there are no second chances to get the packet again. And whilst this compromise may be a great exchange for an auction site or betting service, for other streaming services, it may be better to use CMAF with 3-second latency.

In this talk, Limelight Networks Video Architect Andrew Crowe introduces WebRTC and explains how it can be deployed. He starts by talking about the video codecs it contains. VP9 has recently been added to the options and for a long time, it was a VP8 technology. Andrew explains how the codecs it carries does have a knock-on effect on its compatibility with browsers. UDP is the underlying technology to all low-latency technologies since the bureaucracy of TCP/IP gets in the way of real-time media streams. Andrew also explains how security pervades WebRTC from its use of DTLS (which is like HTTPS/TLS for UDP) to secure RTP and SRTCP.

The last part of the talk discusses the architectures that CDN LimeLight uses to enable large-scale WebRTC streams including the need to get through firewalls. Andrew discusses how some features of the technology suit small-scale events, but can’t be used with thousands of viewers. He also discusses how adaptive bitrate streams can be delivered, although not within WebRTC itself, there is enough information to implement ABR in addition to the standard stream.

Watch now!
Speakers

Andrew Crowe Andrew Crowe
Video Architect,
Limelight Networks