Low latency

Video: WebRTC: Mostly the video bits

Posted on by Russell Trafford-Jones

Who better to dig below the surface of WebRTC, which delivers sub-second latency, than Sean DuBois, creator of the Pion WebRTC library? This video takes a different look at WebRTC to others that focus on latency or scaling. Rather Sean looks at congestion control and managing the impacts of congestion noting that people remember how bad the video got and not how nice your sign-up page was.

Congestion is inevitable in large ‘unmanaged’ networks such as the internet and on wifi and cellular networks. Sean points out that the use of MPEG codecs which add dependencies between frames magnify the effect of lost packets. With frame-by-frame codecs, dropping a frame and repeating the last one is barely noticeable, but with MPEG, many more could be damaged. WebRTC was implemented over UDP so it could use its own congestion control.

RTP and RTCP are the key to WebRTC’s congestion control. RTP is well known for carrying real-time media as it’s used for AES67 audio, SMPTE ST 2110 and ST 2022-6 to name just a few standards. RTCP is RTP’s sidekick. Whilst RTP does the legwork of carrying the media, the RTP Control Protocol (RTCP) passes messages to control the flow. In this case, Sean explains, the RTCP channel is used to tell the sender that it’s sending too much video or which packets it’s lost. In terms of mitigating congestion, the source can adjust the bitrate directly or change the resolution or the framerate of the video to bring the bitrate down indirectly.

 

 

Sean shows a summary diagram of congestion controller flow which is built to handle jitter and out of order packets. Buffers are the normal way of fixing out-of-order packets but they have a big downside of adding latency and exacerbating timing problems. WebRTC has to use the RTCP channel to make sure it can map packet timing with NTP, using Sender Reports, as each packet’s timing information is only relative. When packet loss is spotted NACK (negative acknowledgements) are sent via RTCP or if things are worse, a Picture Loss Indication is sent which request a new keyframe. Fixing any impairments that do occur can be done either with FEC or by concealing the error with some form of masking, nowadays this may be based on machine learning.

The talk finishes with a look at a number of innovative projects which use WebRTC in one way or another, including for file transfer.

Watch now!
Speakers

Sean DuBois Sean DuBois
Creator, Pion WebRTC
Developer, Apple

Video: How to Up Your Sports Streaming Game

Posted on by Russell Trafford-Jones

As countries seek to wrest themselves from lockdowns, however long that takes, we see the name of the game will be come out big and make the most of the renewed freedoms. Streaming has certainly seen a boost over the last year despite the challenges, but in order to make the most of that, as we switch up a gear in public life, now’s the time up your game. Sports streaming is likely to see gradual improvement in the number of live fixtures to cover and employees should be able to find protuctivity gains in working more closely with their colleagues when the time is right the share space again.

In this panel from Streaming Media Connect, Jeff Jacobs from VENN talks to Magnus Svensson, from Eyevinn Technology, Ali Hodjat from Intertrust Technologies, Live Sports’ Jef Kethley and Darcy Lorincz from Engine Media. Magnus kicks off the discussion highlighting the state of the sports streaming industry and the trends he’s seeing. Magnus says that streaming providers are moving away from mimicing broadcast services and inovating in their own right. The younger audience are still more interested in highlights clips then older viewers and esports wiith its on-screen chat and interactivity represents a big departure from what we are used to from broadcasters. Low-latency streaming remains important but keeping feeds synchronised within the home is often seen as more important than the absolute latency.
 

 
Jef speaks about the complete cloud infrastructure he built for the Drone Racing League (DRL) which gave a computer to each player and ran the program and drone simulation in the cloud. Looking to the future, he sees streaming as now allowing monetisation of newer sports. Now that it’s easier and/or cheaper to produce lower-interest sports, they can be economoical to monetise and deliver even to a small audience.

Darcy represents workflows where AI is doing the work. AI’s understanding the goals, the numbers on shirts and much of the action within a game. Darcy’s trying to find as many things AI can do to reduce our reliance on humans. Visualisation of data is grown in demand making these stats easily digestable for viewers by overlaying information in new ways on to the screen.

Ali’s view is from the security angle. He’s been focussed on protecting live sports. Weith the push to lower and lower latencies, the value of the streams has increased as they’re more useful to use for betting. At the same time, lower latency makes it harder to add encryption. On top of encryption watermarking individual feeds and quickly identifying them online is a major focus. Protection, though, needs to extend from the media back to the web site itself, the payment gateway, the applications and much else.

The panel session finishes after discussing low-latency, the pros and cons of remote working, co-streaming, low-latency for backhaul/contribution and finishes with a round of advice to use with your service.

Watch now!
Speakers

Magnus Svensson Magnus Svensson
VP Sales and Business Development,
Eyevinn Technology
Ali Hodjat Ali Hodjat
Director Product Marketing,
Intertrust
Jef Kethley Jef Kethley
Executive Director / President
LiveSports, LLC
Darcy Lorincz Darcy Lorincz
Global head of Esports & Business Development,
Engine Media Inc.
Jeff Jacobs Moderator: Jeff Jacobs
General Manager,
VENN

Video: Low Latency Live from a Different Vantage Point

Posted on by Russell Trafford-Jones

Building a low-latency live streaming platform is certainly possible nowadays, but not without its challenges and compromises. Traditionally, HLS-style delivery keeps latency high because of chunk sizes being between 5 and 10 seconds. Pushing that down to 2 seconds, generally seen as the minimum viable chunk size can then cause problems estimating bandwidth and thus breaking ABR.

Tackling these challenges are a host of technologies such as CMAF, Low-Latency HLS (LHLS) and Apple’s LLHLS but this talk takes a different approach to deliver streams with only 3-4 seconds of latency.

Michelle Munson from Eluvio explains that theoretically you could stream chunks in realtime and the delay would be the propagation time over the internet. In reality, though, encoding and transcoding delay add up, plus the CDN can gradually add to a drift of the signal to 15 seconds. ABR is tricky when delivering chunks in a streamed manner because the standard method of determining available bandwidth by measuring the download time gets broken since all chunks come in real-time.

Tackling this, Michelle introduces her to the decentralised fabric which Eluvio have put together which uses dispersed nodes to hold data acting, in some ways as a CDN but the trick here is that the nodes work together to share video. Each node can transcode just in time and also can create playlists on-demand from the distributed metadata in response to client requests. Being able to bring things together dynamically an on the fly removes a lot of latency pinch points from the system.

The result is a system which can deliver content from the encoder to the nodes in around 250ms then a further 50 or so for distribution. To make ABR easier, the player works one segment behind live so it always has a whole segment to download as quickly as it can and thus enabling ABR to work normally.

Michelle finishes by highlighting the results of testing both over time and at scale. The results show that node load stayed low and even in both scenarios delivering 3.5 seconds of latency.

Watch now!
Speakers

Michelle Munson Michelle Munson
CEO and Founder,
Eluvio

Video: Player Optimisations

Posted on by Russell Trafford-Jones

If you’ve ever tried to implement your own player, you’ll know there’s a big gap between understanding the HLS/DASH spec and getting an all-round great player. Finding the best, most elegant, ways of dealing with problems like buffer exhaustion takes thought and experience. The same is true for low-latency playback.

Fortunately, Akamai’s Will Law is here to give us the benefit of his experience implementing his own and helping customers monitor the performance of their players. At the end of the day, the player is the ‘kingpin’ of streaming, comments Will. Without it, you have no streaming experience. All other aspects of the stream can be worked around or mitigated, but if the player’s not working, no one watches anything.

Will’s first tip is to implement ‘segment abandonment’. This is when a video player foresees that downloading the current segment is taking too long; if it continues, it will run out of video to play before the segment has arrived. A well-programmed player will sport this and try to continue the download of this segment from another server or CDN. However, Will says that many will simply continue to wait for the download and, in the meantime, the download will fail.

Tip two is about ABR switching in low-latency, chunked transfer streams. The playback buffer needs to be longer than the chunk duration. Without this precaution, there will not be enough time for the player to make the decision to switch down layers. Will shows a diagram of how a 3-second playback buffer can recover as long as it uses 2-second segments.

Will’s next two suggestions are to put your initial chunk in the manifest by base64-encoding it. This makes the manifest larger but removes the round-trip which would otherwise be used to request the chunk. This can significantly improve the startup performance as the RTT could be a quarter of a second which is a big deal for low-latency streams and anyone who wants a short time-to-play. Similarly, advises Will, make those initial requests in parallel. Don’t wait for the init file to be downloaded before requesting the media segment.

Whilst many of points in this talk focus on the player itself, Will says it’s wise for the player to provide metrics back to the CDN, hidden in the request headers or query args. This data can help the CDN serve media smarter. For instance, the player could send over the segment duration to the CDN. Knowing how long the segment is, the CDN can compare this to the download time to understand if it’s serving the data too slow. Perhaps the simplest idea is for the player to pass back a GUID which the CDN can put in the logs. This helps identify which of the millions of lines of logs are relevant to your player so you can run your own analysis on a player-by-player level.

Will’s other points include advice on how to avoid starting playing at the lowest bandwidth and working up. This doesn’t look great and is often unnecessary. The player could run its own speed test or the CDN could advise based on the initial requests. He advises never trusting the system clock; use an external clock instead.

Regarding playback latency, it pays to be wise when starting out. If you blindly start an HLS stream, then your latency will be variable within the duration of a segment. Will advocates HEAD requests to try to see when the next chunk is available and only then starting playback. Another technique is to vary your playback rate o you can ‘catch up’. The benefit of using rate adjustment is that you can ask all your players to be at a certain latency behind realtime so they can be close to synchronous.

Two great tips which are often overlooked: Request multiple GOPs at once. This helps open up the TCP windows giving you a more efficient download. For mobile, it can also help the battery allowing you to more efficiently cycle the radio on and off. Will mentions that when it comes to GOPs, for some applications its important to look at exactly how long your GOP should be. Usually aligning it with an integer number of audio frames is the way to choose your segment duration.

The talk finishes with an appeal to move to using CMAF containers for streaming ask they allow you to deliver HLS and DASH streams from the same media segments and move to a common DRM. Will says that CBCS encrypted content is now becoming nearly all-pervasive. Finally, Will gives some tips on how players are best to analyse which CDN to use in multi-CDN environments.

Watch now!
Speaker

Will Law Will Law
Chief Architect,
Akamai