Video: S-Frame in AV1: Enabling better compression for low latency live streaming.

Streaming is such a success because it manages to deliver video even as your network capacity varies while you are watching. Called ABR (Adaptive Bitrate), this short talk asks how we can allow low-latency streams to nimbly adapt to network conditions whilst keeping the bitrate low in the new AV1 codec.

Tarek Amara from Twitch explains the idea in AV1 of introducing S-Frames, sometimes called ‘switch frames’, which take the role of the more traditional I or IDR frames. If a frame is marked as an IDR frame, this means the decoder knows it can start decoding from this frame without worrying that it’s referencing some data that came before this frame. By doing this, you can allow frequent points at which a decoder can enter a stream. IDR frames are typically I frames which are the highest bandwidth frames, by a large proportion. This is because they are a complete rendition of a frame without any of the predictions you find in P and B frames.

Because IDR frames are so large, if you want to keep overall bandwidth down, you should reduce the number of them. However, reducing the number of frames reduces the number if ‘in points’ for for the stream meaning a decoder then has to wait longer before it can start displaying the stream to the viewer. An S-Frame brings the benefits of an IDR in that it still marks a place in the stream where the decoder can join, free of dependencies on data previously sent. But the S-Frame is takes up much less space.

Tarek looks at how an S-Frame is created, the parameters it needs to obey and explains how the frames are signalled. To finish off he presents tests run showing the bitrate improvements that were demonstrated.
Watch now!
Speaker

Tarek Amara Tarek Amara
Engineering Manager, Video Encoding,
Twitch

Video: 2019 What did I miss? – SRT

We’re looking at the most popular posts of 2019 now as The Broadcast Knowledge takes a break over the holiday season. Twitch’s Alex Converse had one of the most visited posts of the year in his video detailing how SRT works. It’s a great technical resource for developers and engineers wanting to understand more than just the highlights of SRT. Did it do well because it was Alex? Because the San Francisco’s Video Tech meet up is a well known part of Demuxed’s community for ‘engineers working with video’ or because its title? Any or all of these could be true and it wouldn’t invalidate it’s usefulness or its popularity. So if you haven’t already, read more about it here, or click play below.

Another SRT talk of interest this year you may want to catch up on was the IBC SRT Open Source Technical panel which looked at the general SRT features and looked at the pros and cons against SRT. The panel looked at a case study with Red Bee Media and South American broadcaster Globo and the use of RTP and SRT together. Read more detail here or click here to watch for free

Speaker

Alex Converse Alex Converse
Streaming Video Software Engineer,
Twitch

Video: Understanding esports production

Esports is here to stay and brings a new dimension on big events which combine the usual challenges of producing and broadcasting events at scale with less usual challenges such as non-standard resolutions and frame rates. This session from the IBC 2019 conference looks at the reality of bringing such events to life.

The talk starts with a brief introduction to some Esports-only terms before heading into the discussions starting with Simon Eicher who talks about his switch toward typical broadcast tools for Esports which has helped drive better production values and story telling. Maxwell Trauss from Riot Games explains how they incubated a group of great producers and were able keep production values high by having them working on shows remotely worldwide.

Blizzard uses the technique of using a clean ‘world feed’ which is shared worldwide for regions to regionalise it with graphics and language before then broadcasting this to the world. In terms of creating better storytelling, Blizzard have their own software which interprets the game data and presents it in a more consumable way to the production staff.

Observers are people who control in-game cameras. A producer can call out to any one of the observers. The panel talks about how separating the players from the observers from the crowd allows them to change the delay between what’s happening in the game and each of these elements seeing it. At the beginning of the event, this creates the opportunity to move the crowd backwards in time so that players don’t get tipped off. Similarly they can be isolated from the observers for the same effect. However, by the end of the game, the delays have been changed to bring everyone back into present time for a tense finale.

Corey Smith from Blizzard explains the cloud setup including clean feeds where GFX is added in the cloud. This would lead to a single clean feed from the venue, in the end. ESL, on the other hand choose to create their streams locally.

Ryan Chaply from Twitch explains their engagement models some of which reward for watching. Twitch’s real-time chat banner also changes the way productions are made because the producers have direct feedback from the viewers. This leads, day by day, to tweaks to the formats where a production may stop doing a certain thing by day three if it’s not well received, conversely when something is a hit, they can capitalise on this.

Ryan also talks about what they are weighing up in terms of when they will start using UHD. Riot’s Maxwell mentions the question of whether fans really want 4K at the moment, acknowledging it’s an inevitability, he asks whether the priority is actually having more/better stats.

The panel finishes with a look to the future, the continued adoption of broadcast into Esports, timing in the cloud and dealing with end-to-end metadata and a video giving a taste of the Esports event.

Watch now!
Speakers

Simon Eicher Simon Eicher
Executive Producer, Director of Broadcast, eSports Services,
ESL
Ryan Chaply Ryan Chaply
Senior Esports Program Manager,
Twitch
Corey Smith Corey Smith
Director, Live Operations Broadcast Technology Group,
Blizzard
Maxwell Trauss Maxwell Trauss
Broadcast Architect,
Riot Games
Jens Fischer Jens Fischer
Global Esport Specialist and Account Manager D.A.CH,
EVS

Video: Towards a healthy AV1 ecosystem for UGC platforms


Twitch is an ambassador for new codecs and puts its money where its mouth is; it is one of the few live streaming platforms which streams with VP9 – and not only at, with cloud FPGA acceleration thanks to Xylinx’s acquisition of NGCODEC.

As such, they have a strong position on AV1. With such a tech savvy crowd, they stream most of their videos at the highest bitrate (circa 6mbps). With millions of concurrent videos, they are highly motivated to reduce bandwidth where they can and finding new codecs is one way to do that.

Principal Research Engineer, Yueshi discusses Twitch’s stance on AV1 and the work they are doing to contribute in order to get the best product at the end of the process which will not only help them, but the worldwide community. He starts by giving an overview of Twitch which, while many of us are familiar with the site, the scale and needs of the site may be new information and drive the understanding of the rest of the talk.

Reduction in bitrate is a strong motivator, but also the fact that supporting many codecs is a burden. AV1 promises a possibility of reducing the number of supported codecs/formats. Their active contribution in AV1 is also determined by the ‘hand wave’ latency; a simple method of determining the approximate latency of a link which is naturally very important to a live streaming platform. This led to Twitch submitting a proposal for SWITCH_FRAME which is a technique, accepted in AV1, which allows more frequent changes by the player between the different quality/bitrate streams available. This results in a better experience for the user and also reduced bitrate/buffers.

YueShi then looks at the projected AV1 deployment roadmap and discusses when GPU/hardware support will be available. The legal aspect of AV1 – which promises to be a free-to-use codec is also discussed with the news that a patent pool has formed around AV1.

The talk finishes with a Q&A.

Watch now!

Speakers

Yueshi Shen Yueshi Shen
Principal (Level 7) Research Engineer & Engineering Manager,
Twitch