Video: AES67/ST 2110-30 over WAN

Dealing with professional audio, it’s difficult to escape AES67 particularly as it’s embedded within the SMPTE ST 2110-30 standard. Now, with remote workflows prevalent, moving AES67 over the internet/WAN is needed more and more. This talk brings the good news that it’s certainly possible, but not without some challenges.

Speaking at the SMPTE technical conference, Nicolas Sturmel from Merging Technologies outlines the work being done within the AES SC-02-12M working group to define the best ways of working to enable easy use of AES67 on the WAn. He starts by outlining the fact that AES67 was written to expect short links on a private network that you can completely control which causes problems when using the WAN/internet with long-distance links on which your bandwidth or choice of protocols can be limited.

To start with, Nicolas urges anyone to check they actually need AES67 over the WAN to start with. Only if you need precise timing (for lip sync for example) with PCM quality and low latencies from 250ms down to as a little as 5 milliseconds do you really need AES67 instead of using other protocols such as ACIP, he explains. The problem being that any ping on the internet, even to something fairly close, can easily take 16 to 40ms for the round trip. This means you’re guaranteed 8ms of delay, but any one packet could be as late as 20ms known as the Packet Delay Variation (PDV).

Link

Not only do we need to find a way to transmit AES67, but also PTP. The Precise Time Protocol has ways of coping for jitter and delay, but these don’t work well on WAN links whether the delay in one direction may be different to the delay for a packet in the other direction. PTP also isn’t built to deal with the higher delay and jitter involved. PTP over WAN can be done and is a way to deliver a service but using a GPS receiver at each location is a much better solution only hampered by cost and one’s ability to see enough of the sky.

The internet can lose packets. Given a few hours, the internet will nearly always lose packets. To get around this problem, Nicolas looks at using FEC whereby you are constantly sending redundant data. FEC can send up to around 25% extra data so that if any is lost, the extra information sent can be leveraged to determine the lost values and reconstruct the stream. Whilst this is a solid approach, computing the FEC adds delay and the extra data being constantly sent adds a fixed uplift on your bandwidth need. For circuits that have very few issues, this can seem wasteful but having a fixed percentage can also be advantageous for circuits where a predictable bitrate is much more important. Nicolas also highlights that RIST, SRT or ST 2022-7 are other methods that can also work well. He talks about these longer in his talk with Andreas Hildrebrand

Watch now!
Speakers

Nicolas Sturmel Nicolas Sturmel
Product Manager, Senior Technologist,
Merging Technologies

Video: Decoder Complexity Aware AV1 Encoding Optimization

AV1’s been famous for very low encoding speed, but as we’ve seen from panel like this, AV1 encoding times have dropped into a practical range and it’s starting to gain traction. Zoe Liu, CEO of Visionular, is here to talk at Mile High Video 2020 about how careful use of encoding parameters can deliver faster encodes, smooth decodes, and yet balance that balance with codec efficiency.

Zoe starts by outlining the good work that’s been done with the SVT-AV1 encoder which leaves it ready for deployment, as we heard previously from David Ronca of Facebook. Similarly the Dav1d decoder has recently made many speed improvements, now being able to easily decode 24fps on mobiles using between 1.5 and 3 Snapdragon cores depending on resolution. Power consumption has been measured as higher than AVC decoding but less than HEVC. Further to that, hardware support is arriving in many devices like TVs.

Zoe then continues to show ways in which encoding can be sped up by reducing the calculations done which, in turn, increased decoder speed. Zoe’s work has exposed settings that significantly speed up decoding but have very little effect on the compression efficiency of the codec which opens up use cases where decoding was the blocker and a 5% reduction in the ability to compress is a price worth paying. One example cited is ignoring partition sizes of less than 8×8. These small partitions can be numerous and bog down calculations but their overall contribution to bitrate reduction is very low.

All of these techniques are brought together under the heading of Decoder Complexity Aware AV1 Encoding Optimization which, Zoe explains, can result in an encoding speed-up of over two times the original framerate i.e. twice real-time on an Intel i5. Zoe concludes that this creates a great opportunity to apply AV1 to VOD use cases.

Watch now!
Speaker

Zoe Liu Zoe Liu
CEO,
Visionular

Video: LCEVC, The Compression Enhancement Standard

MPEG released 3 codecs last year, VVC, LCEVC and EVC. Which one was unlike the others? LCEVC is the only one that is an enhancement codec, working in tandem with a second codec running underneath. Each MPEG codec from last year addressed specific needs with VVC aiming at comprehensive bitrate savings while EVC aims to push encoding further whilst having a patent-free base layer.

In this talk, we hear from Guido Meardi from V-Nova who explains why LVECV is needed and how it works. LCEVC was made, Guido explains, to cater to an increasingly crowded network environment with more and more devices sending and receiving video both in residential and enterprise. LCEVC helps by reducing the bitrate needed for a certain quality level but, crucially, reduces the computation needed to achieve good quality video which not only benefits IoT and embedded devices but also general computing.

LCEVC uses a ‘base codec’ which is any other codec, often AVC or HEVC, which runs at a lower resolution than the source video. By using this hybrid technique, LCEVC aims to get the best video compression out of the codec yet by running the encode at a quarter resolution, allowing this to be done on low-power hardware. LCEVC then deals with reconstructing two enhancement layers and a, relatively simple, super-resolution upsample. This is all achieved with a simple toolset and all of the LCEVC computation can be done in CPU, GPU or other types of computation; it’s not bound to hardware acceleration.

Guido presents a number of results from tests against a whole range of codecs from VVC to AV1 to plain old AVC. These tests have been done by a number of people including Jan Ozer who undertook a whole range of tests. All of these tests point to the ability of LCEVC to extend bandwidth savings of existing codecs, new and old.

Guido shows an example of a video only comprising edges (apart from mid-grey) and says that LCEVC encodes this not only better than HEVC but also with an algorithm two orders of magnitude less. We then see an example of a pure upsample and an LCEVC encode. Upsampling alone can look good, but it can’t restore information and when there are small textual elements, the benefit of having an enhancement layer bringing those back into the upsampled video is clear.

On the decode side, Guido presents tests showing that decode is also quicker by at least two times if nor more, and because most of the decoding work is involved in decoding the base layer, this is still done using hardware acceleration (for AVC, HEVC and other codecs depending on platform). Because we can still rely on hardware decoding, battery life isn’t impacted.

Watch now!
Speakers

Guido Meardi Guide Meardi
CEO & Co-Founder,
V-Nova

Video: Winner takes all: Unlocking the opportunity in video games and esports.

Even without the pandemic, esports was set to continue its growth over 2020. By the end of 2020, esports had had quite a boost while other sports were canceled. And whilst esports is a large market, it’s still often misunderstood by those unfamiliar with it. This panel recently looked at not only how Covid had changed esports but also how traditional broadcasters can engage with this popular entertainment segment.

The session starts with an overview of the Asian esports market with Daniel Ahmad from Niko Partners. In 2019 there were 1.3 billion gamers in the whole market. In China, there were 321 million PC gamers who spent around $14.6 billion, plus a mobile gaming population which, by 2024, will have doubled their spending to $32 billion across 737 million gamers.

With esports clearly on the rise, the Sports Video Group’s Jason Dachman has brought some of the key players in esports together, Anna Lockwood from Telstra, Steven Jalicy from ESL, David Harris from Guinevere Capital and Yash Patel from Telstra Ventures. Straight off the bat, they tackle the misconceptions that mainstream media has regarding esports. Steven from ESL says people are quick to dismiss the need for quality in esports. In some ways, the quality needs, he says, are more demanding. David Harris says that people overstate esports’ size today and underestimate how big it will be in the future. Anna Lockwood on the other hand sees that people don’t realise how different and powerful the stories told in esports are.
 

 
Asked to talk about how Covid changed ESL’s plans in 2020, he explained that at the final count, they had actually done more events than last year. ESL had already switched to remote working for much of the technical roles in 2018, at the time seen as quite a forward-thinking idea. Covid forced the rest of the workflows to change as stadium appearances were canceled and gamers competed remotely. Fortunately, the nature of esports makes it relatively easy to move the players. Post-Covid, Steven says that arenas will be back as they are very popular and an obvious focus for tournaments. Seeing players in the flesh is an important part of being a fan. But much of the technical changes, are likely to stay at least in part.

Jason Cacheman asks the panel why esports on linear TV hasn’t been very successful. Many of the panelists agree that the core fans simply aren’t that interested in watching on linear TV as they already have a set up to watch streamed which suits them, often, much better. After a question from the audience, their suggestions for incorporating linear TV into esports is to acknowledge that you’re talking to a group of people who are interested but really don’t know, possibly, anything at all. Linear TV is a great place for documentaries and magazine shows which can educate the audience about the different aspects of esports and help them relate. For instance, a FIFA or NBA esports tournament is easier to understand than a Magic: The Gathering or League of Legends tournament. Linear TV can also spend time focussing on the many stories that are involved in esports both in-game and out. Lastly, esports can be a conduit for traditional broadcasters to bring people onto their digital offerings. As an example, the BBC have an online-only channel, BBC Three. By linking esports content on both BBC Two and BBC Three, they can get interested viewers of their broadcast channel to take an interest in their online channel and also have the potential to appeal to core esports fans using their digital-only channel.

Other questions from the audience included the panel’s opinion on VR in esports, use of AI, how to start working in esports, whether it’s easier to bring esports engineers into broadcast or the other way round. The session finished with a look ahead to the rest of 2021. The thoughts included the introduction of bargaining agreements, salary caps, more APIs for data exchange, and that what we saw in 2020 was a knee-jerk reaction to a new problem; 2021 will see real innovation around staying remote and improving streams for producers and, most importantly, the fans.

Watch now!
Speakers

David Harris David Harris
Managing Director,
Guinevere Capital
Steven Jalicy Steven Jalicy
Global Head of Streaming,
ESL Gaming
Anna Lockwood Anna Lockwood
Head of Global Sales,
Telstra Broadcast Services
Yash Patel Yash Patel
General Partner,
Telstra Ventures
Jason Dachman Moderator: Jason Dachman
Chief Editor,
Sports Video Group