Video: The Video Codec Landscape 2020

2020 has brought a bevy of new codecs from MPEG. These codecs represent a new recognition that the right codec is the one which fits your hardware and your business case. We have the natural evolution of HEVC, namely VVC which trades on complexity to achieve impressive bit rate savings. There’s a recognition that sometimes a better codec is one that has lower computation, namely LCEVC which enables a step-change in quality for lower-power equipment. And there’s also EVC which has a license-free mode to reduce the risk for companies which prefer low-risk deployments.

Christian Feldmann from Bitmovin takes the stage in this video to introduce these three new contenders in an increasingly busy codec landscape. Christian starts by talking about the incumbents namely AVC, HEVC, VP9 and AV1. He puts their propositions up against the promises of these new codecs which are all at the point of finalisation/publication. With the current codecs, Christian looks at what the hardware and software support is like as well as the licencing.

EVC (Essential Video Codec) is the first focus of the presentation whose headline feature is more reliably licence landscape. The first offer is the baseline profile which has no licencing as it uses technologies which are old enough to be outside of patents. The main profile does require licencing and does allow much better performance. Furthermore, the advanced tools in the main profile can each be turned off individually hence avoiding patents that you don’t want to licence. The hope is that this will encourage the patent holders to licence the technology in a timely manner else the customer can, relatively easily, walk away. Using the baseline only should provide 32% better than AVC and the main profile can give up to a 25% benefit over HEVC.

LCEVC (Low Complexity Enhancement Video Coding) is next which is a new technique for encoding which is actually two codecs working together. It uses a ‘base’ codec at low resolution like AVC, HEVC, AV1 etc. This low fidelity version is then accompanied by enhancement information so that the low-resolution base can be upscaled to the desired resolution can be corrected with relevant edges etc. added. The overall effect is that complexity is kept low. It’s designed as a software codec which can fit into almost any hardware by using the hardware decoders in SoCs/CPUs (i.e. Intel QuickSync) plus the CPU itself which deals with the enhancement application. This ability to fit around hardware makes the codec ideal for improving the decoding capability to existing hardware. It stands up well against AVC providing at least 36% improvement and at worst improves slightly upon HEVC bitrates but with much-reduced encoder computation.

VVC (Versatile Video Coding) is discussed by Christian but not in great detail as Bitmovin will be covering that separately. As an evolution of HEVC, it’s no surprise that bitrate is reduced by at least 40%, though encoding complexity has gone up 10-fold. This is similar to HEVC compared to its predecessor AVC. VVC has some built-in features not delivered as standard before such as special modes for screen content (such as computer games) and 360-degree video.

Free to watch now!

Speaker

Christian Feldmann Christian Feldmann
Lead encoding engineer,
Bitmovin

Video: High-Throughput JPEG 2000 (HTJ2K) for Content Workflows

Published last year, high-throughput JPEG 2000 (HTJ2K) is an update to the J2K we know well in the broadcast industry making it much faster. Whilst JPEG 2000 has found a home in low-latency broadcast contribution, it’s also part of the archive exchange format (AXF) because, unlike most codecs, JPEG 2000 has a mathematically lossless mode. HTJ2K takes JPEG 2000 and replaces some of the compression with a much faster algorithm allowing for much faster decoding of well 10 to 28 times faster in many circumstances.

The codec market seems waking up to the fact that multiple types of codec are needed to support the thousands of use cases that we have in the Media and Entertainment and beyond. It’s generally well known that codecs live in a world where they are optimising bitrate at the expense of latency and quality. But the advent of MPEG 5 Part 2, also known as LCEVC show that there is value in optimising to reduce complexity of encoding. In some ways, this is similar to saying reduce the latency, but in the LCEVC example, the aim is to allow low-power or low-complexity equipment to deal with HD or UHD video where otherwise that might not have been possible. With HTJ2K we have a similar situation where it’s worth getting 10x more throughput when managing and processing your archive at the expense of 5% more bitrate.

This talk from the EBU’s Network Technology Seminar hears from Pierre-Anthony Lemieux and Michael Smith who explain the need for this codec and the advantages. One important fact is that the encoding itself hasn’t been changed, just some of the maths around it. This means that you can take previously encoded files and process them into HTJ2K without changing any of the video data. This allows lossy J2K files to be converted without any degradation due to re-encoding and minimises conversion time for lossless files. Another motivator for this codec is cloud workflows where speed of compression is important to reduce costs. Michael Smith also explores the similarities and differences of High-Throughput J2K with JPEG XS

Watch now!
Speakers

Pierre-Anthony Lemieux Pierre-Anthony Lemieux
Sandflow Consulting
Michael Smith
Wavelet Consulting

Video: HLS and DASH Multi-Codec Encoding & Packaging

As we saw yesterday, there’s an increasingly buoyant market for video codecs and whilst this is a breath of fresh air after AVC’s multi-decade dominance, we will likely never again see a market which isn’t fragmented with several dominant players, say AV1, AVC, VVC and VP9, each sharing 85% market share relatively equally and then ‘the rest’ bringing up the rear. So multi-codec distribution to home viewers is going to have to deal with delivering different codecs to different people.

fuboTV do this today and Nick Krzemienski is here to tell us how. Starting with an overview of fuboTV primarily streams both live and on VOD. Nick shows us the workflow they use and then explains how their AVC & HEVC combined workflow is set up. Starting with the ideal case where a single fmp4 is encoded into both AVD and HEVC, he proposes you would simply package both into an HLS and DASH manifest and let players work out the rest. Depending on your players, you may have to split out your manifests into single-codec files.

DRM’s very important for a sports broadcaster so Nick looks at how this might be achieved. CMAF allows you to deliver m3u8 and mpd files using CENC (Common ENCryption). This promises a single DRM process ahead of packaging, but the reality, we hear from Nick, is that you’ll need two sets of media for HLS and DASH if you’re going to use CENC.

When you’re delivering multiple manifest and, hence, multiple sources, how do you manage this? Nick outlines, and shows the code, of how he achieves this at the edge. Using Lamda, he’s able to look at the incoming requests and existing files at the CDN to deliver the right asset with the logic done close to the viewer. Nick closes by with his thoughts on the future for streaming and answering questions from the audience.

Watch now!
Download the presentation
Speakers

Nick Krzemienski Nick Krzemienski
Engineering Lead, VOD Encoding & Operations, fuboTV
Maintainer & Editor, awesome.video
Dom Robinson Host: Dom Robinson
Director and Creative Firestarter, id3as
Contributing Editor, StreamingMedia.com, UK

Video: Outlook on the future codec landscape

VVC has now been released, MPEG’s successor to HEVC. But what is it? And whilst it brings 50% bitrate savings over HEVC, how does it compare to other codecs like AV1 and the other new MPEG standards? This primer answers these questions and more.

Christian Feldmann from Bitmovin starts by looking at four of the current codecs, AVC, HEVC, VP9 and AV1. VP9 isn’t often heard about in traditional broadcast circles, but it’s relatively well used online as it’s supported on Android phones and brings bitrate savings over AVC. Google use VP9 on Youtube for compatible players and see a higher retention rate. Netflix and Twitch also use it. AV1 is also in use by the tech giants, though its use outside of those who built it (Netflix, Facebook etc.) is not yet apparent. Christian looks at the compatibility of the codecs, hardware decoding, efficiency and cost.

Looking now at the other upcoming MPEG codecs, Christian examines MPEG-5 Essential Video Coding (EVC) which has two profiles: Baseline and Main. The baseline profile only uses technologies which are old enough to be outside of patent claims. This allows you to use the codec without the concern that you may be asked for a fee from a patent holder who comes out of the woodwork. The main profile, however, does have patented technology and performs better. Businesses which wish to use this codec can then pay licences but if an unexpected patent holder appears, each individual tool in the codec can be disabled, allowing you to protect continue using, albeit without that technology. Whilst it is a shame that patents are so difficult to account for, this shows MPEG has taken seriously the situation with HEVC which famously has hundreds of licensable patents with over a third of eligible companies not part of a patent pool. EVC performs 32% better than AVC using the baseline profile and 25% better than HEVC with the main profile.

Next under the magnifying glass is Low Complexity Enhancement Video Coding (LCEVC). We’ve already heard about this on The Broadcast Knowledge from Guido, CEO of V-Nova who gave a deeper look at Demuxed 2019 and more recently at Streaming Media West. Whilst those are detailed talks, this is a great overview of the technology which is actually a hybrid approach to encoding. It allows you to take any existing codec such as AVC, AV1 etc. and put LCEVC on top of it. Using both together allows you to run your base encoder at a lower resolution (say HD instead of UHD) and then deliver to the decoder this low-resolution encode plus a small stream of enhancement information which the decoder uses to bring the video back up to size and add back in the missing detail. The big win here, as the name indicates, is that this method is very flexible and can take advantage of all sorts of available computing power in embedded technology as and in servers. In set-top boxes, parts of the SoC which aren’t used can be put to use. In phones, both the onboard HEVC decoding chip and the CPU can be used. It’s also useful in for automated workflows as the base codec stream is smaller and hence easier to decode, plus the enhancement information concentrates on the edges of objects so can be used on its own by AI/machine learning algorithms to more readily analyse video footage. Encoding time drops by over a third for AVC and EVC.

Now, Christian looks at the codec-du-jour, Versatile Video Codec (VVC), explaining that its enhancements over HEVC come not just from bitrate improvements but techniques which better encode screen content (i.e. computer games), allow for better 360 degree video and reduce delay. Subjective results show up to 50% improvement. For more detail on VVC, check out this talk from Microsoft’s Gary Sullivan.

The talk finishes with answers so audience questions: Which will be the winner, what future device & hardware support will be and which is best for real-time streaming.

Watch now!
Speakers

Christian Feldmann Christian Feldmann
Team lead, Encoding,
Bitmovin