EVC

Video: Outlook on the future codec landscape

Posted on by Russell Trafford-Jones

VVC has now been released, MPEG’s successor to HEVC. But what is it? And whilst it brings 50% bitrate savings over HEVC, how does it compare to other codecs like AV1 and the other new MPEG standards? This primer answers these questions and more.

Christian Feldmann from Bitmovin starts by looking at four of the current codecs, AVC, HEVC, VP9 and AV1. VP9 isn’t often heard about in traditional broadcast circles, but it’s relatively well used online as it’s supported on Android phones and brings bitrate savings over AVC. Google use VP9 on Youtube for compatible players and see a higher retention rate. Netflix and Twitch also use it. AV1 is also in use by the tech giants, though its use outside of those who built it (Netflix, Facebook etc.) is not yet apparent. Christian looks at the compatibility of the codecs, hardware decoding, efficiency and cost.

Looking now at the other upcoming MPEG codecs, Christian examines MPEG-5 Essential Video Coding (EVC) which has two profiles: Baseline and Main. The baseline profile only uses technologies which are old enough to be outside of patent claims. This allows you to use the codec without the concern that you may be asked for a fee from a patent holder who comes out of the woodwork. The main profile, however, does have patented technology and performs better. Businesses which wish to use this codec can then pay licences but if an unexpected patent holder appears, each individual tool in the codec can be disabled, allowing you to protect continue using, albeit without that technology. Whilst it is a shame that patents are so difficult to account for, this shows MPEG has taken seriously the situation with HEVC which famously has hundreds of licensable patents with over a third of eligible companies not part of a patent pool. EVC performs 32% better than AVC using the baseline profile and 25% better than HEVC with the main profile.

Next under the magnifying glass is Low Complexity Enhancement Video Coding (LCEVC). We’ve already heard about this on The Broadcast Knowledge from Guido, CEO of V-Nova who gave a deeper look at Demuxed 2019 and more recently at Streaming Media West. Whilst those are detailed talks, this is a great overview of the technology which is actually a hybrid approach to encoding. It allows you to take any existing codec such as AVC, AV1 etc. and put LCEVC on top of it. Using both together allows you to run your base encoder at a lower resolution (say HD instead of UHD) and then deliver to the decoder this low-resolution encode plus a small stream of enhancement information which the decoder uses to bring the video back up to size and add back in the missing detail. The big win here, as the name indicates, is that this method is very flexible and can take advantage of all sorts of available computing power in embedded technology as and in servers. In set-top boxes, parts of the SoC which aren’t used can be put to use. In phones, both the onboard HEVC decoding chip and the CPU can be used. It’s also useful in for automated workflows as the base codec stream is smaller and hence easier to decode, plus the enhancement information concentrates on the edges of objects so can be used on its own by AI/machine learning algorithms to more readily analyse video footage. Encoding time drops by over a third for AVC and EVC.

Now, Christian looks at the codec-du-jour, Versatile Video Codec (VVC), explaining that its enhancements over HEVC come not just from bitrate improvements but techniques which better encode screen content (i.e. computer games), allow for better 360 degree video and reduce delay. Subjective results show up to 50% improvement. For more detail on VVC, check out this talk from Microsoft’s Gary Sullivan.

The talk finishes with answers so audience questions: Which will be the winner, what future device & hardware support will be and which is best for real-time streaming.

Watch now!
Speakers

Christian Feldmann Christian Feldmann
Team lead, Encoding,
Bitmovin

Video: Futuristic Codecs and a Healthy Obsession with Video Startup Time

Posted on by Russell Trafford-Jones

These next 12 months are going to see 3 new MPEG standards being released. What does this mean for the industry? How useful will they be and when can we start using them? MPEG’s coming to the market with a range of commercial models to show it’s learning from the mistakes of the past so it should be interesting to see the adoption levels in the year after their release. This is part of the second session of the Vienna Video Tech Meetup and delves into startup time for streaming services.

In the first talk, Dr. Christian Feldmann explains the current codec landscape highlighting the ubiquitous AVC (H.264), UHD’s friend, HEVC (H.265), and the newer VP9 & AV1. The latter two differentiate themselves by being free to used and are open, particularly AV1. Whilst slow, both the latter are seeing increasing adoption in streaming, but no one’s suggesting that AVC isn’t still the go-to codec for most online streaming.

Christian then introduces the three new codecs, EVC (Essential Video Coding), LCEVC (Low-Complexity Enhancement Video Coding) and VVC (Versatile Video Coding) all of which have different aims. We start by looking at EVC whose aim is too replicate the encoding efficiency of HEVC, but importantly to produce a royalty-free baseline profile as well as a main profile which improves efficiency further but with royalties. This will be the first time that you’ve been able to use an MPEG codec in this way to eliminate your liability for royalty payments. There is further protection in that if any of the tools is found to have patent problems, it can be individually turned off, the idea being that companies can have more confidence in deploying the new technology.

The next codec in the spotlight is LCEVC which uses an enhancement technique to encode video. The aim of this codec is to enable lower-end hardware to access high resolutions and/or lower bitrates. This can be useful in set-top boxes and for online streaming, but also for non-broadcast applications like small embedded recorders. It can achieve a light improvement in compression over HEVC, but it’s well known that HEVC is very computationally heavy.

LCEVC reduces computational needs by only encoding a lower resolution version (say, SD) of the video in a codec of your choice, whether that be AVC, HEVC or otherwise. The decoder will then decode this and upscale the video back to the original resolution, HD in this example. This would look soft, normally, but LCEVC also sends enhancement data to add back in the edges and detail that would have otherwise been lost. This can be done in CPU whilst the other decoding could be done by the dedicated AVC/HEVC hardware and naturally encoding/decoding a quarter-resolution image is much easier than the full resolution.

Lastly, VVC goes under the spotlight. This is the direct successor to HEVC and is also known as H.266. VVC naturally has the aim of improving compression over HEVC by the traditional 50% target but also has important optimisations for more types of content such as 360 degree video and screen content such as video games.

To finish this first Vienna Video Tech Meetup, Christoph Prager lays out the reasons he thinks that everyone involved in online streaming should obsess about Video Startup Time. After defining that he means the time between pressing play and seeing the first frame of video. The longer that delay, the assumption is that the longer the wait, the more users won’t bother watching. To understand what video streaming should be like, he examines Spotify’s example who have always had the goal of bringing the audio start time down to 200ms. Christophe points to this podcast for more details on what Spotify has done to optimise this metric which includes activating GUI elements before, strictly speaking, they can do anything because the audio still hasn’t loaded. This, however, has an impact of immediacy with perception being half the battle.

“for every additional second of startup delay, an additional 5.8% of your viewership leaves”

Christophe also draws on Akamai’s 2012 white paper which, among other things, investigated how startup time puts viewers off. Christophe also cites research from Snap who found that within 2 seconds, the entirety of the audience for that video would have gone. Snap, of course, to specialise in very short videos, but taken with the right caveats, this could indicate that Akamai’s numbers, if the research was repeated today, may be higher for 2020. Christophe finishes up by looking at the individual components which go towards adding latency to the user experience: Player startup time, DRM load time, Ad load time, Ad tag load time.

Watch now!
Speakers

Christian Feldmann Dr. Christian Feldmann
Team Lead Encoding,
Bitmovin
Christoph Prager Christoph Prager
Product Manager, Analytics
Bitmovin
Markus Hafellner Markus Hafellner
Product Manager, Encoding
Bitmovin

Video: Panel Discussion: Hardware is Dead!?

Posted on by Russell Trafford-Jones

The broadcast industry is still producing many new hardware-based products with FPGAs and encoding ASICs still ruling the roost for many companies when it comes to fitting video products into small, power efficient spaces. But the battle continues as software-based products continue to ramp up, server-based products continue to improve and the need to be able to virtualise or place functions into the cloud drives the desire for software-based solutions.

We all know that hardware isn’t dead and that the interest of the topic is where we are today, what is possible and why people are choosing this route and that’s what Broadcast Solutions’ panel discusses in this video. Often called COTS – commercial off-the shelf – hardware, the idea is that you can buy the same server that any other industry does and run your broadcast-related functions on it. When it’s in the cloud, you’re not even selecting the hardware as much as saying how many CPUs and other resources you’d like.

The first comments made come from Marcel Koustaal from Grass Valley who feels that the industry doesn’t entirely appreciate the value software as it’s less tangible than hardware but Pierre Mestrez from Simplylive makes the point that creating products quickly in a modular way is an important part of that company’s success. Zero Density makes the point that they can work quickly as they can build their software on top of other software, Unreal Engine, for example.

Troubleshooting changes for those who run of the systems, we hear from Laurent Petit from EVS. It takes a different set of thinking and processes compared to the idea of swapping a card. The transition to IP, adds Marcel, creates a training opportunity where the technology and the workflows are changing at the same time.

Kuban Altan compares the ability with audio to be processed in real time, easily, by CPUs, by consumer laptops with the future of video processing. Whilst now it’s not so easy to process video with CPUs at the moment, this will change over the coming decade as CPUs improve significantly. Moreover, Kuban looks towards a day where IO is reduced between devices and rather stays within the same CPU/GPU.

The move to software is a global trend, states Laurent, partly because of the imperative to work quickly and efficiently in our small industry whereby we can benefit by building on software developed for similar uses in other industries. The move will take time, however explains Marcel, and will take longer than bringing online the technology itself.

The video ends with a discussion of how clearly hardware-bound devices such as cameras can still embrace software in order, in the future, to create lighter, more flexible cameras which will improve the range of what you can do with each camera and, ultimately, enhance the creative options available to programme makers.

Watch now!
Speakers

Kuban Altan Kuban Altan
Vice President Research and Development,
Zero Density
Marcel Koutstaal Marcel Koutstaal
Senior Vice President and General Manager of Camera Product Group,
Grass Valley
Pierre Mestrez Pierre Mestrez
VP Pre-Sales & Channel Partners,
Simplylive
Laurent Petit Laurent Petit
SVP Product,
EVS

Video: MPEG-5 EVC

Posted on by Russell Trafford-Jones

The MPEG-5: Essential Video Codec (EVC) promises to do what no MPEG standard has done before, deliver great improvements in compression and give assurances over patents. With a novel standardisation process, EVC provides a royalty-free base layer plus licensing details are provided upfront.

SMPTE 2019 saw Jonatan Samuelson take us through the details. Founder of Divideon and an editor of the evolving standard. Jonatan starts by explaining the codec landscape in terms of the new and recent codecs coming online showing how EVC differs including from it’s sister codec, VVC in parallel with which EVC is being developed.

Jonatan explains how the patents are being dealt with, comparing to HEVC, he shows that there is a much more simplified range of patent holders. But importantly, the codec has very granular tools to turn on and off separate tools so that you can exclude any that you don’t wish to use for licensing reasons. This is the first time this level of control has been possible. Along with the royalty-free base layer, this codec hopes to provide companies the control they need in order to safely use the codec with predictable costs and without legal challenges.

Target applications for EVC are realtime encoding, video conferencing but also newer ’emerging’ video formats such as 8K with HDR & WCG. To do this, Jonatan explains the different blocks that create the codec itself ahead of walking us through the results.

Watch now!
Speaker

Jonatan Samuelsson Jonatan Samuelsson
Founder,
Divideon