Video: VVC – The new Versatile Video Coding standard

Posted on by Russell Trafford-Jones

The Codec landscape is a more nuanced place than 5 years ago, but there will always be a place for a traditional Codec that cuts file sizes in half while harnessing recent increases in computation. Enter VVC (Versatile Video Codec) the successor to HEVC, created by MPEG and the ITU by JVET (Joint Video Experts Team), which delivers up to 50% compression improvement by evolving the HEVC toolset and adding new features.

In this video Virginie Drugeon from Panasonic takes us through VVC’s advances, its applications and performance in this IEEE BTS webinar. VVC aims not only to deliver better compression but has an emphasis on delivering at higher resolutions with HDR and as 10-bit video. It also acknowledges that natural video isn’t the only video used nowadays with much more content now including computer games and other computer-generated imagery. To achieve this, VVC has had to up its toolset.

 

 

Any codec is comprised of a whole set of tools that carry out different tasks. The amount that each of these tools is used to encode the video is controllable, to some extent, and is what gives rise to the different ‘profiles’, ‘levels’ and ‘tiers’ that are mentioned when dealing with MPEG codecs. These are necessary to allow for lower-powered decoding to be possible. Artificially constraining the capabilities of the encoder gives maximum performance guarantees for both the encoder and decoder which gives manufacturers control over the cost of their software and hardware products. Virginie walks us through many of these tools explaining what’s been improved.

Most codecs split the image up into blocks, not only MPEG codecs but the Chinese AVS codecs and AV1 also do. The more ways you have to do this, the better compression you can achieve but this adds more complexity to the encoding so each generation adds more options to balance compression against the extra computing power now available since the last codec. VVC allows rectangles rather than just squares to be used and the size of sections can now be 128×128 pixels, also covered in this Bitmovin video. This can be done separately for the chroma and luma channels.

Virginie explains that the encoding is done through predicting the next frame and sending the corrections on top of that. This means that the encoder needs to have a decoder within it so it can see what is decoded and understand the differences. Virginie explains there are three types of prediction. Intra prediction uses the current frame to predict the content of a block, inter prediction which uses other frames to predict video data and also a hybrid mode which uses both, new to VVC. There are now 93 directional intra prediction angles and the introduction of matrix-based intra prediction. This is an example of the beginning of the move to AI for codecs, a move which is seen as inevitable by The Broadcast Knowledge as we see more examples of how traditional mathematical algorithms are improved upon by AI, Machine Learning and/or Deep Learning. A good example of this is super-resolution. In this case, Virginie says that they used machine learning to generate some matrices which are used for the transform meaning that there’s no neural network within the codec, but that the matrices were created based on real-world data. It seems clear that as processing power increases, a neural network will be implemented in future codecs (whether MPEG or otherwise).

For screen encoding, we see that intra block copying (IBC) is still present from HEVC, explained here from 17:30 IBC allows part of a frame to be copied to another which is a great technique for computer-generated content. Whilst this was in HEVC it was not in the basic package of tools in HEVC meaning it was much less accessible as support in the decoders was often lacking. Two new tools are block differential pulse code modulation & transform skip with adapted residual coding each discussed, along with IBC in this free paper.

Virginie moves on to Coding performance explaining that the JVET reference software called VTM has been used to compare against HEVC’s HM reference and has shown, using PSNR, an average 41% improvement on luminance with screen content at 48%. Fraunhofer HHI’s VVenc software has been shown to be 49%.

Along with the ability to be applied to screen content and 360-degree video, the versatility in the title of the codec also refers to the different layers and tiers it has which stretch from 4:2:0 10 bit video all the way up to 4:4:4 video including spatial scalability. The main tier is intended for delivery applications and the high for contribution applications with framerates up to 960 fps, up from 300 in HEVC. There are levels defined all the way up to 8K. Virginie spends some time explaining NAL units which are in common with HEVC and AVC, explained here from slide 22 along with the VCL (Video Coding Layer) which Virginie also covers.

Random access has long been essential for linear broadcast video but now also streaming video. This is done with IDR (Instantaneous Decoding Refresh), CRA (Clean Random Access) and GDR (Gradual Decoding Refresh). IDR is well known already, but GDR is a new addition which seeks to smooth out the bitrate. With a traditional IBBPBBPBBI GOP structure, there will be a periodic peak in bitrate because the I frames are much larger than the B and, indeed, P frames. The idea with GDR is to have the I frame gradually transmitted over a number of frames spreading out the peak. This disadvantage is you need to wait longer until you have your full I frame available.

Virginie introduces subpictures which are a major development in VVC allowing separately encoded pictures within the same stream. Effectively creating a multiplexed stream, sections of the picture can be swapped out for other videos. For instance, if you wanted a picture in picture, you could swap the thumbnail video stream before the decoder meaning you only need one decoder for the whole picture. To do the same without VVC, you would need two decoders. Subpictures have found use in 360 video allowing reduced bitrate where only the part which is being watched is shown in high quality. By manipulating the bitstream at the sender end.

Before finishing by explaining that VVC can be carried by both MPEG’s ISO BMFF and MPEG2 Transport Streams, Virginie covers Reference Picture Resampling, also covered in this video from Seattle Video Tech allows reference frames of one resolution to be an I frame for another resolution stream. This has applications in adaptive streaming and spatial scalability. Virginie also covers the enhanced timing available with HRD

Watch now!
Video is free to watch
Speaker

Virginie Drugeon Virginie Drugeon
Senior Engineer Digital TV Standardisation,
Panasonic

Video: Embracing a Native IP Media Distribution Format as a Next Generation Broadcast Solution

Posted on by Russell Trafford-Jones

The shift to IP hasn’t gone unnoticed in the transmission part of the broadcast chain. ATSC 3.0 is a shift to 100% IP delivery over RF and seamless integration with internet-delivered programming. So, too, has DVB seen this change which they describe as a move from thinking of distribution to thinking about a service. Merging broadcast RF delivery with internet delivery is an aspiration not only to merge broadband and broadcast, but to unify what we now see as fragmentation of services.

DVB’s Emily Dubs highlights some of the standards which form this native-IP strategy such as DVB-MABR, DVB-TA and DVB-I all of which have been featured here on The Broadcast Knowledge.

Thomas Wrede from the DVB CM-S group takes a look at the use cases motivating this work. Connected devices are now ubiquitous like smart TVs, phones and tablets and the stats show that viewers want to access both linear and non-linear content on all their devices. Indeed, Thomas outlines some statistics showing that consumption of linear TV remains high. So given this, Richard Lhermitte from ENENSYS explains that they’re interested in solving a problem for customers with low bandwidth broadband. The work in the CM-S group involves satellite rather than terrestrial RF transmission so the solutions use a gateway device. Looking at business-to-business applications Richard envisions feeding 5G cell towers by delivering video services over satellite, feeding wifi routers in schools, public buildings, malls etc. and delivery to aeroplanes, boats and similar.

 

 

Delivering to consumers, we see is envisaged as bringing the satellite and/or terrestrial RF into a demodulator then a gateway device which would deliver into a wifi router where it’s delivered alongside broadband. The internet feed can also be used as a return path for DRM and other signalling. Using this method, only one headend is needed for both the OTT and broadcast feeds a change which AWS noted recently brings around cost savings and was what drove several features in AWS MediaConnect. Another similar use case is to do the same but without the return path. This is more difficult but is part of the work being undertaken. Richard also highlights the ability to deliver parts of a programme, say extra languages, via broadband but using broadcast to deliver the main parts that everyone will want access to. Similarly, using MPEG LCEVC or similar scalable codecs, the HD video can be delivered over the air with the UHD enhancement layer delivered over broadband.

Lastly, Jean-Claude Sachot from Broadpeak briefly talks about the work done with Malaysian pay-TV broadcaster Astro SINI who have been working to extend their proposition with successful tests delivering via satellite into wifi hotspots in public places. This works well as not only does it improve Astro’s proposition to the viewers, but it increases their time spent in the locations where they can watch for free over Wi-Fi rather than having to use their mobile minutes. This is a benefit to the businesses working with Astro to provide the service and for them, the added infrastructure is very low cost as this plugs into their existing Wi-Fi infrastructure.

The video concludes with a Q&A which is summarised here and the main slides can be downloaded here

Watch now!
Speakers

Jean-Claude Sachot Jean-Claude Sachot
Business Development Director,
Broadpeak
Richard Lhermitte Richard Lhermitte
CTO, VP Solutions, New Market development
ENENSYS
Thomas Wrede Thomas Wrede
DVB CM-S Chair
SES
Emily Dubs Moderator: Emily Dubs
Head of Technology,
DVB Project

Video: How to Deploy an IP-Based Infrastructure

Posted on by Russell Trafford-Jones

An industry-wide move to any new technology takes time and there is a steady flow of people new to the technology. This video is a launchpad for anyone just coming into IP infrastructures whether because their company is starting or completing an IP project or because people are starting to ask the question “Should we go IP too?”.

Keycode Media’s Steve Dupaix starts with an overview of how SMPTE’s suite of standards called ST 2110 differs from other IP-based video and audio technologies such as NDI, SRT, RIST and Dante. The key takeaways are that NDI provides compressed video with a low delay of around 100ms with a suite of free tools to help you get started. SRT and RIST are similar technologies that are usually used to get AVC or HEVC video from A to B getting around packet loss, something that NDI and ST 2110 don’t protect for without FEC. This is because SRT and RIST are aimed at moving data over lossy networks like the internet. Find out more about SRT in this SMPTE video. For more on NDI, this video from SMPTE and VizRT gives the detail.

 

 

ST 2110’s purpose is to get high quality, usually lossless, video and audio around a local area network originally being envisaged as a way of displacing baseband SDI and was specced to work flawlessly in live production such as a studio. It brings with it some advantages such as separating the essences i.e. video, audio, timing and ancillary data are separate streams. It also brings the promise of higher density for routing operations, lower-cost infrastructure since the routers and switches are standard IT products and increased flexibility due to the much-reduced need to move/add cables.

Robert Erickson from Grass Valley explains that they have worked hard to move all of their product lines to ‘native IP’ as they believe all workflows will move IP whether on-premise or in the cloud. The next step, he sees is enabling more workflows that move video in and out of the cloud and for that, they need to move to JPEG XS which can be carried in ST 2110-20. Thomas Edwards from AWS adds their perspective agreeing that customers are increasingly using JPEG XS for this purpose but within the cloud, they expect the new CDI which is a specification for moving high-bandwidth traffic like 2110-20 streams of uncompressed video from point to point within the cloud.

John Mailhot from Imagine Communications is also the chair of the VSF activity group for ground-cloud-cloud-ground. This aims to harmonise the ways in which vendors provide movement of media, whatever bandwidth, into and out of the cloud as well as from point to point within. From the Imagine side, he says that ST 2110 is now embedded in all products but the key is to choose the most appropriate transport. In the cloud, CDI is often the most appropriate transport within AWS and he agrees that JPEG XS is the most appropriate for cloud<->ground operations.

The panel takes a moment to look at the way that the pandemic has impacted the use of video over IP. As we heard earlier this year, the New York Times had been waiting before their move to IP and the pandemic forced them to look at the market earlier than planned. When they looked, they found the products which they needed and moved to a full IP workflow. So this has been the theme and if anything has driven, and will continue to drive, innovation. The immediate need provided the motivation to consider new workflows and now that the workflow is IP, it’s quicker, cheaper and easier to test new variation. Thomas Edwards points out that many of the current workflows are heavily reliant on AVC or HEVC despite the desire to use JPEG XS for the broadcast content. For people at home, JPEG XS bandwidths aren’t practical but RIST with AVC works fine for most applications.

Interoperability between vendors has long been the focus of the industry for ST 2110 and, in John’s option, is now pretty reliable for inter-vendor essence exchanges. Recently the focus has been on doing the same with NMOS which both he and Robert report is working well from recent, multi-vendor projects they have been involved in. John’s interest is working out ways that the cloud and ground can find out about each other which isn’t a use case yet covered in AMWA’s NMOS IS-04.

The video ends with a Q&A covering the following:

  • Where to start in your transition to IP
  • What to look for in an ST 2110-capable switch
  • Multi-Level routing support
  • Using multicast in AWS
  • Whether IT equipment lifecycles conflict with Broadcast refresh cycles

    • Watch now!
    Speakers

    John Mailhot John Mailhot
    CTO & Director of Product Management, Infrastructure & Networking,
    Imagine Communications
    Ciro Noronha Ciro Noronha
    Executive Vice-President of Engineering,
    Cobalt Digital
    Thomas Edwards Thomas Edwards
    Principal Solutions Architect & Evangelist,
    Amazon Web Services
    Robert Erickson Robert Erickson
    Strategic Account Manager Sports and Venues,
    Grass Valley
    Steve Dupaix Steve Dupaix
    Senior Account Executive,
    Key Code Media

    Video: Disney Streaming Fireside Chat

    Posted on by Russell Trafford-Jones

    Disney Streaming encompasses ESPN+, Hulu, Star as well as the much discussed Disney+. The latter was launched just a month before the initial detection of Covid-19 but all the services have been seeing continual roll-outs globally ever since. And it’s because of this global perspective, and one of perpetually dealing with growth, that today’s conversation with Disney as part of Streaming Media Connect is so interesting.

    Eric Klein, Pankaj Chaudharim and Robert Colantuoni join moderators Tim Siglin and Eric Schumacher-Rasmussen to explain their approach to growing their streaming business. We’ve heard from Disney Streaming Services (DSS) before on The Broadcast Knowledge so we know that they have a strong interest in pushing forward Open Caching but also building their own solutions. Today’s video only reinforces that view.

     

     

    When asked about DSS’s route to market, Eric points out that they have a ‘mesh’ that takes in to account that a single route to market for every location isn’t going to work. There are different limitations and constraints for every geography so whether it’s using multi-tenanting or Open Caching, for example, the mix needs to be tuned for each launch. Eric says that launching into a territory is a difficult time as they never know where exactly their customers will be and how much bandwidth will be needed.

    Rob explains that Disney has a ‘Decision Science’ department which has helped them build models of new territories partly based on new data and partly based on finding similar locales already served by Disney Streaming Service. Talking about Open Cache, Rob says this is a big part of the mesh they run but sending out instructions is only one part of the challenge. The other is getting back data quickly enough and in the right format so that DSS can quickly understand it and use it for business decisions. You need to have a control plane that can handle all of these data sources and track performance relative to the options in those regions.

    Pankaj adds that they constantly work with providers to ensure a base level of functionality like TLS 1.3, as an example, as well as helping them implement APIs from the Streaming Video Alliance to understand capacity. It’s critical for them to be able to understand the CDN’s capabilities as they wax and wain thanks to ‘internet weather’ as Rob puts it.

    Eric talks further about how they work with their partners helping them to deal with the traffic which is coming their way and encouraging them to be part of the CTA or SVA so they can keep on top of the latest developments and also have a voice in what happens. Eric then talks about environmental work and discusses how PVoD, i.e. the latest Mulan film changes the way they deliver.

    Watch now!
    Speakers

    Eric Klein Eric Klein
    Director of Media Distribution & Technology, CDN
    Disney Streaming
    Robert Colantuoni Robert Colantuoni
    CDN Performance Architect,
    Disney Streaming Services
    Pankaj Chaudhari Pankaj Chaudhari
    Architect – Video Delivery,
    Hulu
    Eric Schumacher-Rasmussen Moderator: Eric Schumacher-Rasmussen
    Editor in Chief,
    Streaming Media Magazine
    Tim Siglin Moderator: Tim Siglin
    Contributing Editor,
    Streaming Media Magazine