Hardware encoding is more pervasive with Intel’s Quick Sync embedding CUDA GPUs inside GPUs plus NVIDIA GPUs have MPEG NVENC encoding support so how does it compare with software encoding? For HEVC, can Xilinx’s FPGA solution be a boost in terms of quality or cost compared to software encoding?
Jan Ozer has stepped up to the plate to put this all to the test analysing how many real-time encodes are possible on various cloud computing instances, the cost implications and the quality of the output. Jan’s analytical and systematic approach brings us data rather than anecdotes giving confidence in the outcomes and the ability to test it for yourself.
Over and above these elements, Jan also looks at the bit rate stability of the encodes which can be important for systems which are sensitive to variations such services running at scale. We see that the hardware AVC solutions perform better than x264.
Jan takes us through the way he set up these tests whilst sharing the relevant ffmpeg commands. Finally he shares BD plots and example images which exemplify the differences between the codecs.
Video is so pervasive in our world that we need to move past thinking of codecs and compression being about reducing bitrate. That will always be a major consideration, but speed of compression and the computation needed can also be deal breakers. Millions of embedded devices need to encode video which don’t have the grunt available to the live AV1-encoding clusters in the cloud. Further more, the structure of the final data itself can be important for later processing and decoding. So we can see how use-cases can arise out needs of various industries, far beyond broadcast, which mean that codecs need to do more than make files small.
This year LCEVC from MPEG will be standardised. Called Low Complexity Enhancement Video Coding, this codec provides compression both where computing is constrained and where it is plentiful. Guido Meardi, CEO of V-Nova, talks us through what LCEVC is starting with a chart showing how computation has increased vastly as compression has improved. It’s this trend that this codec intends to put an end to by adding, Guido explains, an enhancement layer over some lower-resolution video. By encoding a lower-resolution, computational processing is minimised. When displayed, an enhancement layer allows this low resolution video to be sharpened again to bring it back to the original.
After demonstrating the business benefits, we see the block diagram of the encoder and decoder which helps visualise how this enhancement might be calculated and work. Guido then shows us what the enhancement layer looks like – a fairy flat image with lots of thin edges on it but, importantly, it also captures a lot of almost random detail which can’t be guessed by upsamplers. This, of course, is the point. If it were possible to upscale the low-resolution video and guess/infer all the data, then we would always do that. Rather, downscaling and upscaling is a lossy process. Here, that loss is worth it because of the computational gains and because the enhancement layer will put back much of what was once lost.
In order to demonstrate LCEVC’s ability, Guido shows graphs comparing LCEVC at UHD for x264 showing improvements of between 20 and 45% and image examples of artefacts which are avoided using LCEVC. We then see that when applied to AVC, HEVC and VVC it speeds up encodes at least two fold. Guido finishes this presentation showing how you can test out the encoder and decoder yourself.
The last segment of this video, Tarek Amara from Twitch sits down to talk with Guido about the codec and the background behind it. Their talk covers V-Nova’s approach to open source, licensing, LCEVC’s gradual improvements as it went through the proving process as part of MPEG standardisation plus questions from the floor.
CMAF is often seen as the best hope for streaming to match the latency of broadcast. Fully standards based, many see this as the best route over Apple’s LL-HLS. Another benefit of it over LL-HLS is that it’s already a completed standard with growing support.
This talk from Tomas Bacik starts by explaining CMAF to us. Standing for the Common Media Application Format, it’s based on the standardised ISOBMFF container format and whilst CMAF isn’t by default low-latency, it is flexible enough to deliver just that. However, as Tomas from CDN77 points out, there are other major benefits in terms of its use of the Common Encryption format, reduces storage fees and more.
MPEG DASH is a commonly found streaming format based on ISO BMFF. It has always had the benefit of supporting other codecs such as HEVC and AV1 over HLS which is an AVC-only specification. CMAF is an extension of MPEG DASH which goes one step further in that it can deal with both HLS-style manifest files (.hls) as well as MPEG DASH format (.mpd) inheriting, of course, the multi-codec ability of DASH itself.
Next is central theme of the talk, looking at VoD workflows showing how CMAF fits in and, indeed, changes workflows for the better. CMAF directly impacts packaging, storage and CDN which is where we focus now. Given that some devices can play HLS and some can play DASH, if you try to serve both, you will double your requirements of packaging, storage etc. Dynamic packaging allows for immediately repackaging your chunks into either HLS or DASH as needed. Whilst this reduces the storage requirements, it increases processing and also increases the time to first byte. As you might expect, using CMAF throughout, Tomas explains in this talk, allows you to package once and store once which solves these problems.
Tomas continues by explaining the DRM abilities of CMAF including AES-CBC and finishes by taking questions from the audience.
We hear about so many new and improved cloud products and solutions to improve production that, once in a while, you really just need to step back and hear how people have put them together. This session is just that, a look at the whole post production workflow for FOX Sports’ production of the Women’s World Cup.
This panel from the Live Streaming Summit at Streaming Media West is led by FOX Sports’ Director of Post Production, Brandon Potter as he talks through the event with three of his key vendors, IBM Aspera, Telestream and Levels Beyond.
Brandon starts by explaining that this production stood on the back of the work they did with the Men’s World Cup in Russia, both having SDI delivery of media in PAL at the IBC. For this event, all the edit crew was in LA which created problems with some fixed frame-rate products still in use in the US facility.
Data transfer, naturally is the underpinning of any event like this with a total of a petabyte of data being created. Network connectivity for international events is always tricky. With so many miles of cable whether on land or under the sea, there is a very high chance of the fibre being cut. At the very least, the data can be switched to take a different path an in that moment, there will be data loss. All of this means that you can’t assume the type of data loss, it could be seconds, minutes or hours. On top of creating, and affording, redundant data circuits, the time needed for transfer of all the data needs to be considered and managed.
Ensuring complete transfer of files in a timely fashion drove the production to auto archive of all content in real time into Amazon S3 in order to avoid long post-match ingest times of multiple hours, “every bit of high-res content was uploaded.” stated Michael Flathers, CTO of IBM Aspera.
Dave Norman, from Telestream explains how the live workflows stayed on-prem with the high-performance media and encoders and then, “as the match ended, we would then transition…into AWS”. In the cloud, the HLS proxies would then being rendered into a single mp4 proxy editing files.
David Gonzales explains the benefits of the full API integrations they chose to build their multi-vendor solution around, rather than simple watch-folders. For all platforms to know where the errors were was very valuable and was particularly useful for the remote users to know in detail where their files were. This reduces the number of times they would need to ask someone for help and meant that when they did need to ask, they had a good amount of detail to specify what the problem was.
The talk comes to a close with a broad analysis of the different ways that files were moved and cached in order to optimise the workflow. There were a mix of TCP-style workflows and Aspera’s UDP-based transfer technology. Worth noting, also, that HLS manifests needed to be carefully created to only reference chunks that had been transferred, rather than simply any that had been created. Use of live creation of clips from growing files was also an important tool, the in- and out-points being created by viewing a low-latency proxy stream then the final file being clipped from the growing file in France and delivered within minutes to LA.
Overall, this case study gives a good feel for the problems and good practices which go hand in hand with multi-day events with international connectivity and shows that large-scale productions can successfully, and quickly, provide full access to all media to their production teams to maximise the material available for creative uses.
Brandon Potter
Director of Post Production,
FOX Sports
Dave Norman
Principal Sales Engineer,
Telestream
Daniel Gonzales
Senior Solutions Architect,
Levels Beyond
Subscribe to get daily updates
Views and opinions expressed on this website are those of the author(s) and do not necessarily reflect those of SMPTE or SMPTE Members.
This website is presented for informational purposes only. Any reference to specific companies, products or services does not represent promotion, recommendation, or endorsement by SMPTE
