FPGAs are flexible, reprogrammable chips which can do certain tasks faster than CPUs, for example, video encoding and other data-intensive tasks. Once the domain of expensive hardware broadcast appliances, FPGAs are now available in the cloud allowing for cheaper, more flexible encoding.
In fact, according to NGCodec founder Oliver Gunasekara, video transcoding makes up a large percentage of cloud work loads and this increasing year on year. The demand for more video and the demand for more efficiently-compressed video both push up the encoding requirements. HEVC and AV1 both need much more encoding power than AVC, but the reduced bitrate can be worth it as long as the transcoding is quick enough and the right cost.
Oliver looks at the likely future adoption of new codecs is likely to playout which will directly feed into the quality of experience: start-up time, visual quality, buffering are all helped by reduced bitrate requirements.
It’s worth looking at the differences and benefits of CPUs, FPGAs and ASICs. The talk examines the CPU-time needed to encode HEVC showing the difficulty in getting real-time frame rates and the downsides of software encoding. It may not be a surprise that NGCodec was acquired by FPGA manufacturer Xilinx earlier in 2019. Oliver shows us the roadmap, as of June 2019, of the codecs, VQ iterations and encoding densities planned.
The talk finishes with a variety of questions like the applicability of Machine Learning on encoding such as scene detection and upscaling algorithms, the applicability of C++ to Verilog conversion, the need for a CPU for supporting tasks.
Twitch is an ambassador for new codecs and puts its money where its mouth is; it is one of the few live streaming platforms which streams with VP9 – and not only at, with cloud FPGA acceleration thanks to Xylinx’s acquisition of NGCODEC.
As such, they have a strong position on AV1. With such a tech savvy crowd, they stream most of their videos at the highest bitrate (circa 6mbps). With millions of concurrent videos, they are highly motivated to reduce bandwidth where they can and finding new codecs is one way to do that.
Principal Research Engineer, Yueshi discusses Twitch’s stance on AV1 and the work they are doing to contribute in order to get the best product at the end of the process which will not only help them, but the worldwide community. He starts by giving an overview of Twitch which, while many of us are familiar with the site, the scale and needs of the site may be new information and drive the understanding of the rest of the talk.
Reduction in bitrate is a strong motivator, but also the fact that supporting many codecs is a burden. AV1 promises a possibility of reducing the number of supported codecs/formats. Their active contribution in AV1 is also determined by the ‘hand wave’ latency; a simple method of determining the approximate latency of a link which is naturally very important to a live streaming platform. This led to Twitch submitting a proposal for SWITCH_FRAME which is a technique, accepted in AV1, which allows more frequent changes by the player between the different quality/bitrate streams available. This results in a better experience for the user and also reduced bitrate/buffers.
YueShi then looks at the projected AV1 deployment roadmap and discusses when GPU/hardware support will be available. The legal aspect of AV1 – which promises to be a free-to-use codec is also discussed with the news that a patent pool has formed around AV1.
AV1 is famous for its promise to deliver better compression than HEVC but also for it being far from real-time. This talk has a demonstration of the world’s first real-time AV1 video call showing that speed improvement are on the way and, indeed, some have arrived.
Encoding is split into ‘tools’ so where you might hear of ‘h.264’ or ‘MPEG 2’, these are names for a whole set of different ways of looking at – and squeezing down – a picture. They also encompass the rules of how they should act together to form a cohesive encoding mechanism. (To an extent, such codecs tend to define only how the decode should happen, leaving encoding open to innovation.) AV1 contains many tools, many of which are complex and so require a lot of time even from today’s fast computers.
Cisco’s Thomas Davies, who created the BBC’s Dirac codec which is now standardised under SMPTE’s VC-2 standard, points out that whilst these tools are complex, AV1 also has a lot of them and this diversity of choice is actually a benefit for speed and in particular for the speed of software codecs.
After demonstrating the latency and bandwidth benefits of their live, bi-directional, AV1 implementation against AVC, Thomas looks at the deployment possibilities and of AV1. The talk finishes with a summary of what AV1 brings in benefits to sum up why this new effort, with the Alliance of Open Media, is worth it.
Cisco Media Engineering, UK
Subscribe to get daily updates
Views and opinions expressed on this website are those of the author(s) and do not necessarily reflect those of SMPTE or SMPTE Members.
This website is presented for informational purposes only. Any reference to specific companies, products or services does not represent promotion, recommendation, or endorsement by SMPTE