JPEG XS

Video: Live Production Forecast: Cloudy for the Foreseeable Future

Posted on by Russell Trafford-Jones

Our ability to work remotely during the pandemic is thanks to the hard work of many people who have developed the technologies which have made it possible. Even before the pandemic struck, this work was still on-going and gaining momentum to overcome more challenges and more hurdles of working in IP both within the broadcast facility and in the cloud.

SMPTE’s Paul Briscoe moderates the discussion surrounding these on-going efforts to make the cloud a better place for broadcasters in this series of presentation from the SMPTE Toronto section. First in the order is Peter Wharton from TAG V.S. talking about ways to innovate workflows to better suit the cloud.

Peter first outlines the challenges of live cloud production, namely keeping latency low, signal quality high and managing the high bandwidths needed at the same time as keeping a handle on the costs. There is an increasing number of cloud-native solutions but how many are truly innovating? Don’t just move workflows into the cloud, advocates Peter, rather take this opportunity to embrace the cloud.

Working with the cloud will be built on new transport interfaces like RIST and SRT using a modular and open architecture. Scalability is the name of the game for ‘the cloud’ but the real trick is in building your workflows and technology so that you can scale during a live event.

Source: TAG V.S.

There are still obstacles to be overcome. Bandwidth for uncompressed video is one, with typical signals up to 3Gbps uncompressed which then drives very high data transfer costs. The lack of PTP in the cloud makes ST 2110 workflows difficult, similarly the lack of multicast.

Tackling bandwidth, Peter looks at the low-latency ways to compress video such as NDI, NDI|HX, JPEG XS and Amazon’s lossless CDI. Peter talks us through some of the considerations in choosing the right codec for the task in hand.

Finishing his talk, Peter asks if this isn’t time for a radical change. Why not rethink the entire process and embrace latency? Peter gives an example of a colour grading workflow which has been able to switch from on-prem colour grading on very high-spec computers to running this same, incredibly intensive process in the cloud. The company’s able to spin up thousands of CPUs in the cloud and use spot pricing to create temporary, low cost, extremely powerful computers. This has brought waiting times down for jobs to be processed significantly and has reduced the cost of processing an order of magnitude.

Lastly Peter looks further to the future examining how saturating the stadium with cameras could change the way we operate cameras. With 360-degree coverage of the stadium, the position of the camera can be changed virtually by AI allowing camera operators to be remote from the stadium. There is already work to develop this from Canon and Intel. Whilst this may not be able to replace all camera operators, sports is the home of bleeding-edge technology. How long can it resist the technology to create any camera angle?

Source: intoPIX

Jean-Baptiste Lorent is next from intoPIX to explain what JPEG XS is. A new, ultra-low-latency, codec it meets the challenges of the industry’s move to IP, its increasing desire to move data rather than people and the continuing trend of COTS servers and cloud infrastructure to be part of the real-time production chain.

As Peter covered, uncompressed data rates are very high. The Tokyo Olympics will be filmed in 8K which racks up close to 80Gbps for 120fps footage. So with JPEG XS standing for Xtra Small and Xtra Speed, it’s no surprise that this new ISO standard is being leant on to help.

Tested as visually lossless to 7 or more encode generations and with latency only a few lines of video, JPEG XS works well in multi-stage live workflows. Jean-Baptiste explains that it’s low complexity and can work well on FPGAs and on CPUs.

JPEG XS can support up to 16-bit values, any chroma and any colour space. It’s been standardised to be carried in MPEG TSes, in SMPTE ST 2110 as 2110-22, over RTP (pending) within HEIF file containers and more. Worst case bitrates are 200Mbps for 1080i, 390Mbps for 1080p60 and 1.4Gbps for 2160p60.

Evolution of Standards-Based IP Workflows Ground-To-Cloud

Last in the presentations is John Mailhot from Imagine Communications and also co-chair of an activity group at the VSF working on standardising interfaces for passing media place to place. Within the data plane, it would be better to avoid vendors repeatedly writing similar drivers. Between ground and cloud, how do we standardise video arriving and the data you need around that. Similarly standardising new technologies like Amazon’s CDI is important.

John outlines the aim of having an interoperability point within the cloud above the low-level data transfer, closer to 7 than to 1 in the OSI model. This work is being done within AIMS, VSF, SMPTE and other organisations based on existing technologies.

Q&A
The video finishes with a Q&A and includes comments from AWS’s Evan Statton whose talk on CDI that evening is not part of this video. The questions cover comparing NDI with JPEG XS, how CDI uses networking to achieve high bandwidths and high reliability, the balance between minimising network and minimising CPU depending on workflow, the increasingly agile nature of broadcast infrastructure, the need for PTP in the cloud plus the pros and cons of standards versus specifications.

Watch now!
Speakers

Peter Wharton Peter Wharton
Director Corporate Strategy, TAG V.S.
President, Happy Robotz
Vice President of Membership, SMPTE
Jean-Baptiste Lorent Jean-Baptiste Lorent
Director Marketing & Sales,
intoPIX
John Mailhot John Mailhot
Co-Chair Cloud-Gounrd-Cloud-Ground Activity Group, VSF
Directory & NMOS Steering Member, AMWA
Systems Architect for IP Convergence, Imagine Communcations
Paul Briscoe Moderator: Paul Briscoe
Canadian Regional Governor, SMPTE
Consultant, Televisionary Consulting
Evan Statton Evan Statton
Principal Architect, Media & Entertainment
Amazon Web Services

Video: JPEG XS Interoperability Activity Group Update

Posted on by Russell Trafford-Jones


JPEG XS is a low-latency, light-compression codec often called a ‘mezzanine’ codec. Encoding within milliseconds, JPEG XS can compress full-bandwidth signals by 4x or more allowing scope for several generations of compression without significant degradation. The low-latency and resilience to de-generation make it ideal for enabling remote production.

John Dale from Media Links joins us to look at what’s being done within the Video Services Forum (VSF) to ensure interoperability. As a new standard, JPEG XS is yet to be or is still being implemented in many companies’ products. Therefore this is the perfect time to be looking at how to standardise interconnects,

Running JPEG XS over MPEG TS is one approach which is being written up in ‘VSF TR-07’ (Technical Reference 7) which will be imminently completed. It defines capabilities for 2K, 4K and 8K video with and without HDR. They have split the video formats into capability sets meaning that a vendor can comply with the specification by stating which subset(s) it can cope with. All formats up to 1080p60 are under capability set ‘A’ with ‘B’ covering UHD resolutions. After this work, they will look at JPEG XS over ST 2110-22 instead of MPEG TS. This is yet to start and will share much of the work from previous work.

Watch now!
Speaker

John Dale John Dale
Company Director and CMO,
Media Links.

Video: High-Throughput JPEG 2000 (HTJ2K) for Content Workflows

Posted on by Russell Trafford-Jones

Published last year, high-throughput JPEG 2000 (HTJ2K) is an update to the J2K we know well in the broadcast industry making it much faster. Whilst JPEG 2000 has found a home in low-latency broadcast contribution, it’s also part of the archive exchange format (AXF) because, unlike most codecs, JPEG 2000 has a mathematically lossless mode. HTJ2K takes JPEG 2000 and replaces some of the compression with a much faster algorithm allowing for much faster decoding of well 10 to 28 times faster in many circumstances.

The codec market seems waking up to the fact that multiple types of codec are needed to support the thousands of use cases that we have in the Media and Entertainment and beyond. It’s generally well known that codecs live in a world where they are optimising bitrate at the expense of latency and quality. But the advent of MPEG 5 Part 2, also known as LCEVC show that there is value in optimising to reduce complexity of encoding. In some ways, this is similar to saying reduce the latency, but in the LCEVC example, the aim is to allow low-power or low-complexity equipment to deal with HD or UHD video where otherwise that might not have been possible. With HTJ2K we have a similar situation where it’s worth getting 10x more throughput when managing and processing your archive at the expense of 5% more bitrate.

This talk from the EBU’s Network Technology Seminar hears from Pierre-Anthony Lemieux and Michael Smith who explain the need for this codec and the advantages. One important fact is that the encoding itself hasn’t been changed, just some of the maths around it. This means that you can take previously encoded files and process them into HTJ2K without changing any of the video data. This allows lossy J2K files to be converted without any degradation due to re-encoding and minimises conversion time for lossless files. Another motivator for this codec is cloud workflows where speed of compression is important to reduce costs. Michael Smith also explores the similarities and differences of High-Throughput J2K with JPEG XS

Watch now!
Speakers

Pierre-Anthony Lemieux Pierre-Anthony Lemieux
Sandflow Consulting
Michael Smith
Wavelet Consulting

Video: Case Study on a Large Scale Distributed ST 2110 Deployment

Posted on by Russell Trafford-Jones

We’re “past the early-adopter stage” of SMPTE 2110, notes Andy Rayner from Nevion as he introduces this case study of a multi-national broadcaster who’s created a 2110-based live production network spanning ten countries.

This isn’t the first IP project that Nevion have worked on, but it’s doubtless the biggest to date. And it’s in the context of these projects that Andy says he’s seen the maturing of the IP market in terms of how broadcasters want to use it and, to an extent, the solutions on the market.

Fully engaging with the benefits of IP drives the demand for scale as people are freer to define a workflow that works best for the business without the constraints of staying within one facility. Part of the point of this whole project is to centralise all the equipment in two, shared, facilities with everyone working remotely. This isn’t remote production of an individual show, this is remote production of whole buildings.

SMPTE ST-2110, famously, sends all essences separately so where an 1024×1024 SDI router might have carried 70% of the media between two locations, we’re now seeing tens of thousands of streams. In fact, the project as a whole is managing in the order of 100,000 connections.

With so many connections, many of which are linked, manual management isn’t practical. The only sensible way to manage them is through an abstraction layer. For instance, if you abstract the IP connections from the control, you can still have a panel for an engineer or operator which says ‘Playout Server O/P 3’ which allow you to route it with a button that says ‘Prod Mon 2’. Behind the scenes, that may have to make 18 connections across 5 separate switches.

This orchestration is possible using SDN – Software Defined Networking – where router decisions are actually taken away from the routers/switches. The problem is that if a switch has to decide how to send some traffic, all it can do is look at its small part of the network and do its best. SDN allows you to have a controller, or orchestrator, which understands the network as a whole and can make much more efficient decisions. For instance, it can make absolutely sure that ST 2022-7 traffic is routed separately by diverse paths. It can do bandwidth calculations to stop bandwidths from being oversubscribed.

Whilst the network is, indeed, based on SMPTE ST 2110, one of the key enablers is JPEG XS for international links. JPEG XS provides a similar compression level to JPEG 2000 but with much less latency. The encode itself requires less than 1ms of latency, unlike JPEG 2000’s 60ms. Whilst 60ms may seem small, when a video needs to move 4 or even 10 times as part of a production workflow, it soon adds up to a latency that humans can’t work with. JPEG XS promises to allow such international production to feel responsive and natural. Making this possible was the extension of SMPTE ST 2110, for the first time, to allow carriage of compressed video in ST 2110-22.

Andy finishes his overview of this uniquely large case study talking about conversion between types of audio, operating SDN with IGMP multicast islands, and NMOS Control. In fact, it’s NMOS which the answer to the final question asking what the biggest challenge is in putting this type of project together. Clearly, in a project of this magnitude, there are challenges around every corner, but problems due to quantity can be measured and managed. Andy points to NMOS adoption with manufacturers still needing to be pushed higher whilst he lays down the challenge to AMWA to develop NMOS further so that it’s extended to describe more aspects of the equipment – to date, there are not enough data points.

Watch now!
Speakers

Andy Rayner Andy Rayner
Chief Technologist,
Nevion