Video: Live Production Forecast: Cloudy for the Foreseeable Future

Our ability to work remotely during the pandemic is thanks to the hard work of many people who have developed the technologies which have made it possible. Even before the pandemic struck, this work was still on-going and gaining momentum to overcome more challenges and more hurdles of working in IP both within the broadcast facility and in the cloud.

SMPTE’s Paul Briscoe moderates the discussion surrounding these on-going efforts to make the cloud a better place for broadcasters in this series of presentation from the SMPTE Toronto section. First in the order is Peter Wharton from TAG V.S. talking about ways to innovate workflows to better suit the cloud.

Peter first outlines the challenges of live cloud production, namely keeping latency low, signal quality high and managing the high bandwidths needed at the same time as keeping a handle on the costs. There is an increasing number of cloud-native solutions but how many are truly innovating? Don’t just move workflows into the cloud, advocates Peter, rather take this opportunity to embrace the cloud.

Working with the cloud will be built on new transport interfaces like RIST and SRT using a modular and open architecture. Scalability is the name of the game for ‘the cloud’ but the real trick is in building your workflows and technology so that you can scale during a live event.

Source: TAG V.S.

There are still obstacles to be overcome. Bandwidth for uncompressed video is one, with typical signals up to 3Gbps uncompressed which then drives very high data transfer costs. The lack of PTP in the cloud makes ST 2110 workflows difficult, similarly the lack of multicast.

Tackling bandwidth, Peter looks at the low-latency ways to compress video such as NDI, NDI|HX, JPEG XS and Amazon’s lossless CDI. Peter talks us through some of the considerations in choosing the right codec for the task in hand.

Finishing his talk, Peter asks if this isn’t time for a radical change. Why not rethink the entire process and embrace latency? Peter gives an example of a colour grading workflow which has been able to switch from on-prem colour grading on very high-spec computers to running this same, incredibly intensive process in the cloud. The company’s able to spin up thousands of CPUs in the cloud and use spot pricing to create temporary, low cost, extremely powerful computers. This has brought waiting times down for jobs to be processed significantly and has reduced the cost of processing an order of magnitude.

Lastly Peter looks further to the future examining how saturating the stadium with cameras could change the way we operate cameras. With 360-degree coverage of the stadium, the position of the camera can be changed virtually by AI allowing camera operators to be remote from the stadium. There is already work to develop this from Canon and Intel. Whilst this may not be able to replace all camera operators, sports is the home of bleeding-edge technology. How long can it resist the technology to create any camera angle?

Source: intoPIX

Jean-Baptiste Lorent is next from intoPIX to explain what JPEG XS is. A new, ultra-low-latency, codec it meets the challenges of the industry’s move to IP, its increasing desire to move data rather than people and the continuing trend of COTS servers and cloud infrastructure to be part of the real-time production chain.

As Peter covered, uncompressed data rates are very high. The Tokyo Olympics will be filmed in 8K which racks up close to 80Gbps for 120fps footage. So with JPEG XS standing for Xtra Small and Xtra Speed, it’s no surprise that this new ISO standard is being leant on to help.

Tested as visually lossless to 7 or more encode generations and with latency only a few lines of video, JPEG XS works well in multi-stage live workflows. Jean-Baptiste explains that it’s low complexity and can work well on FPGAs and on CPUs.

JPEG XS can support up to 16-bit values, any chroma and any colour space. It’s been standardised to be carried in MPEG TSes, in SMPTE ST 2110 as 2110-22, over RTP (pending) within HEIF file containers and more. Worst case bitrates are 200Mbps for 1080i, 390Mbps for 1080p60 and 1.4Gbps for 2160p60.

Evolution of Standards-Based IP Workflows Ground-To-Cloud

Last in the presentations is John Mailhot from Imagine Communications and also co-chair of an activity group at the VSF working on standardising interfaces for passing media place to place. Within the data plane, it would be better to avoid vendors repeatedly writing similar drivers. Between ground and cloud, how do we standardise video arriving and the data you need around that. Similarly standardising new technologies like Amazon’s CDI is important.

John outlines the aim of having an interoperability point within the cloud above the low-level data transfer, closer to 7 than to 1 in the OSI model. This work is being done within AIMS, VSF, SMPTE and other organisations based on existing technologies.

Q&A
The video finishes with a Q&A and includes comments from AWS’s Evan Statton whose talk on CDI that evening is not part of this video. The questions cover comparing NDI with JPEG XS, how CDI uses networking to achieve high bandwidths and high reliability, the balance between minimising network and minimising CPU depending on workflow, the increasingly agile nature of broadcast infrastructure, the need for PTP in the cloud plus the pros and cons of standards versus specifications.

Watch now!
Speakers

Peter Wharton Peter Wharton
Director Corporate Strategy, TAG V.S.
President, Happy Robotz
Vice President of Membership, SMPTE
Jean-Baptiste Lorent Jean-Baptiste Lorent
Director Marketing & Sales,
intoPIX
John Mailhot John Mailhot
Co-Chair Cloud-Gounrd-Cloud-Ground Activity Group, VSF
Directory & NMOS Steering Member, AMWA
Systems Architect for IP Convergence, Imagine Communcations
Paul Briscoe Moderator: Paul Briscoe
Canadian Regional Governor, SMPTE
Consultant, Televisionary Consulting
Evan Statton Evan Statton
Principal Architect, Media & Entertainment
Amazon Web Services

Video: Maximise your video density with ST 2110

What can ST 2110 do for you? What problems can it solve? These questions and more are tackled in this video from BBright and Matrox.

Guillaume Arthuis from BBright kicks off the video by highlighting that SMPTE ST 2110 sends all media as separate streams. Called essences, all aspects of a signal are delivered separately such as metadata, audio and video. For a device which looks at subtitling, this saves having to receive a 3Gb/s stream just to get a few Kbps of data. Sending of the video has also been improved as no blanking data is sent which can see bandwidth savings of up to 30% depending on the video format. It shouldn’t be forgotten that network cables are bi-directional and typically can carry many streams. This means the number of cables in a facility can be greatly reduced.

Marwan al-Habbal from Matrox compares the pros and cons of SDI against ST 2110. SDI has incredible interoperability, has good reliability and ‘discovery’ is not really a problem since everything is point-to-point connected with uni-directional cabling. These latter two points are, of course, downsides compared to ST 2110. Marwan looks at whether we can be confident in 2110’s reliability, discovery and connectivity. Within the standard, ‘narrow’ and ‘wide’ senders are specified. Marwan makes the point that using narrow senders everywhere will give better determinism and can avoid momentary ‘blips’ in the network. Any problems on the network can be mitigated by using ST 2022-7 seamless switching whereby two feeds are sent over the network(s) and a single stream is reassembled from the received packets. Testing is the key to interoperability. JT-NM’s testing programme is, by another name, a ‘plugfest’ whereas many vendors as possible connect to other vendors’ equipment in order to test compatibility. This is leading to confidence in terms of inter-vendor workflows being generally accessible.

Another major benefit of ST 2110 is density. Guillaume takes us through calculations showing that you can implement a 512×512 router using just a 1U switch at an approximate cost of $80. He also looks at future scaling approaches. One approach outlined is to use 25G interfaces today to leave room for expansion but the other is to implement JPEG XS running ST 2110-22. This is a relatively new standard which brings in the ability to use compressed video in 2110 for the first time. This would allow ‘HD’ bitrates for low-latency UHD streams.

Watch now!
Speakers

Guillaume Arthuis Guillaume Arthuis
President,
BBright
Marwan al-Habbal Marwan Al-Habbal
OEM Product Manager.
Matrox

Video: RAVENNA AM824 & SMPTE ST 2110-31 Applications



Audio has a long heritage in IP compared to video, so there’s plenty of overlap and there are edge cases abound when working between RAVENNA, AES67 and SMPTE ST 2110-30 and -31. SMPTE’s 2110 suite of standards currently holds two methods of carrying audio including a way of carrying encoded audio such as Dolby AC4 and Dolby E.

RAVENNA Evangelist Andreas Hildebrand is joined by Dolby Labs architect James Cowdrey to discuss the compatibility of -30 and -31 with AES67 and how non-PCM data can be carried in -31 whether that be lightly compressed audio, object audio for immersive experiences or even just pure metadata.

Andreas starts by revising the key differences between AES67 and RAVENNA. The core of AES67 fits neatly within RAVENNA’s capabilities including the transport of up to 24-bit linear PCM with 48 samples per packet and up to 8 channels of 48kHz audio. RAVENNA offers more sample rates, more channels and adds discovery and redundancy with modes such as ‘MADI’ and ‘High performance’ which help constrain and select the relevant parameters.

SMPTE ST 2110-30 is based on AES67 but adds its own constraints such that any -30 stream can be received by an AES67 decoder, however, an AES67 sender needs to be aware of -30’s constraints for it to be correctly decoded by a -30 receiver. Andreas says that all AES67 senders now have this capability.


In contrast to 2110-30, 2110-31 is all about AES3 and the ability of AES3 to carry both linear PCM and non-PCM data. We look at the structure of the AES3 which contains audio blocks each of which has 192 Frames. These frames are split into 2, in the case of stereo, 64 in the case of MADI. Within each of these subframes, we finally find the preamble and the 24-bit data. Andreas explains how this is linked to AM824 and the SDP details needed.

James Cowdery leads the second part of today’s talk first talking about SMPTE ST 337 which details how to send non-PCM audio and data in an AES3 serial digital audio interface. It can carry AC-3, AC-4 for object audio delivering immersive audio experiences, Dolby E and also the metadata standards KLV and Serial ADM.

‘Why use Dolby E?’ asks James. Dolby E has a number of advantages although as bandwidth has become more available, it is increasingly replaced by uncompressed audio. However legacy workflows may now be reliant on IP infrastructure between the receiver and decoder, so it’s important to be able to carry it. Dolby E also packs a whole set of surround sound within a single data stream removing any problems of relative phase and can be carried over MPEG-2 transport streams so it still has plenty of flexibility and uses cases.

Its strength can bring fragility and one way which you can destroy a Dolby E feed is by switching between two videos containing Dolby E in the middle of the data rather than waiting for the gap between packets which is called the guardband. Dolby E needs to be aligned to the video so that you can crossfade and switch between videos without breaking the audio. James makes the point that one reason to use -31 and not -30 to carry Dolby E, or any other non-PCM data, is that -30 assumes that a sample rate converter can be used and so there is usually little control over when an SRC is brought in to use. A sample rate converter, of course, would destroy any non-PCM data.

RAVENNA 824 and 2110-31 gateways will preserver the line position of Dolby data. Can support Dolby E transport can therefore be supported by a vendor without Dolby support. James notes that your Dolby E packets need to be 125 microseconds to achieve packet-level switching without missing a guardband and corrupting data.

Immersive audio requires metadata. sADM is an open specification for metadata interchange, the aim of which is to help interoperability between vendors. sADM metadata can be embedded in SDI, transported uncompressed as SMPTE 302 in MPEG-2 Transport Streams and for 2110, is carried in -31. It’s based on XML description of metadata from the Audio Definition Model and James advises using the GZip compression mode to reduce the bitrate as it can be sent per-frame. An alternative metadata standard is SMPTE ST 336 which is an open format providing a binary payload which makes it a lower-latency method for sending Metadata. These methods of sending metadata made sense in the past, but now, with SMPTE ST 2110 having its own section for metadata essences, we see 2110-41 taking shape to allow data like this to be carried on its own.

Watch now!
Speakers

James Cowdery James Cowdery
Senior Staff Architect
Dolby Laboratories
Andreas Hildebrand Andreas Hildebrand
RAVENNA Evangelist,
ALC NetworX

Video: Case Study: Dropbox HQ ST 2110

Dropbox is embedded in many production workflows – official and otherwise – so it’s a beautiful symmetry that they’re using Broadcast’s latest technology, SMPTE ST 2110, within their own headquarters. Dropbox have AV throughout their building and a desire to create professional video from anywhere. This desire was a driving factor in an IP-based production facility as, to allow mobile production platforms to move from room to room with only a single cable needed to connect to the wall and into the production infrastructure.

David Carroll’s integration company delivered this project and joins Wes Simpson to discuss this case-study with colleague Kevin Gross. David explains that they delivered fibre to seventy locations throughout the building making most places into potential production locations.

Being an IT company at heart, the ST 2110 network was built to perform in the traditional way, but with connections into the corporate network which many broadcasters wouldn’t allow. ST 2110 works best with two separate networks, often called Red and Blue, both delivering the same video. This uses ST 2022-7 to seamlessly failover if one network loses a packet or even if it stops working all together. This is the technique used with dropbox, although there these networks are connected together so are not one hundred per cent isolated. This link, however, has the benefit of allowing PTP traffic between the two networks.

PTP topology typically sees two grandmasters in the facility. It makes sense to connect one to the red network, the other to the blue. In order to have proper redundancy, though, there should really be a path from both grandmasters to both networks. This is usually done with a specially-configured ‘PTP only’ link between the two. In this case, there are other reasons for a wider link between networks which also serves as the PTP link. Another element of PTP topology is acknowledging the need for two PTP domains. A PTP domain allows two PTP systems to operate on the same network but without interfering with one another. Dante requires PTP version 1 whereas 2110, and most other things, require v2. Although this is in the process of improving, the typical way to solve this now is to run the two separately and block v1 from areas of the network in which it’s not needed.

PTP disruptions can also happen with multicast packet loss. If packets are lost at the wrong time, a grandmaster election can happen. Finally, on PTP, they also saw the benefits of using boundary clock switches to isolate the grandmasters. These grandmasters have to send out the time eight times a second. Each end-device then replies to ascertain the propagation delay. Dealing with every single device can overwhelm grandmasters, so boundary clock switches can be very helpful. On a four-core Arista, David and Kevin found that one core would be used dealing with the PTP requests.

A more extensive write-up of the project can be found here from David Carroll

Watch now!

Speakers

Kevin Gross Kevin Gross
Media Network Consultant
AVA Networks
David Carroll David Carroll
President,
David Carroll Associates, Inc.
Wes Simpson Wes Simpson
Owner, LearnIPVideo.com