Video: Remote Production in Real Life

Remote production is in heightened demand at the moment, but the trend has been ongoing for several years. For each small advance in technology, it becomes practical for another event to go remote. Remote production solutions have to be extremely flexible as a remote production workflow for one copmany won’t work for the next This is why the move to remote has been gradual over the last decade.

In this video, Dirk Skyora from Lawo gives three examples of remote production projects stretching back as far as 2016 to the present day in this RAVENNA webinar with evangelist Andreas Hildebrand.

The first case study is remote production for Belgian second division football. Working with Belgian telco Proximus along with Videohouse & NEP, LAWO setup remote production for stadia kitted out with 6 cameras and 2 commentary positions. With only 1 gigabit connectivity to the stadiums, they opted to use JPEG 2000 encoding at 100 Mbps for both the cameras out of the stadia but also the two return feeds back in for the commentators.

The project called for two simultaneous matches feeding into an existing gallery/PCR. Deployment was swift with flightcases deployed remotely and a double set of equipment being installed into the fixed PCR. Overall latency was around 2.5 frames one-way, so the camera viewfinders were about 5 frames adrift once transport and en/decoding delay were accounted for.

The main challenges were with the MPLS network into the stadia which would spontanously reroute and be loaded with unrelated traffic at around 21:00. Although there was packet loss, none was noticable on the 100Mbps J2K feeds. Latency for the commentators was a problem so some local mixing was needed and lastly PTP wasn’t possible over the network. Timing was, therefore, derived from the return video feed into the stadium which had come from the PTP-locked gallery. Locally this incoming timing was used to lock a locally generated PTP signal.

The next case study is inter-country links for the European Council connecting the Luxembourg and Brussels buildings for the European Council. The project was to move all production to a single tech control room in Brussells and relied on two 10GbE links between the buildings going through an Arista 7280 carrying 18 videos in one direction and two in return. Although initially reluctant to compress, the Council realised after testing that VC2 which offers around 4x compression would work well and deliver no noticable latency (approx 20ms end to end). Thanks to using VC2, the 10Gig links saw low usage from the project and the Council were able to migrate other business activities onto the link. PTP was generated in Brussels and Luxembourg re-generated their PTP from the Brussels signal, to be distributed locally. Overall latency was 1 frame.

Lastly, Dirk outlines the work done for the Belgium Daily News which had been bought out by DPG Media. This buy-out prompted a move from Brussels to Antwerp where a new building opened. However, all of the techinical equipmennt was already in Brussels. This led to the decision to remote control everything in Brussels from Antwerp. The production staff moved to Antwerp, causing some issues with the disconnect between production and technical, but also due to personnel relocating and getting used to new facilities.

The two locations were connected with a 400GbE, redundant infrastructure using IP<->SDI gateways. Latency was 1 frame and, again, PTP on one site was created from the incoming PTP from the other.

The video finishes with a detailed Q&A.

Watch now!
Speakers

Dirk Sykora Dirk Sykora
Technical Sales Manager,
Lawo
Andreas Hildebrand Andreas Hildebrand
RAVENNA Evangelist,
ALC NetworX

Video: PTP in WAN Applications & Update on PTP v2.1

PTP is evolving as is our ability to use it over WANs. This video explains what’s new in PTP’s second revision and the evolving techniques of using PTP over a wide area network such as the internet. As PTP was built assuming the use of LANs, the longer and more unpredictable latency of WANs throws off the timing calculations, so what can be done to compensate?

In this video from RAVENNA, Andreas Hildebrand from ALC Networx takes us through PTP 2.1, the 2019 revision of PTP following on from PTP 2.0 in 2008 and from the original 1.0 in 2002. Famously, 2.0 and 1.0 were not compatible with each other which has caused problems with some hardware implementations of DANTE which were first written when 1.0 was the only edition. Importantly, Andreas highlights, version 2.1 is backwards compatible with version 2.0. If you’re looking for a PTP primer before digging in, have a look at this intro video from Daniel Boldt, Meinberg

Andreas explains the use of PTP profiles within both AES67 and SMTPTE 2110 which standardise some of the parameters for using PTP such as message frequency. Whilst they do have different defaults, there is an overlap between the two allowing for use of AES67 streams withing both an AES67 ecosystem and with a 2110-30 ecosystem. These overlaps are detailed in the joint AES and SMPTE document, AES-R16-2016.

“What’s new in PTP v2.1?” asks Andreas. Apart from clearer language, accuracy, flexibility and robustness have been enhanced.

Flexibility
Flexibility comes from the ability to mixed multicast and unicast messages. Announce and sync messages are still sent in multicast, but queries like delayresponse & delayrequest can now be sent unicast which provides better scalability as, for many scenarios, the reply never needs to go back to any other computers. Another example of flexibility is modular transparent clocks i.e. ones in SFPs. Another flexibility improvement is that it’s now possible to isolate profiles without using different PTP domain numbers. This is by adding a Profile ID called ‘SdoId’.

Robustness and Security
PTP now allows inter-domain interactions effectively allowing multiple GMs to vote onto a single ‘multi-domain clock’. This becomes a very robust clock as it’s created from the insight of three grandmasters. PTP v2.1 adds source integrity checking to allow identification of false, injected, messages. A long-needed improvement as security, even of a LAN, can’t be assumed nowadays.

Performance and Accuracy
Stats reporting has been added to PTP v2.1 allowing monitoring of the average, minimum, max and standard deviation of a number of metrics from the leader clock, delay metrics and message reception counters. Accuracy has been boosted to sub-nanosecond by the CERN-related White Rabbit Project which is an overall benefit to PTP even if sub-nanosecond timing isn’t needed.

Source: ALC NetworX

The second part of the video features Meinberg’s Daniel Boldt who discusses how to transmit PTP over the WAN. This is more challenging than a WAN because it’s more likely to be affected by queueing delays – particularly if the WAN in question is the internet. Queueing delays happen for a number of reasons but it all boils down to the fact the switches and routers often have to hold packets in a buffer, something that tends to happen more when there is more load. As such, this means that the delay is variable leading to varying jitter on the measurements.

Another problem often encountered is path changes where a switch happens in the network to divert the signal to a different path. Whilst this is a great way to route around problems, it does mean a sudden change in path length and therefore propagation delay. A conventional ping time may change from 100ms to 250ms in a second. This could have a big impact on the accuracy of a PTP timing signal if undetected.

Finally, the PTP timing algorithm assumes that it takes just as long, and no longer, to get the timing information from A to B as it does to get the follow-up reply from B to A. When one direction takes longer than the other, for instance when one direction is forced through a 100Mbps link rather than 1000Mbps, the PTP timing will have a constant timing error.

Source: Meinberg

Daniel explains that these issues can be mitigated by more thorough processing of the incoming packets. For instance, having a high-quality oscillator which can maintain an accurate frequency for a long time without external input is helpful. Having a local GM on GPS is another good way to avoid problems, in the cases when this is practical, where the WAN PTP becomes an ‘aide’ to timing rather than the authority. Finally, the ‘lucky packets’ technique is demonstrated.

Daniel explains that if you look at the delay of each packet incoming over, say, a two-second period, you can collect all the packets that, based on the timestamp, were lucky enough not to be delayed a lot. By discarding those very-delayed packets, the accuracy of the PTP signal becomes much higher and jitter can reduce, as we see from two case studies, by an order of magnitude.

Watch now!
Speakers

Andreas Hildebrand Andreas Hildebrand
RAVENNA Evangelist,
ALC NetworX
Daniel Boldt Daniel Boldt
Head of Software Development,
Meinberg

Video: RAVENNA AM824 & SMPTE ST 2110-31 Applications



Audio has a long heritage in IP compared to video, so there’s plenty of overlap and there are edge cases abound when working between RAVENNA, AES67 and SMPTE ST 2110-30 and -31. SMPTE’s 2110 suite of standards currently holds two methods of carrying audio including a way of carrying encoded audio such as Dolby AC4 and Dolby E.

RAVENNA Evangelist Andreas Hildebrand is joined by Dolby Labs architect James Cowdrey to discuss the compatibility of -30 and -31 with AES67 and how non-PCM data can be carried in -31 whether that be lightly compressed audio, object audio for immersive experiences or even just pure metadata.

Andreas starts by revising the key differences between AES67 and RAVENNA. The core of AES67 fits neatly within RAVENNA’s capabilities including the transport of up to 24-bit linear PCM with 48 samples per packet and up to 8 channels of 48kHz audio. RAVENNA offers more sample rates, more channels and adds discovery and redundancy with modes such as ‘MADI’ and ‘High performance’ which help constrain and select the relevant parameters.

SMPTE ST 2110-30 is based on AES67 but adds its own constraints such that any -30 stream can be received by an AES67 decoder, however, an AES67 sender needs to be aware of -30’s constraints for it to be correctly decoded by a -30 receiver. Andreas says that all AES67 senders now have this capability.


In contrast to 2110-30, 2110-31 is all about AES3 and the ability of AES3 to carry both linear PCM and non-PCM data. We look at the structure of the AES3 which contains audio blocks each of which has 192 Frames. These frames are split into 2, in the case of stereo, 64 in the case of MADI. Within each of these subframes, we finally find the preamble and the 24-bit data. Andreas explains how this is linked to AM824 and the SDP details needed.

James Cowdery leads the second part of today’s talk first talking about SMPTE ST 337 which details how to send non-PCM audio and data in an AES3 serial digital audio interface. It can carry AC-3, AC-4 for object audio delivering immersive audio experiences, Dolby E and also the metadata standards KLV and Serial ADM.

‘Why use Dolby E?’ asks James. Dolby E has a number of advantages although as bandwidth has become more available, it is increasingly replaced by uncompressed audio. However legacy workflows may now be reliant on IP infrastructure between the receiver and decoder, so it’s important to be able to carry it. Dolby E also packs a whole set of surround sound within a single data stream removing any problems of relative phase and can be carried over MPEG-2 transport streams so it still has plenty of flexibility and uses cases.

Its strength can bring fragility and one way which you can destroy a Dolby E feed is by switching between two videos containing Dolby E in the middle of the data rather than waiting for the gap between packets which is called the guardband. Dolby E needs to be aligned to the video so that you can crossfade and switch between videos without breaking the audio. James makes the point that one reason to use -31 and not -30 to carry Dolby E, or any other non-PCM data, is that -30 assumes that a sample rate converter can be used and so there is usually little control over when an SRC is brought in to use. A sample rate converter, of course, would destroy any non-PCM data.

RAVENNA 824 and 2110-31 gateways will preserver the line position of Dolby data. Can support Dolby E transport can therefore be supported by a vendor without Dolby support. James notes that your Dolby E packets need to be 125 microseconds to achieve packet-level switching without missing a guardband and corrupting data.

Immersive audio requires metadata. sADM is an open specification for metadata interchange, the aim of which is to help interoperability between vendors. sADM metadata can be embedded in SDI, transported uncompressed as SMPTE 302 in MPEG-2 Transport Streams and for 2110, is carried in -31. It’s based on XML description of metadata from the Audio Definition Model and James advises using the GZip compression mode to reduce the bitrate as it can be sent per-frame. An alternative metadata standard is SMPTE ST 336 which is an open format providing a binary payload which makes it a lower-latency method for sending Metadata. These methods of sending metadata made sense in the past, but now, with SMPTE ST 2110 having its own section for metadata essences, we see 2110-41 taking shape to allow data like this to be carried on its own.

Watch now!
Speakers

James Cowdery James Cowdery
Senior Staff Architect
Dolby Laboratories
Andreas Hildebrand Andreas Hildebrand
RAVENNA Evangelist,
ALC NetworX

Video: AES67 over WAN

Deeply embedded in the audio industry and adopted into SMPTE ST 2110, AES67 workflows surround us. Increasingly our workflows are in multiple locations so moving AES67 on the WAN and the internet is essential. If networks were always perfect, this would be easy but as that’s not the case, this RAVENNA talk examines what the problems are and how to solve them.

Andreas Hildebrand introduces the video with an examination of how the WAN, whether that’s a company’s managed wide area network or the internet at large, is different from a LAN. Typical issues are packet loss, varying latency meaning the packets arrive with jitter, lack of PTP and multicast. With this in mind, Nicolas Sturmel from Merging Technologies takes the reins to examine the solutions.

Nicolas explains the typically EBU Tech 3326 (also known as ACIP) is used for WAN contribution which specifies how a sender and receiver communicate and the codecs to be used. Although PCM is available, many codecs such as AptX are also prescribed for use. Nicolas says that ACIP is great for most applications but if you need low-latency, precise timing and PCM-quality staying AES67 may be the best policy, even over the WAN.

Having identified your AES6-over-WAN workflow, the question is how to pull it off. Nicolas looks at three methods, one is FEC whereby you are constantly sending redundant data. FEC can send up to around 25% extra data so that if any is lost, the extra information sent can be leveraged to determine the lost values and reconstruct the stream. This is can work well but requires sending this extra data constantly therefore putting up your bandwidth. It can also only deal with certain losses requiring them to be of a short duration.

Instead of FEC, you can use RIST, SRT or a similar re-transmission technology. These will actively recover any lost packets and have the benefit that you only transmit more data when you have lost data. Lastly, he mentions SMPTE ST 2022-7 which uses two paths of identical data to cover losses in any one of them. Although this is 100% extra data, the benefit is that it can deal with any type of loss including a complete path failure which neither of the others can do. It is, however possible to combine FEC or RIST with a 2022-7 workflow so you can have two levels of protection.

Timing over the WAN is not ideal as PTP loses accuracy over long-latency links and it assumes symmetry. On the internet, it’s possible to get links where the latency is longer in one direction than the other. An easy, though potentially costly, workaround for distributing PTP over the WAN is to use GPS, GLONASS or similar to synchronise grandmaster clocks at each location.

Watch now!
Speakers

Nicolas Sturmel Nicolas Sturmel
Product Manager & Senior Technologist
Merging Technologies
Andreas Hildebrand Andreas Hildebrand
RAVENNA Evangelist,
ALCNetworx