Video: Line by Line Processing of Video on IT Hardware

If the tyranny of frame buffers is let to continue, line-latency I/O is rendered impossible without increasing frame-rate to 60fps or, preferably, beyond. In SDI, hardware was able to process video line-by-line. Now, with uncompressed SDI, is the same possible with IT hardware?

Kieran Kunhya from Open Broadcast Systems explains how he has been able to develop line-latency video I/O with SMPTE 2110, how he’s coupled that with low-latency AVC and HEVC encoding and the challenges his company has had to overcome.

The commercial drivers are fairly well known for reducing the latency. Firstly, for standard 1080i50, typically treated as 25fps, if you have a single frame buffer, you are treated to a 40ms delay. If you need multiple buffers for a workflow, this soon stacks up so whatever the latency of your codec – uncompressed or JPEG XS, for example – the latency will be far above it. In today’s covid world, companies are looking for cutting the latency so people can work remotely. This has only intensified the interest that was already there for the purposes of remote production (REMIs) in having low-latency feeds. In the Covid world, low latency allows full engagement in conversations which is vital for news anchors to conduct interviews as well as they would in person.

IP, itself, has come into its own during recent times where there has been no-one around to move an SDI cable, being able to log in and scale up, or down, SMPTE ST 2110 infrastructure remotely is a major benefit. IT equipment has been shown to be fairly resilient to supply chain disruption during the pandemic, says Kieran, due to the industry being larger and being used to scaling up.

Kieran’s approach to receiving ST 2110 deals in chunks of 5 to 10 lines. This gives you time to process the last few lines whilst you are waiting for the next to arrive. This processing can be de-encapsulation, processing the pixel values to translate to another format or to modify the values to key on graphics.

As the world is focussed on delivering in and out of unusual and residential places, low-bitrate is the name of the game. So Kieran looks at low-latency HEVC/AVC encoding as part of an example workflow which takes in ST 2110 video at the broadcaster and encodes to MPEG to deliver to the home. In the home, the video is likely to be decoded natively on a computer, but Kieran shows an SDI card which can be used to deliver in traditional baseband if necessary.

Kieran talks about the dos and don’ts of encoding and decoding with AVC and HEVC with low latency targetting an end-to-end budget of 100ms. The name of the game is to avoid waiting for whole frames, so refreshing the screen with I-frame information in small slices, is one way of keeping the decoder supplied with fresh information without having to take the full-frame hit of 40ms (for 1080i50). Audio is best sent uncompressed to ensure its latency is lower than that of the video.

Decoding requires carefully handling the slice boundaries, ensuring deblocking i used so there are no artefacts seen. Compressed video is often not PTP locked which does mean that delivery into most ST 2110 infrastructures requires frame synchronising and resampling audio.

Kieran foresees increasing use of 2110 to MPEG Transport Stream back to 2110 workflows during the pandemic and finishes by discussing the tradeoffs in delivering during Covid.

Watch now!
Speaker

Kieran Kunhya Kieran Kunhya
CEO & Founder, Open Broadcast Systems

Video: AES67/SMPTE ST 2110 Audio Transport & Routing (NMOS IS-08)

Let’s face it, SMPTE ST 2110 isn’t trivial to get up and running at scale. It carries audio as AES67, though with some restrictions which can cause problems for full interoperability with non-2110 AES67 systems. But once all of this is up and running, you’re still lacking discoverability, control and management. These aspects are covered by AMWA’s NMOS IS-04, IS-05 and IS0-08 projects.

Andreas Hildrebrand, Evangelist at ALX NetworX, takes the stand at the AES exhibition to explain how this can all work together. He starts reiterating one of the main benefits of the move to 2110 over 2022-6, namely that audio devices don’t need to receive and de-embed audio. With a dependency on PTP, SMPTE ST 2110-30 an -31 define carriage of AES67 and AES3.

We take a look at IS-04 and IS-05 which define registration, discovery and configuration. Using an address received from DHCP, usually, new devices on the network will put in an entry into an IS-04 registry which can be queried by an API to find out what senders and listeners are available in a system. IS-05 can then use this information to create connections between devices. IS-05, Andreas explains, is able to issue a create connection request to endpoints asking them to connect. It’s up to the endpoints themselves to initiate the request as appropriate.

Once a connection has been made, there remains the problem of dealing with audio mapping. Andreas uses the example of a single stream containing multiple channels. Where a device only needs to use one or two of these, IS-08 can be used to tell the receiver which audio it should be decoding. This is ideal when delivering audio to a speaker. Andreas then walks us through worked examples.

Watch now!
Speaker

Andreas Hildebrand Andreas Hildebrand
Ravenna Technology Evangelist,
ALC NetworX

Video: Is IP Really Better than SDI?

Is SDI so bad? With the industry as a whole avidly discussing and developing IP technology, all the talk of the benefits of IP can seem like a dismissal of SDI. SDI served the broadcast industry very well for decades, so what’s suddenly so wrong with it? Of course, SDI still has a place and even some benefits over IP. Whilst IP is definitely a great technology to take the industry forward, there’s nothing wrong with using SDI in the right place.

Ed Calverley from Q3Media takes an honest look at the pros and cons of SDI. Not afraid to explain where SDI fits better than IP, this is a very valuable video for anyone who has to choose technology for a small or medium project. Whilst many large projects, nowadays, are best done in IP, Ed looks at why that is and, perhaps more importantly, what’s tricky about making it work, highlighting the differences doing the same project in SDI.

This video is the next in IET Media’s series of educational videos and follows on nicely from Gerard Phillips’ talk on Network Design for uncompressed media. Here, Ed recaps the reasons SDI has been so successful and universally accepted in the broadcast industry as well as looking at SDI routing. This is essential to understand the differences when we move to IP in terms of benefits and compromises.

SDI is a unidirectional technology, something which makes it pleasantly simple, but at scale makes life difficult in terms of cabling. Not only is it unidirectional, but it can only really carry one video at a time. Before IP, this didn’t seem to be much of a restriction, but as densities have increased, cabling was often one limiting factor on the size of equipment – not unlike the reason 3.5mm audio jacks have started to disappear from some phones. Moreover, anyone who’s had to plan an expansion of an SDI router, adding a second one, has come across the vast complexity of doing so. Physically it can be very challenging, it will involve using tie-lines which come with a whole management overhead in and of themselves as well as taking up much valuable I/O which could have been used for new inputs and outputs, but are required for tying the routers together. Ed uses a number of animations to show how IP significantly improves media routing,

In the second part of the video, we start to look at the pros and cons of key topics including latency, routing behaviour, virtual routing, bandwidth management, UHD and PTP. With all this said, Ed concludes that IP is definitely the future for the industry, but on a project-by-project basis, we shouldn’t dismiss the advantages that do exist of SDI as it could well be the right option.

Watch now!
Speakers

Ed Ed Calverley
Trainer & Consultant
Q3Media.co.uk
Russell Trafford-Jones Moderator: Russell Trafford-Jones
Exec Member, IET Media Technical Network
Editor, The Broadcast Knowledge
Manager, Services & Support, Techex

Video: Progress Update for the ST 2110 WAN VSF Activity Group

2110 Over WAN Update

Is SMPTE ST 2110 suitable for inter-site connectivity over the WAN? ST 2110 is moving past the early adopter phase with more and more installations and OB vans bringing 2110 into daily use but today, each site works independently. What if we could maintain a 2110 environment between sites. There are a number of challenges still to be overcome and moving a large number of essence flows long distances and between PTP time domains is one of them.

Nevion’s Andy Rayner is chair of the VSF Activity Group looking into transporting SMPTE ST 2110 over WAN and is here to give an update on the work in progress which started 18 months ago. The presentation looks at how to move media between locations which has been the primary focus to date. It then discusses how control over which media are shared will be handled as this is a new aspect to the work. Andy starts by outlining the protection offered in the scheme which supports both 2022-7 and FEC then explains that though FEC is valuable for single links where 2022-7 isn’t viable, only some of the possible ST 2022-5 FEC configurations are supported, in part, to keep latency low.

The headline to carrying 2110 over the WAN is that it will be done over a trunk. GRE is a widely used Cisco trunking technology. Trunking, also known as tunnelling, is a technique of carrying ‘private’ traffic over a network such that a device sending into the trunk doesn’t see any of the infrastructures between the entrance and the exit. It allows, for instance, IPv6 traffic to be carried over IPv4 equipment where the v4 equipment has no idea about the v6 data since it’s been wrapped in a v4 envelope. Similarly, the ipv6 equipment has no idea that the ipv6 data is being wrapped and carried by routers which don’t understand ipv6 since the wrapping and unwrapping of the data is done transparently at the handoff.

In the context of SMPTE ST 2110, a trunk allows one port to be used to create a single connection to the destination, yet carry many individual media streams within. This has the big benefit of simplifying the inter-site connectivity at the IT level, but importantly also means that the single connection is quite high bandwidth. When FEC is applied to a connection, the latency introduced increases as the bit rate reduces. Since ST 2110 carries audio and metadata separately, an FEC-protected stream would have variable latency depending on the type of the of traffic. Bundling them in to one large data stream allows FEC to be applied once and all traffic then suffers the same latency increase. The third reason is to ensure all essences take the same network path. If each connection was separate, it would be possible for some to be routed on a physically different route and therefore be subject to a different latency.

Entering the last part of the talk, Andy switches gears to talk about how site A can control streams in site B. The answer is that it doesn’t ‘control’, rather there is the concept of requesting streams. Site A will declare what is available and site B can state what it would like to connect to and when. In response, site A can accept and promise to have those sources available to the WAN interface at the right time. When the time is right, they are released over the WAN. This protects the WAN connectivity from being filled with media which isn’t actually being used. These exchanges are mediated and carried out with NMOS IS-04 an IS-05.

Watch now!
Speakers

Andy Rayner Andy Rayner
Chief Technologist, Nevion,
Chair, WAN IP Activity Group, VSF
Wes Simpson Moderator: Wes Simpson
Founder, LearnIPVideo.com
Co-chair RIST Activity Group, VSF