Video: Line by Line Processing of Video on IT Hardware

If the tyranny of frame buffers is let to continue, line-latency I/O is rendered impossible without increasing frame-rate to 60fps or, preferably, beyond. In SDI, hardware was able to process video line-by-line. Now, with uncompressed SDI, is the same possible with IT hardware?

Kieran Kunhya from Open Broadcast Systems explains how he has been able to develop line-latency video I/O with SMPTE 2110, how he’s coupled that with low-latency AVC and HEVC encoding and the challenges his company has had to overcome.

The commercial drivers are fairly well known for reducing the latency. Firstly, for standard 1080i50, typically treated as 25fps, if you have a single frame buffer, you are treated to a 40ms delay. If you need multiple buffers for a workflow, this soon stacks up so whatever the latency of your codec – uncompressed or JPEG XS, for example – the latency will be far above it. In today’s covid world, companies are looking for cutting the latency so people can work remotely. This has only intensified the interest that was already there for the purposes of remote production (REMIs) in having low-latency feeds. In the Covid world, low latency allows full engagement in conversations which is vital for news anchors to conduct interviews as well as they would in person.

IP, itself, has come into its own during recent times where there has been no-one around to move an SDI cable, being able to log in and scale up, or down, SMPTE ST 2110 infrastructure remotely is a major benefit. IT equipment has been shown to be fairly resilient to supply chain disruption during the pandemic, says Kieran, due to the industry being larger and being used to scaling up.

Kieran’s approach to receiving ST 2110 deals in chunks of 5 to 10 lines. This gives you time to process the last few lines whilst you are waiting for the next to arrive. This processing can be de-encapsulation, processing the pixel values to translate to another format or to modify the values to key on graphics.

As the world is focussed on delivering in and out of unusual and residential places, low-bitrate is the name of the game. So Kieran looks at low-latency HEVC/AVC encoding as part of an example workflow which takes in ST 2110 video at the broadcaster and encodes to MPEG to deliver to the home. In the home, the video is likely to be decoded natively on a computer, but Kieran shows an SDI card which can be used to deliver in traditional baseband if necessary.

Kieran talks about the dos and don’ts of encoding and decoding with AVC and HEVC with low latency targetting an end-to-end budget of 100ms. The name of the game is to avoid waiting for whole frames, so refreshing the screen with I-frame information in small slices, is one way of keeping the decoder supplied with fresh information without having to take the full-frame hit of 40ms (for 1080i50). Audio is best sent uncompressed to ensure its latency is lower than that of the video.

Decoding requires carefully handling the slice boundaries, ensuring deblocking i used so there are no artefacts seen. Compressed video is often not PTP locked which does mean that delivery into most ST 2110 infrastructures requires frame synchronising and resampling audio.

Kieran foresees increasing use of 2110 to MPEG Transport Stream back to 2110 workflows during the pandemic and finishes by discussing the tradeoffs in delivering during Covid.

Watch now!

Kieran Kunhya Kieran Kunhya
CEO & Founder, Open Broadcast Systems

Video: RIST Unfiltered – Q&A Session

RIST is a protocol which allows for reliable streaming over lossy networks like the internet. Whilst many people know that much, they may not know more and may have questions. Today’s video aims to answer the most common questions. For a technical presentation of RIST, look no further than this talk and this article

Kieran Kunhya deals out the questions to the panel from the RIST Forum, RIST members and AWS. Asking:
Does RIST need 3rd party equipment?
Is there an open-source implementation of RIST?
Whether there are any RIST learning courses?
as well as why companies should use RIST over SRT.
RIST, we hear is based on RTP which is a very widely deployed technology for real-time media transport and is widely used for SMPTE 2022-2 and 6 streams, SMPTE 2110, AES67 and other audio protocols. So not only is it proven, but it’s also based on RFCs along with much of RIST. SRT, the panel says, is based on the UDT file transfer protocol which is not an RFC and wasn’t designed for live media transport although SRT does perform very well for live media.

“Why are there so many competitors in RIST?” is another common question which is answered by talking about the need for interoperability. Fostering widespread interoperability will grow the market for these products much more than it would with many smaller protocols. “What new traction is RIST getting?” is answered by David Griggs from AWS who says they are committed to the protocol and find that customers like the openness of the protocol and are thus willing to invest their time in creating workflows based on it. Adi Rozenberg lists many examples of customers who are using the technology today. You can hear David Griggs explain RIST from his perspective in this talk.

Other questions handled are the licence that RIST is available under and the open-source implementations, the latency involved in using RIST and whether it can carry NDI. Sergio explains that NDI is a TCP-based protocol so you can transmit it by extracting UDP out of it, using multicast or using a VizRT-tool for extracting the media without recompressing. Finally, the panel looks at how to join the RIST Activity Group in the VSF and the RIST Forum. They talk about the origin of RIST being in an open request to the industry from ESPN and what is coming in the upcoming Advanced Profile.

Watch now!

Rick Ackermans Rick Ackermans
RIST AG Chair,
Director of RF & Transmission Engineering, CBS Television
David Griggs David Griggs
Senior Product Manager, Media Services,
AWS Elemental
Sergio Ammirata Sergio Ammirata
RIST AG Member,
Chief Science Officer, SipRadius
Adi Rozenberg Adi Rozenberg
RIST Forum Director
AG Member, Co-Founder & CTO, VideoFlow
Ciro Noronha Ciro Noronha
RIST Forum President and AG Member
EVP of Engineering, Cobalt Digital
Paul Atwell Paul Atwell
RIST Forum Director,
President, Media Transport Solutions
Wes Simpson Wes Simpson
RIST AG Co-Chair,
President & Founder,
Kieran Kunhya Kieran Kunhya
RIST Forum Director
Founder & CEO, Open Broadcast Systems

Video: A video transport protocol for content that matters

What is RIST and why’s it useful? The Reliable Internet Stream Protocol was seeing as strong uptake by broadcasters and other users wanting to use the internet to get their video from A to B over the internet even before the pandemic hit.

Kieran Kunhya from Open Broadcast Systems explains what RIST is trying to do. It comes from a history of expensive links between businesses, with fixed lines or satellite and recognises the increased use of cloud. With cloud computing increasingly forming a key part of many companies’ workflows, media needs to be sent over the internet to get into the workflow. Cloud technology, he explains, allows broadcasters to get away from the traditional on-prem model where systems need to be created to handle peak workload meaning there could be a lot of underutilised equipment.

Whilst the inclination to use the internet seems only too natural given this backdrop, RIST exists to fix the problems that the internet brings with it. It’s not controversial to say that it loses packets and adds jitter to signals. On top of that, using common file transfer technologies like HTTP on TCP leaves you susceptible to drops and variable latency. For broadcasters, it’s also important to know what your latency will be, and know it won’t change. This isn’t something that typical TCP-based technologies offer. On top of solving these problems, RIST also sets out to provide an authenticated, encrypted link.

Ways of doing this have been done before, with Zixi and VideoFlow being two examples that Kieran cites. RIST was created in order to allow interoperability between equipment in a vendor-neutral way. To underline it’s open nature, Kieran shows a table of the IETF RFCs used as part of the protocol.

RIST has two groups of features, those in the ‘Simple Profile’ such as use of RTP, packet loss recovery, bonding and hitless switching. Whereas the ‘Main Profile’ adds on top of that tunnelling (including the ability to choose which direction you set up your connection), encryption, authentication and null packets removal. Both of these are available as published specifications today. A third group of features is being planned under the ‘enhanced profile’ to be released around the beginning of Q2 2021.

Kieran discusses real-world proof points such as a 10-month link which had lost zero packets, though had needed to correct for millions of lost packets. He discusses deployments and moves on to SRT. SRT, Secure Reliable Transport, is a very popular technology which achieves a lot of what RIST does. Although it is an open-source project, it is controlled by one vendor, Haivision. It’s easy to use and has seen very wide deployment and it has done much to educate the market so people understand why they need a protocol such as RIST and SRT so has left a thirst in the market. Kieran sees benefit in RIST having brought together a whole range of industry experts, including Haivision, to develop this protocol and that it already has multipath support, unlike SRT. Furthermore, at 15% packet loss, SRT doesn’t work effectively whereas RIST can achieve full effectiveness with 40% packet loss, as long as you have enough bandwidth for a 200% overhead.

Watch now!

Kieran Kunhya Kieran Kunhya
Director, RIST Forum
Founder & CEO, Open Broadcast Systems

Video: Demystifying Video Delivery Protocols

Let’s face it, there are a lot of streaming protocols out there both for contribution and distribution. Internet ingest in RTMP is being displaced by RIST and SRT, whilst low-latency players such as CMAF and LL-HLS are vying for position as they try to oust HLS and DASH in existing services streaming to the viewer.

This panel, hosted by Jason Thibeault from the Streaming Video Alliance, talks about all these protocols and attempts to put each in context, both in the broadcast chain and in terms of its features. Two of the main contribution technologies are RIST and SRT which are both UDP-based protocols which implement a method of recovering lost packets whereby packets which are lost are re-requested from the sender. This results in a very high resilience to packet loss – ideal for internet deployments.

First, we hear about SRT from Maxim Sharabayko. He lists some of the 350 members of the SRT Alliance, a group of companies who are delivering SRT in their products and collaborating to ensure interoperability. Maxim explains that, based on the UDT protocol, it’s able to do live streaming for contribution as well as optimised file transfer. He also explains that it’s free for commercial use and can be found on github. SRT has been featured a number of times on The Broadcast Knowledge. For a deeper dive into SRT, have a look at videos such as this one, or the ones under the SRT tag.

Next Kieran Kunhya explains that RIST was a response to an industry request to have a vendor-neutral protocol for reliable delivery over the internet or other dedicated links. Not only does vendor-neutrality help remove reticence for users or vendors to adopt the technology, but interoperability is also a key benefit. Kieran calls out hitless switching across multiple ISPs and cellular. bonding as important features of RIST. For a summary of all of RIST’s features, read this article. For videos with a deeper dive, have a look at the RIST tag here on The Broadcast Knowledge.

Demystifying Video Delivery Protocols from Streaming Video Alliance on Vimeo.

Barry Owen represents WebRTC in this webinar, though Wowza deal with many protocols in their products. WebRTC’s big advantage is sub-second delivery which is not possible with either CMAF or LL-HLS. Whilst it’s heavily used for video conferencing, for which it was invented, there are a number of companies in the streaming space using this for delivery to the user because of it’s almost instantaneous delivery speed. Whilst a perfect rendition of the video isn’t guaranteed, unlike CMAF and LL-HLS, for auctions, gambling and interactive services, latency is always king. For contribution, Barry explains, the flexibility of being able to contribute from a browser can be enough to make this a compelling technology although it does bring with it quality/profile/codec restrictions.

Josh Pressnell and Ali C Begen talk about the protocols which are for delivery to the user. Josh explains how smoothstreaming has excited to leave the ground to DASH, CMAF and HLS. They discuss the lack of a true CENC – Common Encryption – mechanism leading to duplication of assets. Similarly, the discussion moves to the fact that many streaming services have to have duplicate assets due to target device support.

Looking ahead, the panel is buoyed by the promise of QUIC. There is concern that QUIC, the Google-invented protocol for HTTP delivery over UDP, is both under standardisation proceedings in the IETF and is also being modified by Google separately and at the same time. But the prospect of a UDP-style mode and the higher efficiency seems to instil hope across all the participants of the panel.

Watch now to hear all the details!

Ali C. Begen Ali C. Begen
Technical Consultant, Comcast
Kieran Kunhya Kieran Kunhya
Founder & CEO, Open Broadcast Systems
Director, RIST Forum
Barry Owen Barry Owen
VP, Solutions Engineering
Wowza Media Systems
Joshua Pressnell Josh Pressnell
Penthera Technologies
Maxim Sharabayko Maxim Sharabayko
Senior Software Developer,
Jason Thibeault Moderator: Jason Thibeault
Executive Director,
Streaming Video Alliance